Robots TXT Deep

Paid deep robots.txt audit — per-UA directive counts, sitemap, 34 AI-bot directive detection (GPTBot/ClaudeBot/Google-Extended/CCBot/PerplexityBot/Meta-ExternalAgent/Bytespider/etc), crawl-delay, anti-pattern flags, robots_score 0-100 A-F grade via x402

Open endpoint ↗

Price per call: $0.000
Chain: Base
Method: GET
Paid calls · 30 days: 0

Call it from code

This API is run by its own provider and paid directly through x402. We list it so you can find it; we don't route or hold the payment.

import { x402Client, wrapFetchWithPayment } from "@x402/fetch";
import { registerExactEvmScheme } from "@x402/evm/exact/client";

const client = new x402Client();
registerExactEvmScheme(client, { signer: agentAccount });

const pay = wrapFetchWithPayment(fetch, client);
const res = await pay("https://tfgmq-38-190-144-122.run.pinggy-free.link/api/robots-txt-deep", { method: "GET" });
console.log(await res.json());

https://tfgmq-38-190-144-122.run.pinggy-free.link/api/robots-txt-deep