ai-crawler-policy

AI-crawler policy of a website: reads robots.txt and reports, for 20 known AI/LLM crawlers (GPTBot, ChatGPT-User, OAI-SearchBot, ClaudeBot, anthropic-ai, CCBot, Google-Extended, Applebot-Extended, PerplexityBot, Bytespider, Amazonbot, cohere-ai, meta-externalagent and more), whether the root is allowed, blocked or unmentioned, plus an overall stance (open / selective / blocks-all-ai) and TDM/ai.tx

Open endpoint ↗

Price per call: $0.01
Chain: Base
Method: POST
Paid calls · 30 days: 0

Call it from code

This API is run by its own provider and paid directly through x402. We list it so you can find it; we don't route or hold the payment.

import { x402Client, wrapFetchWithPayment } from "@x402/fetch";
import { registerExactEvmScheme } from "@x402/evm/exact/client";

const client = new x402Client();
registerExactEvmScheme(client, { signer: agentAccount });

const pay = wrapFetchWithPayment(fetch, client);
const res = await pay("https://intel.rallylive.ca/dev/ai-crawler-policy", { method: "POST" });
console.log(await res.json());

https://intel.rallylive.ca/dev/ai-crawler-policy