ah-ai-crawlers

Given a website domain, reports which AI crawlers may fetch its home page under its robots.txt (RFC 9309): GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBot, Google-Extended, Applebot-Extended, CCBot and more, each allowed or blocked, plus Googlebot, listed sitemaps, whether /llms.txt is published and page-level noindex/nosnippet, with fixes. Static check, no browser.

Open endpoint ↗

Price per call: $0.01
Chain: Base
Method: GET
Paid calls · 30 days: 4

Call it from code

This API is run by its own provider and paid directly through x402. We list it so you can find it; we don't route or hold the payment.

import { x402Client, wrapFetchWithPayment } from "@x402/fetch";
import { registerExactEvmScheme } from "@x402/evm/exact/client";

const client = new x402Client();
registerExactEvmScheme(client, { signer: agentAccount });

const pay = wrapFetchWithPayment(fetch, client);
const res = await pay("https://x402.agentharbor.agency/ai-crawlers", { method: "GET" });
console.log(await res.json());

https://x402.agentharbor.agency/ai-crawlers