Website Crawler to Markdown
Website crawler: read a whole website or one section of it (for example every page under /docs) as clean Markdown: starts at a URL, follows same-site links breadth-first and returns up to 20 pages with title and description. Use for documentation, product catalogs or company sites. For one page use web-scrape, for a known list of URLs web-read-batch (up to 5) or web-scrape-batch (up to 20). Charge
- Price per call
- $0.005
- Chain
- Base
- Method
- POST
- Paid calls · 30 days
- 0
Call it from code
This API is run by its own provider and paid directly through x402. We list it so you can find it; we don't route or hold the payment.
import { x402Client, wrapFetchWithPayment } from "@x402/fetch";
import { registerExactEvmScheme } from "@x402/evm/exact/client";
const client = new x402Client();
registerExactEvmScheme(client, { signer: agentAccount });
const pay = wrapFetchWithPayment(fetch, client);
const res = await pay("https://agentsvc.io/api/v1/proxy/site-crawl", { method: "POST" });
console.log(await res.json());https://agentsvc.io/api/v1/proxy/site-crawl