x402 Bazaar Logox402 Bazaar
← Back to discover
httpthirdwebcdp

x402.agentharbor.agency

Given a website domain, reports which AI crawlers may fetch its home page under its robots.txt (RFC 9309): GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBot, Google-Extended, Applebot-Extended, CCBot and more, each allowed or blocked, plus Googlebot, listed sitemaps, whether /llms.txt is published and page-level noindex/nosnippet, with fixes. Static check, no browser. Not charged if robots.txt cannot be read (no response, or a bot block).

Resource URL

https://x402.agentharbor.agency/ai-crawlers

Payment options

exactBase (8453)

$0.01 USDC

Max amount

Pay to
0x840C829530DF7B4CCF700Fa3A0060cF2C80b014e
Timeout
300s

Tags

ai-crawlersrobots-txtllms-txtgptbotseo

For agents

Prompt snippet for AI agents

Find and pay for this x402 resource:
URL: https://x402.agentharbor.agency/ai-crawlers
Type: http
Sources: thirdweb, cdp

Developer