← Back to discover
httppayai
Robots TXT Deep
Paid deep robots.txt audit — per-UA directive counts, sitemap, 34 AI-bot directive detection (GPTBot/ClaudeBot/Google-Extended/CCBot/PerplexityBot/Meta-ExternalAgent/Bytespider/etc), crawl-delay, anti-pattern flags, robots_score 0-100 A-F grade via x402
Resource URL
https://6xrkjcfa3yrzvefcpvanurzou4.srv.us/api/robots-txt-deep
Payment options
exactBase (8453)
$0 USDC
Max amount
- Pay to
- 0xCa0a6c6Aa7A8F0D5893636CF166Ea2b44fb6500c
- Timeout
- 60s
Tags
robotsapiurl-metadatax402sitemap
For agents
Prompt snippet for AI agents
Find and pay for this x402 resource: URL: https://6xrkjcfa3yrzvefcpvanurzou4.srv.us/api/robots-txt-deep Type: http Sources: payai