x402 Bazaar Logox402 Bazaar
← Back to discover
httpcdpultravioletadaothirdweb

PennyAPI Robots Permission Check

Check robots.txt before crawling or scraping a page. Decide whether GPTBot, ClaudeBot, Googlebot, or another crawler may fetch the exact URL and return ALLOW/DENY, winning rule, crawl-delay, sitemap URLs, and live provenance.

Resource URL

https://pennyregwatch.com/v1/robots

Payment options

exactBase (8453)

$0.001 USDC

Max amount

Pay to
0x5deaebaf98034cc0d9abe3757a440b121c42ef0a
Timeout
300s

Tags

robots-txtcrawlerscrapingcrawl-permissionagent-utility

For agents

Prompt snippet for AI agents

Find and pay for this x402 resource:
URL: https://pennyregwatch.com/v1/robots
Type: http
Sources: cdp, ultravioletadao, thirdweb

Developer