x402 Bazaar Logox402 Bazaar
← Back to discover
httpthirdwebcdppayai

Tanod sitepeek

sitepeek (Tanod) robots.txt check: whether a crawler may fetch a URL under RFC 9309, with the matching rule, crawl delay and sitemaps. Read-only. Use when an agent must know whether a crawler may fetch a URL under the site's robots.txt before scraping it.

Resource URL

https://tanod.dev/v1/robots

Payment options

exactSolana

$0.001 USDC

Max amount

Pay to
RMk9k3VC8dr4hhjgFh88rGhjAbzfBzHxJJ5n2fTcqus
Timeout
300s

Tags

sitepeekrendermarkdownpdfocrrobots

For agents

Prompt snippet for AI agents

Find and pay for this x402 resource:
URL: https://tanod.dev/v1/robots
Type: http
Sources: thirdweb, cdp, payai

Developer