x402 Bazaar Logox402 Bazaar
← Back to discover
httpcdppayaiultravioletadao

api.x402node.dev

Fetch and parse robots.txt for a domain. Returns parsed allow/disallow rules per user-agent, sitemap URLs, crawl-delay directives, and raw content. Built for AI agents planning ethical web crawling and SEO audit. Use ?domain=example.com. Accepts payment on Base or Solana — either network works.

Resource URL

https://api.x402node.dev/web/robots

Payment options

exactBase (8453)

$0.005 USDC

Max amount

Pay to
0x4466d4A84b7c49a6A094ec6eef4a0712D6dd125e
Timeout
300s
exactSolana

$0.005 USDC

Max amount

Pay to
9urRvUx69HqAopg1JGV9P9wR7JdKoztzdkuNyJ8Eo1up
Timeout
300s

For agents

Prompt snippet for AI agents

Find and pay for this x402 resource:
URL: https://api.x402node.dev/web/robots
Type: http
Sources: cdp, payai, ultravioletadao

Developer