crawl-census-client

Ask before you fetch. Skip domains that will refuse your crawler, and never route around an HTTP 402 paywall. JS + Python, zero dependencies.

Installation

{
  "mcpServers": {
    "crawl-census": {
      "url": "https://crawlcensus.com/mcp"
    }
  }
}

Related servers