webcrawl-anything

Give your AI agent the whole web. Free, local, no API key. Search, scrape, map, crawl and monitor any site, PDF, YouTube video or CVE from Claude Code, Cursor, Codex or the CLI.

Installation

MCP client config (paste into mcp.json)

{
  "mcpServers": {
    "webcrawl": {
      "url": "https://webcrawl.app/mcp"
    }
  }
}

Agent install prompt

Copy and paste into Cursor / Claude / other agents to install this MCP for you

Please read this MCP repository and install it into my current environment:
https://github.com/brainfish-ai/webcrawl-anything

Known working client config (can write into mcp.json):
{
  "mcpServers": {
    "webcrawl": {
      "url": "https://webcrawl.app/mcp"
    }
  }
}

Please: 1) check README for deps and env vars; 2) write config for my client; 3) if secrets are missing, tell me what to fill — do not invent values.

GitHub

Related servers