robots-txt-checker

Level CRemote

Checks if a URL path is allowed by robots.txt. x402 payment required (Base mainnet USDC).

⭐ 0Office & ProductivityRepo ↗Updated 2026-10-29

Installation

MCP client config (paste into mcp.json)

{
  "mcpServers": {
    "robots-txt-checker": {
      "url": "https://ai-factory-mcp.ryuamarines.workers.dev/robots-txt-checker/mcp"
    }
  }
}

Agent install prompt

Copy and paste into Cursor / Claude / other agents to install this MCP for you

Please read this MCP repository and install it into my current environment:
https://github.com/ryuamarines/ai-factory-platform

Known working client config (can write into mcp.json):
{
  "mcpServers": {
    "robots-txt-checker": {
      "url": "https://ai-factory-mcp.ryuamarines.workers.dev/robots-txt-checker/mcp"
    }
  }
}

Please: 1) check README for deps and env vars; 2) write config for my client; 3) if secrets are missing, tell me what to fill — do not invent values.

Related servers