crawlsight
Level CLocalCheck a site's AI visibility: crawler access, llms.txt generate and validate, robots.txt lint.
Installation
MCP client config (paste into mcp.json)
{
"mcpServers": {
"crawlsight": {
"command": "npx",
"args": [
"@reneturcios/crawlsight"
]
}
}
}Agent install prompt
Copy and paste into Cursor / Claude / other agents to install this MCP for you
Please read this MCP repository and install it into my current environment:
https://github.com/CloudCorpRecords/crawlsight
Known working client config (can write into mcp.json):
{
"mcpServers": {
"crawlsight": {
"command": "npx",
"args": [
"@reneturcios/crawlsight"
]
}
}
}
Please: 1) check README for deps and env vars; 2) write config for my client; 3) if secrets are missing, tell me what to fill — do not invent values.
Related servers
scholardataGoogle Scholar as JSON: scholarly search results, case-law opinions and citationa11yscanWCAG accessibility audits for URLs and HTML: scans, stored reports and the fullScrapling🕷️ The BEST Web Scraping framework that handles everything from a single requestChrome DevTools MCP谷歌官方:让 AI 操控 Chrome——页面操作、Lighthouse 性能分析、网络监控、控制台日志UI-TARS-desktopThe Open-Source Multimodal AI Agent Stack: Connecting Cutting-Edge AI Models andPlaywright MCP微软出品的浏览器自动化工具,用结构化快照操控网页:点击、填表、截图、抓取,无需视觉模型