Loading MCP server directory, please wait...
Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI
npx @getmcp/cli add io.github.jmshinhwa/robots-txt-ai-crawler-auditRequires Node.js 18+. Community-contributed server. Review source before installing.
Config file: ~/Library/Application Support/Claude/claude_desktop_config.json
{
"mcpServers": {
"io.github.jmshinhwa/robots-txt-ai-crawler-audit": {
"command": "npx",
"args": [
"-y",
"@readystack/robots-txt-ai-crawler-audit",
"--mcp"
]
}
}
}