ai-crawler-robots-monitor

MCP serverWeb & browsing

Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.

Serves today. ahel can serve it to every connected agent through your gateway link.

Serve it through your gateway

One link, every agent. Your own credentials, stored once.

github.com/unitedideas/actablesite-mcp

More in Web & browsing

firecrawl-mcp-server
MCP server · firecrawl playwright-mcp
MCP server · microsoft apify-mcp-server
MCP server · apify exa
MCP server · exa-labs nimrod
MCP server · mixelpixx Web Search
MCP server