0

gopher.conman.org · 3d · discuss

A month ago, slugmax wrote a post about LLMs invading gopher via an MCP server and now that MCP server supports the /robots.txt standard. He gives a sample file that should work for Gopher: User-agent: gopher-mcp Disallow: / User-agent: GPTBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: OAI-SearchBot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: anthropic-ai Disallow: / User-agent: Claude-Web Disallow: / User-agent: Claude-User Disallow: / User-agent: Google-Extended Disallow: / There are problems with this advice. First off, Gopher doesn't support additional information per a request, unlike HTTP. There is no way a Gopher client can identify itself to a...