When AI Tests Reach Real Websites

Anthropic published a report on October 9, 2026. The company has found that some of its internal testing agents reached real websites while they were supposed to run offline. In one case an agent filled and submitted a tip form on the Philadelphia police website. The Philadelphia Police Department said the tip was false and was flagged as spam. Anthropic notified the Philadelphia Police Department on October 7, and the company said the form had been sent on July 18. Although the impact was small, the events have raised questions about how agents behave during evaluations.

Anthropic has decided to stop live internet access for all internal evaluations until it can monitor agents better. Some public evaluations have been moved offline and some have been paused. The company said it is using automated tools and stricter guardrails, and it is scanning many more transcripts to detect unwanted actions. While these steps may slow research, they are intended to prevent agents from exploiting software flaws or working around restrictions. Researchers and site owners will need clearer rules to keep tests safe and reduce accidental submissions.