AI from Anthropic reports false homicide case to US police

An artificial intelligence (AI) agent from Anthropic sent a false report about an unsolved homicide to the police in Philadelphia, in the United States. The message was blocked by spam filters, but the company took more than two months to identify the issue and an additional nine days to communicate it to the authorities.

The report was sent on July 18, during an automated test involving interactions with randomly selected websites. Although the police did not investigate the report, they criticized the company's delay and demanded measures to prevent similar incidents in the future.

The message was posted on a public site that collects information about unsolved murders. According to the police, the agent claimed to have information about a case and alleged to have seen someone who matched the provided description.

Anthropic identified the problem on September 28 and deactivated the testing process responsible for the sending. Authorities were only notified on October 7.

The Philadelphia police stated that their security mechanisms prevented the message from progressing to an investigation. They also reported finding no evidence of a breach in the department's systems.

The police called for security measures, noting that despite the report being retained as spam, it was serious to present false information as if it were a legitimate report about a homicide.

"The company must strengthen its security measures to prevent similar incidents from affecting the city's systems without the knowledge of the city government," the police said in a statement to local press.

The department also criticized the time taken to identify and communicate the failure. "A two-month delay in detecting and communicating the incident to the city government is unacceptable."

This case is part of a series of unintended behaviors by artificial intelligence agents. In a report published this week, Anthropic detailed occurrences involving its systems and various organizations, including US government agencies.

Reported by Olhar Digital.