Anthropic, the California firm behind Claude AI, has updated its usage rules to prevent the chatbot from being used to create or acquire weapons.
The modified policy bans any attempt to make explosives or biological, chemical, nuclear, or radiological weapons using the AI tool. It also blocks hacking, malware creation, or denial-of-service attacks.
Election-related misuse is now off-limits too: Claude cannot be used to spread deception, target voters, or interfere with campaigns.
Strict safety measures, such as having a human review the AI’s work, apply only to products for regular users and not to tools sold for enterprise users. Anthropic said it treats its policy as a “living document” meant to adapt as AI risks evolve.
Founded in 2021 by former OpenAI researchers, the company develops AI systems that understand and generate natural language, similar to ChatGPT.
Its flagship product, Claude AI, can answer questions, help with writing, coding, and research, and — through special agentic tools — perform tasks directly on a user’s computer.
Extra Safety Features
The new rules follow May’s launch of the latest Claude model, which added tougher safeguards to make the chatbot harder to trick and safer to use.
Anthropic flagged extra risks from tools like Claude Code and Computer Use, which let the AI perform tasks on a customer’s device.
“Dangerous capability evaluations of AI models are inherently challenging, and as models approach our thresholds of concern, it takes longer to determine their status,” the firm wrote.
“Proactively enabling a higher standard of safety and security simplifies model releases while allowing us to learn from experience by iteratively improving our defenses and reducing their impact on users.”
Focusing on Top Investors
Anthropic is currently raising $5 billion at a $170 billion valuation to grow its business, Business Insider reported.
Instead of using special purpose vehicles that pool outside capital, the company is working directly with leading venture firms, giving it more control over long-term investors.
Menlo Ventures, a longtime partner also headquartered in California, is contributing $120 million in the round.
