AI Safety Concerns Mount as Industry Races Forward With New Models and Funding

This week's departure of Anthropic researcher Jacob Coxon, who cited worries that artificial intelligence laboratories are "gambling with our lives," has revived long-standing tensions over whether AI development is steering toward outcomes that could threaten human survival.

Coxon is far from isolated in his concerns. Anthropic's alignment science lead has acknowledged the possibility that AI carries more than a 10% probability of "kill all humans" within the next decade. Meanwhile, OpenAI's chief scientist has advocated for slowing AI research. CEO Sam Altman has reportedly informed staff that the company is weighing whether to decelerate cutting-edge AI development and is encouraging competitors to follow suit—provided antitrust regulators approve.

Whether these safety discussions will produce tangible shifts remains uncertain. Yet the contrast is stark: even as these conversations unfold, the sector continues churning out fresh models, agents and applications from Meta, OpenAI (twice over), China's DeepSeek and others. While the reasoning behind this acceleration is understandable, and skepticism toward AI catastrophe scenarios remains warranted given the absence of concrete explanations for why AI cannot be managed, a fundamental tension persists.

One potential brake on expansion is mounting community resistance to data centers. A recent investigation examines precisely why AI demands such substantial computational resources and why near-term solutions remain elusive. Major technology firms face considerable challenges in convincing local communities to permit the infrastructure required for the next phase of technological advancement.

European AI development shows no signs of slowing. Mistral AI secured nearly $3.5 billion in funding this week, while Paris-based Arlequin AI obtained $28 million to investigate a fundamentally different approach to AI architecture.

Financial and Corporate Developments

Oracle appears to be capitalizing on AI momentum, with cloud infrastructure revenue surpassing double its prior level. The company's stock climbed 4% in after-hours trading following a decline during regular hours on a challenging market day. Adobe delivered solid quarterly results, though a conservative forward guidance prompted a slight share decline in extended trading Thursday.

Cognition AI, the autonomous software engineering firm, completed a Series E funding round valued at $32 billion, bringing the company's valuation to $48 billion.

Positron AI secured $875 million in Series C funding to advance its solution for addressing memory constraints in AI systems. The company's appliance employs standard RAM rather than expensive and limited high-bandwidth memory. Backing from established technology veterans including Silicon Graphics' Jim Clark, Forest Baskett and Tom Jermoluk suggests the venture warrants close attention.

Apple unveiled new iPhone models, including a premium foldable variant, though the announcements generated limited enthusiasm in some quarters.

Key Insights and Analysis

Gartner projects that by 2029, organizations will need to rehire approximately 30% of workers displaced by AI, frequently at substantially higher compensation levels.

Notable Personnel Movements

• AI researcher Andrew Tulloch is transitioning from Meta to Anthropic
• WordPress co-founder Matt Mullenweg's status at Automattic remains unclear following recent developments
• Cybersecurity specialist Kevin Mandia is joining Amazon's board of directors

Upcoming Industry Events

• Dreamforce (Salesforce's annual conference) in San Francisco, September 15-17
• AI Infra Summit in Santa Clara, September 15-17
• Workiva Amplify in Las Vegas, September 15-16
• AGNTCon + MCPCon in Amsterdam, September 17-18

Additional Funding and Acquisitions

• Forus raised $150 million in Series C at a $3 billion valuation for prescription workflow automation
• Bending Spoons is acquiring digital planning platform Miro for $1.36 billion
• Analog Devices is acquiring Alif Semiconductor for $1.35 billion to add AI-native processing capabilities
• Anthropic investor is leading $120 million funding for off-grid AI power startup TAR
• Cylake closed a $245 million funding round ahead of beta launch for its next-generation cybersecurity offering
• Elon Musk's The Boring Company raised $3 billion at a $23 billion valuation, with the UAE leading the investment