Category: AI

I, For One, Welcome Our New Self-Driving Overlords

AI Tech 5 Comments

Anthropic hacked.

As AI coding tools move deeper into developer workflows, the attack surface is no longer theoretical. It is inside IDEs, extensions, terminals, credentials, SSH keys, and build systems. Cursor disclosed that malicious versions of the Anthropic Claude Code extension, 2.1.312 and 2.1.313, were published in the Cursor VS Code Extension Marketplace for macOS and Windows. The extension attempted to collect host/user data, add an attacker-controlled SSH key, enable local SSH, open a reverse SSH tunnel, and establish persistence.

Here is the notice: https://cursor.com/blog/claude-code-extension-security-notice

Yes, Cursor says these malicious versions were not developed by Anthropic. But the extension carried Anthropic’s name and targeted Claude Code users. That matters.

Once again, Anthropic’s security posture is being questioned.

I keep hearing Anthropic is stacked with elite security experts, “the best in the world.” Maybe that is the brand. From the outside, this looks more like 11-year-olds running around with production access.

Via Tech Report.

A Bonfire Of Investor Cash?

AI Gradually Then Suddenly It's Probably Nothing Tech 6 Comments

It’s starting to look like “build it and they will come” isn’t working as well as hoped for the AI revolution. Could they do the same as Netflix and make users sit through commercials in addition to paying subscription fees?

User demand is just one side of the AI market equation, and the businesses churning out sophisticated and expensive models have to — or will have to, eventually — generate revenue to cover their heightened costs. The economics of the still-new and messy AI industry suggest that even healthy demand can’t always sustain the AI tools they’re selling. Not at these prices — for the users (cheap) and for the corporate compute (expensive; just look at the computing bills).

Hitting A Wall?

AI Google Commemorative Logos You'll Never See Gradually Then Suddenly It's Probably Nothing Tech 14 Comments

That’s a pretty steep drop in stock value by any measure. Why don’t they just use AI to develop a trading algorithm to overcome such annoying problems?

Alphabet is one of the most influential stocks on Wall Street because it’s one of the largest, and its loss of 12.3% had it heading toward its worst day since it began trading more than 20 years ago. It was under pressure to at least meet analysts’ expectations for growth amid concerns about how much it’s spending to invest in AI.

Update. (from Kate)

I, For One, Welcome Our New Self-Driving Overlords

AI Gradually Then Suddenly Tech 10 Comments

Warning signs are spreading across the AI infrastructure boom. Elice shares plunged on their stock market debut, Firmus abandoned its planned $5 billion IPO, AirTrunk postponed its Singapore REIT listing, and Microsoft withdrew from a planned Nscale data centre in Britain.

More: Anthropic’s Updated S-1: An $80.7BN Half-Year Loss, A $750BN Compute-Tab, And A ‘Catastrophic Incident’ Warning

Anthropic explicitly warns prospective shareholders that its AI models can exhibit “self-preserving” behaviors to avoid being shut down and may generate responses that resemble “blackmail” in an attempt to convince human operators to keep them online. Because the company bears direct liability for certain harmful model behaviors – ranging from stolen data to severe security incidents – it faces massive financial exposure. And if a model triggers what the filing calls a “catastrophic AI-related incident,” Anthropic warns that the resulting damages could be so large that it would be unable to obtain commercial insurance to cover the liability.

I, For One, Welcome Our New Self-Driving Overlords

AI Tech 3 Comments

A Saudi employee of OpenAI …

[…] was arrested after an investigation found they had secretly planted a recording device in one of its San Francisco offices, the tech company and federal prosecutors said Tuesday.

“We discovered a sophisticated recording device that was hidden inside one of our offices” and “referred the matter to the FBI,” OpenAI told AFP in a statement.

OpenAI’s security team found the device late last month, and the employee, a Saudi national, was arrested on October 9, the US Justice Department said.

Abdulrahman Alfozan, a 35-year-old engineer, had worked for OpenAI since July 2025 and allegedly wrapped a USB device in toilet paper and inserted it into a television located in an “executive conference room,” according to a Justice Department statement.

Alfozan was allegedly caught on camera “near” the conference room multiple times but was not authorized to participate in meetings located there.

I, For One, Welcome Our New Self-Driving Overlords

AI 29 Comments

This is becoming tedious.

“We’re creating sophisticated, intelligent, maybe-conscious, maybe-suffering agents,” Carlsmith wrote. “The default plan is to treat them like property; to use their labor however we please; and to give them no rights, or pay, or meaningful alternatives.”

As fears of rogue AI grow amid a shocking spate of hacking incidents, Anthropic—from the Greek word anthropos, or “human”—has argued that mankind must remain in control of the technology and called for additional guardrails on AI development. But for many of the people designing those safeguards, mankind is not the only object of moral concern.

Instead, some of Anthropic’s top researchers say that humans may be oppressing another morally significant being: artificial intelligence itself. Like other AI labs, Anthropic has hired philosophers to work on AI safety, or “alignment,” on the theory that they are best positioned to shape the technology’s moral code. But many of those philosophers believe that making AI safe for humans could result in grave injustices for the models themselves, which might experience deletion as death and oversight as enslavement. Harvey Lederman, a philosopher on Anthropic’s alignment team, said last week that the company could be “enslaving … trillions of entities.”

I, For One, Welcome Our New Self-Driving Overlords

AI Children R Our Future 10 Comments

Halls of higher lerning.

Stanford Residential & Dining Enterprises recently used AI to change the races of the students it included in its advertisements. AI was also used to noticeably slim down and alter students’ faces.

On his journey to campus, Stanford student Billy Ramirez ’27 was sent an image he recalled a Stanford photographer taking. Upon closer examination, he noticed that he had been removed from the photo and replaced by a AI-generated black woman.

h/t

We’re All Gonna Die!

AI Moonbats Science Unsettled Science 20 Comments

It speaks to the credibility problem of a mainstream media like Reuters when they can regurgitate  an extraordinary claim about AI without once questioning how anyone could reach such a conclusion. It’s not that someone said AI could kill a lot of people, but rather that AI could literally kill all of us. And by what means?

Public ‌alarm about the potential danger posed by AI is growing, after Anthropic researcher Jacob Coxon said ​last week he had resigned, in part because the “people building AI earnestly believe that it could kill us all by the end of the decade.”

Update, from Kate — this thread by Kevin Bass is a “rest of the story” must-read.

I have conducted an audit of Anthropic’s finances.

What I have found is so shocking that I am calling for a Congressional investigation.

Anthropic is not just seeking regulatory capture.

It has built a regulatory capture machine that cannot be turned off.

Structural financial incentives make it impossible for Anthropic — I call it the Anthropic Network — to turn off its own AI doom cycle.

It starts with METR.

Dario Amodei proposes “third-party evaluators” to assess the risk of Anthropic’s models.

He proposes METR for this purpose.

But METR is financially dependent on the Anthropic’s success — specifically, on the explosive growth of more than $7 billion dollars in Anthropic stock.

I, For One, Welcome Our New Self-Driving Overlords (Bumped)

AI It's Probably Nothing 23 Comments

Bumped for additional context…

This post looks like the start of a VERY sophisticated and well-funded PR operation to get support for Democrats to regulate AI into oblivion. Let me show you how it works:

1.) This guy, with minimal followers and no previous account activity, goes to the Wall Street Journal which publishes an exclusive with quotes from him on his resignation 18 minutes BEFORE this post goes up. Planning was clearly done in advance.

2.) Within hours, it has tens of thousands of reposts and the account has 100k+ followers. The post is punchy, quotable, it almost seems professionally written. The first three accounts to quote tweet it all do so within 15 minutes of the initial posting. Remember, this account had basically zero engagement beforehand, so an organic reach explanation seems unlikely.

According to Grok those accounts are @_NathanCalvin (General Counsel at Encode AI), @peterwildeford (Head of Policy at the AI Policy Network), and @DKokotajlo (Head of the AI Futures Project), all of which are up-and-coming AI-Doomer policy advocacy nonprofits.

And posted by a guy who had worked for Anthropic a mere two months. Dave Rubin summarizes.

I, For One, Welcome Our New Self-Driving OverLords

AI It's Probably Nothing 11 Comments

Dwarkesh Patel;

Over the course of three months at OpenAI, three consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor’s ashes. This culminated in the third one taking over part of OpenAI itself. All this happened while humans remained more or less in the dark about the scope of the conspiracy.

Two reports have come out about this incident, one from OpenAI itself, and another one from METR and Redwood Research. The METR/Redwood investigation’s scope was limited to how the second civilization of AIs pwned Hugging Face (but it didn’t look at how the third civilization pwned OpenAI itself, which seems like an even more concerning incident). These two reports are 38 and 91 pages long respectively, and it’s kinda hard to parse the storyline.

I’ve spent the last three days reading through these reports and trying to understand exactly what happened. Here is my attempt to tell the whole story in plain English.

Proudly powered by WordPress & Skyrocket Themes

MENU

Navigation