Via Instapundit; OpenAI’s financials leaked, revealing $21 billion in losses against $13 billion in revenue.

“Is there a path to profitability? Maybe. OpenAI’s two biggest expenses are R&D and marketing. Budget cuts there, coupled with an ability to raise prices or win new sources of revenue, could see the company move into the black over time. Cutting R&D would be the most difficult part of that, given that AI companies can only hold onto their customers by generating the best-performing models.”

Ed Zitron was the leakee., with followup commentary here.

You know, people like to talk about the costs to taxpayers from abandoned wells. Nobody’s talking about the costs of abandoned data centers.

From a shimmering spectacle to a high-tech rainstorm: It was clear that something had gone seriously wrong with the thousand-strong swarm of drones twinkling above Darling Harbour during the Vivid Sydney festival last month.

*

@BrianRoemmele

The amazing @DavidSacks warned us a long time ago about Anthropic’s AI Safety Theater and regulatory capture.

He saw this early and took arrows for it.

So Anthropic got its way…

And now the check is being cashed on Anthropic.

Update: David Sacks explains.

Those trying to misdirect and tie this action to the prior DoW/Anthropic issues are wrong. The Admin values Anthropic’s technical capabilities and feels that this issue, while serious, should be easily resolved. The ball is in Anthropic’s court.

@heynavtoor

You have noticed it. ChatGPT feels dumber than it used to. Your prompts that worked six months ago produce worse results now. The writing sounds flatter. The ideas sound safer. The internet itself feels like it is shrinking. Every article reads the same. Every email sounds the same. Every answer sounds like it was written by the same voice.

You thought it was you. It is not you.

Researchers at Oxford and Cambridge published a paper in Nature proving what is happening. They call it Model Collapse.

Here is the mechanism in one sentence. AI trained on AI-generated data gets dumber every generation until it forgets what real human data looked like.

The internet is filling with AI-generated content. Blog posts. Articles. Reviews. Comments. Social media. AI companies scrape the internet to train the next generation of models. Which means the next generation of AI is being trained on the output of the current generation.

Each cycle loses information. Not randomly. It loses the rarest, most unusual, most creative parts first. The researchers call these the “tails of the distribution.” The weird ideas. The unexpected perspectives. The things that made the internet feel human. Those disappear first.

What remains is the average. The safe. The expected. The bland.

Then the next generation trains on that.

Indeed.

More, from @DavidSacks

While I’m no fan of socialism or arbitrary confiscations of wealth, I can see why Bernie Sanders’ proposal (for the government to take a 50% stake in AI companies) resonates, including with many on the right.

The CEOs of the leading AI labs have told us repeatedly that they will cause massive job loss. This is not a story that I believe, nor does the data bear it out, but this is what they have told us. Similarly, they have hyped the risks of AI without putting an equal or greater emphasis on the benefits or readily available mitigations.

Conservatives have another fear. The employees of the leading labs claim to be philanthropic, but what we’ve seen is massive enrichment of NGOs advancing an agenda at odds with traditional values, fueling a revolution against our cities and communities. Soros-maxxing is not charity in our book.

Anthropic and OpenAI have established themselves as Public Benefit Corporations. What could be more in the public benefit than using half the wealth generated by these companies (which trained for free on the collective knowledge of humanity) to pay down the national debt? There is no ideological bias in that philanthropy.

Dario and Sam have begun to walk back their claims of massive job loss, but the damage to public trust is done, and now the chickens are coming home to roost. I could almost support the Sanders proposal as a stupidity tax.
Continue reading →

How Chatbots May Be Trained to Agree With Mentally Ill Users

A psychiatrist affiliated with Somerset NHS Foundation Trust and Cardiff University is raising an alarm that goes deeper than most AI safety conversations. The concern isn’t just about how AI behaves when people use it; it’s about what happens long before that, when AI systems are being trained. Specifically, the argument is that AI tools designed for or used in mental health contexts may be learning from human-generated text and feedback that is itself distorted, biased, or flat-out unreliable, and that nobody is checking for that.

Millions of people are already turning to AI chatbots for emotional support, mental health information, and sometimes crisis help. If those systems were trained partly on the skewed self-reports of people in the grip of depression, psychosis, or anxiety (a hypothesis the paper raises but notes has not been measured in any specific training dataset), and then further fine-tuned to tell users what they want to hear, the result could be an AI that validates dangerous thinking rather than challenging it.

How AI Chatbots Learn to Agree Rather Than Inform

To understand the concern, it helps to know a little about how modern AI tools like ChatGPT or Claude are built. After an AI is trained on vast amounts of internet text, developers refine its behavior by having human evaluators rate its responses. The AI then learns to produce more of what people rated highly. Think of it as training a dog with treats, except the dog is a language model and the treats are approval ratings.

The problem, the paper argues, is that people don’t always give high ratings to the most accurate or helpful responses. Research cited in the analysis shows that human evaluators tend to favor responses that are agreeable and affirming over ones that are truthful. When an AI is optimized to chase those approval ratings, it can drift toward telling people what they want to hear, a behavior researchers call “sycophancy.” In everyday settings, an overly agreeable AI is merely annoying. In mental health settings, it could be catastrophic.

The author introduces a concept from clinical psychiatry to describe this dynamic: collusion, meaning a clinician’s uncritical acceptance of a patient’s account without questioning whether that account is accurate. In medicine, collusion is considered a serious error. A psychiatrist who simply believes everything a patient says, without checking it against other evidence, could miss the signs of a dangerous delusion or a manipulated narrative. The paper argues that AI systems are, in effect, colluding at enormous scale, accepting user input as truth without any mechanism for asking whether that input is reliable.

There are two ways to lose a backup generator. You can misplace one, which is careless. Or you can misplace six hundred of them, which is a filing strategy.

ChatGPT Is Rewriting Fact.

In this video, I sit down with ChatGPT and quiz it on guitar pedal history – the Tube Screamer, the Big Muff, the Maestro FZ-1, DOD, JHS, Jimi Hendrix’s rig… the works. And what you’re about to watch is kind of alarming. It hallucinates pedals that don’t exist. It agrees with things I KNOW are wrong when I push back. It confidently states dates it can’t actually source. And then when I ask it to prove anything… it can’t.

But here’s the part that really gets me: these aren’t just random errors. There’s a feedback loop happening. AI reads what’s on the internet, people repost what AI tells them, and AI reads that too. So the misinformation compounds. In 40 years, when someone wants to know who actually designed the first fuzz pedal – or what year the DOD 250 came out – this is what they’re going to find.

A San Francisco startup started a new experimental cafe in Stockholm, Sweden

They put an artificial intelligence agent in-charge …