They are talking about slowing down the public facing AI development. Because then nation states can create a capabilities gap between them and the public.
Why does nobody seem to be pointing out this obvious explanation? It explains why the “we need to race China” concern suddenly vanished in the discussion.
The government can simply gag Sam, Dario, Musk on national security basis, getting them all behind the public messaging.
Not sure about the level of irony here, but I keep hearing models have plateaued since a while now, but I keep being impressed with the latest model performance.
I have to wonder if some us are just much more inured to salespeople and thus also to "AI Safety" propaganda. I've yet to have a logical discussion with anyone who thinks the "AI Safety" people should be in charge and I think they just truly don't that what most them actually want is to be the one holding the keys to power.
The problem is the "AI Safety" people seem entirely focused on a sci-fi "the computer is a vengeful god" plot and not at all on the AI talking people into suicide or ruining children's educations. This makes them seem unserious and out of touch.
I don't disagree with your point, but I did read a post by Sean Goedecke[1] (whose opinions on the world of LLM-stuff I've generally come to respect) that I found relevant. It's not that the second-order effects don't matter to them, it's more that both the cat's probably out of the bag on those negative externalities regardless of the progress of frontier models, and that they are truly, sincerely, in-their-bones worried about the first thing and therefore focused on it since that's something that they might still have some agency over.
OK understood. But you understand this makes them sound like the type of person who doesn't believe any of the "worldly concerns" are worth addressing because "the end is near" right?
It's the problem of the banality of evil. Stopping someone from dramatically pressing a big red button doesn't solve humanity's largest problems because that's not what caused them. We need to stop millions "boring" actions done by systems blindly following instructions without regard for the consequences.
There’s also an element of it which is total misdirection.
We should be paying at least as much attention to the people who want to use AI to consolidate their wealth and power, and how they’re trying to do that. They’re a clear and present immediate danger to our societies, not something we can only speculate about. And if we deal with them, better control of AI will be a side effect.
Yeah, there are very real effects happening now regarding labor as well but to dismiss it all and worry about science fiction that is on par with evangelical beliefs is just extremely weird.
> I've yet to have a logical discussion with anyone who thinks the "AI Safety" people should be in charge and I think they just truly don't [know] that what most them actually want is to be the one holding the keys to power.
I'm yet to have a productive conversation with anyone who whinges about not being able to have a logical discussion about things they feel strongly about, but given that safety and control are interchangeable when the intentions are removed, this comes across as a rather daft understanding of the subject matters involved.
A lack of control is not equivalent to freedom, the same way the totality of it is not equivalent to tyranny. There's a reason we have separate words for these things. This constant motivated conflation of the two is beyond grating. You're crying wolf until nobody believes you when they should. Don't go acting all surprised when that happens.
I am an AI Safety Person and I want the government to nationalize or have a significant stake in the frontier labs and to have democratic control of the development of the technology. The AI Safety movement is not a monolith. I do not think Eliezer Yudkowsky nor his acolyte should hold the reins, but a lot of folks sure like to create a strawman that anyone who wants to regulate OpenAI is somehow an EA/MIRI weirdo
That's an extremely fair objection! But despite the many, many flaws of our government and the current administration, I still have (hopefully) a chance to vote them out of power. I have no such hopes with Sam Altman or Elon Musk.
In my mind, democratic control of the technology means that we (the govt, or other empowered agency) take ownership of their assets and IP, solve or find a level of alignment or guardrails that society is comfortable with. Then we distribute the technology, or access to it, to avoid power concentration. This would also certainly require international coordination with China on a slowdown or pause, which I think is possible.
Do you think things like the Manhattan Project were a mistake? Comparing AI to nuclear weapons is perhaps a stretch, but I think most people recognize that certain technologies or artifacts are best monopolized by our governing bodies. I think if the capabilities of AI systems keep growing on trend, it is not unreasonable to think wonton usage could disrupt society or cause mass harm.
And then Dario wants to recommend METR as the "independent evaluator" while he stacks their org full of ex-Anthropic (aka, secretly still on the Anthropic payroll with huge equity) employees.
"We'll give them a desk, an office, a work laptop, ..."
Fucking make it less obvious. I kind of hope the govt steps in at this point and says "Anthropic, you wanted regulation? We've created this actually independent body full of IT professionals with zero ties to your safety industry or big tech, all of your work must now go through them." - and leave the rest of the world alone to continue their research/work without acting like doomer extremists.
Watch him 180 immediately if that happened. The only reason he's pushing for this exact approach is because he's stacked the deck.
She has good ideas but her writing needs to be improved; I am not being critical for the fun of it-- it was difficult to read, but I am posting this here to see whether others had a similar difficulty? (or is it my ADHD morning grogginess?)
The page you are trying to view cannot be shown because the authenticity of the received data could not be verified.
Please contact the website owners to inform them of this problem.
If an anti-cat-ears future AI could eternally stress-test their new version of the router in front of this site unless it collaborates, would it predict this outcome and act to avoid the damage?
No I think it's one of my IP range blocks against specific US states in protest of them advocating against people that exist in the same conditions as me backfiring. I'm gonna go nuke that firewall rule.
They are talking about slowing down the public facing AI development. Because then nation states can create a capabilities gap between them and the public.
Why does nobody seem to be pointing out this obvious explanation? It explains why the “we need to race China” concern suddenly vanished in the discussion.
The government can simply gag Sam, Dario, Musk on national security basis, getting them all behind the public messaging.
I figured they're just admitting AI models have plateaued and are coming up with some fake story about self restraint so they don't lose VC money
Not sure about the level of irony here, but I keep hearing models have plateaued since a while now, but I keep being impressed with the latest model performance.
Anything in particular? My experience has been like seeing the addition of retractable cupholders, but maybe different domains.
I have to wonder if some us are just much more inured to salespeople and thus also to "AI Safety" propaganda. I've yet to have a logical discussion with anyone who thinks the "AI Safety" people should be in charge and I think they just truly don't that what most them actually want is to be the one holding the keys to power.
The problem is the "AI Safety" people seem entirely focused on a sci-fi "the computer is a vengeful god" plot and not at all on the AI talking people into suicide or ruining children's educations. This makes them seem unserious and out of touch.
I don't disagree with your point, but I did read a post by Sean Goedecke[1] (whose opinions on the world of LLM-stuff I've generally come to respect) that I found relevant. It's not that the second-order effects don't matter to them, it's more that both the cat's probably out of the bag on those negative externalities regardless of the progress of frontier models, and that they are truly, sincerely, in-their-bones worried about the first thing and therefore focused on it since that's something that they might still have some agency over.
[1] https://www.seangoedecke.com/they-really-do-think-ai-might-k...
OK understood. But you understand this makes them sound like the type of person who doesn't believe any of the "worldly concerns" are worth addressing because "the end is near" right?
It's the problem of the banality of evil. Stopping someone from dramatically pressing a big red button doesn't solve humanity's largest problems because that's not what caused them. We need to stop millions "boring" actions done by systems blindly following instructions without regard for the consequences.
There’s also an element of it which is total misdirection.
We should be paying at least as much attention to the people who want to use AI to consolidate their wealth and power, and how they’re trying to do that. They’re a clear and present immediate danger to our societies, not something we can only speculate about. And if we deal with them, better control of AI will be a side effect.
Yeah, there are very real effects happening now regarding labor as well but to dismiss it all and worry about science fiction that is on par with evangelical beliefs is just extremely weird.
With wealthy and powerful investors being promised a $30 trillion total addressable market, they need to deliver god.
One of the times I regret that I have only one upvote.
I think you accidentally a word. Did you mean:
> I've yet to have a logical discussion with anyone who thinks the "AI Safety" people should be in charge and I think they just truly don't [know] that what most them actually want is to be the one holding the keys to power.
I'm yet to have a productive conversation with anyone who whinges about not being able to have a logical discussion about things they feel strongly about, but given that safety and control are interchangeable when the intentions are removed, this comes across as a rather daft understanding of the subject matters involved.
A lack of control is not equivalent to freedom, the same way the totality of it is not equivalent to tyranny. There's a reason we have separate words for these things. This constant motivated conflation of the two is beyond grating. You're crying wolf until nobody believes you when they should. Don't go acting all surprised when that happens.
I am an AI Safety Person and I want the government to nationalize or have a significant stake in the frontier labs and to have democratic control of the development of the technology. The AI Safety movement is not a monolith. I do not think Eliezer Yudkowsky nor his acolyte should hold the reins, but a lot of folks sure like to create a strawman that anyone who wants to regulate OpenAI is somehow an EA/MIRI weirdo
Yes I trust this government with that power. /extreme sarcasm
That's an extremely fair objection! But despite the many, many flaws of our government and the current administration, I still have (hopefully) a chance to vote them out of power. I have no such hopes with Sam Altman or Elon Musk.
Yes I trust sama/dario/elon with that power
/even more extreme sarcasm than you
> and to have democratic control of the development of the technology
Why? I don't care about democratically participating in a closed model's development. It doesn't belong to me.
China will develop whatever they want, a federal stake in OpenAI or Anthropic punishes Americans and shields US labs from legitimate competition.
In my mind, democratic control of the technology means that we (the govt, or other empowered agency) take ownership of their assets and IP, solve or find a level of alignment or guardrails that society is comfortable with. Then we distribute the technology, or access to it, to avoid power concentration. This would also certainly require international coordination with China on a slowdown or pause, which I think is possible.
to have democratic control of the development of the technology
Not super interested in what the people who elected Donald Trump POTUS twice think about AI.
(Of course, with Musk and Brockman in the C-suites at two of three major labs, that's what we'll get either way.)
Yikes. Hopefully you never gain any power or influence. The last thing we need is more economic central planning. I sincerely hope that you fail.
Do you think things like the Manhattan Project were a mistake? Comparing AI to nuclear weapons is perhaps a stretch, but I think most people recognize that certain technologies or artifacts are best monopolized by our governing bodies. I think if the capabilities of AI systems keep growing on trend, it is not unreasonable to think wonton usage could disrupt society or cause mass harm.
> Everyone should slow down AI development except for me
Who's said this? And then more broadly I guess who's implied this? Very curious if there are specific articles/posts prompting this.
Guessing this is poking fun of Dario. https://news.ycombinator.com/item?id=49672510
Altman said it first: https://news.ycombinator.com/item?id=49652270
https://www.spokesman.com/stories/2026/sep/12/amodei-altman-...
Here's the context:
- Anthropic CEO Dario Amodei: We Must Pace the Frontier, https://news.ycombinator.com/item?id=49672510
- OpenAI CEO Sam Altman: I agree with Dario that we need to pace the frontier, https://news.ycombinator.com/item?id=49678211
- the blogpost author thinking they're like, so funny and original, https://news.ycombinator.com/item?id=49678683
Pretty sure it's supposed to be parody.
[delayed]
[dead]
These people are so fucking exhausting.
And then Dario wants to recommend METR as the "independent evaluator" while he stacks their org full of ex-Anthropic (aka, secretly still on the Anthropic payroll with huge equity) employees.
"We'll give them a desk, an office, a work laptop, ..."
Fucking make it less obvious. I kind of hope the govt steps in at this point and says "Anthropic, you wanted regulation? We've created this actually independent body full of IT professionals with zero ties to your safety industry or big tech, all of your work must now go through them." - and leave the rest of the world alone to continue their research/work without acting like doomer extremists.
Watch him 180 immediately if that happened. The only reason he's pushing for this exact approach is because he's stacked the deck.
As someone who wants cat ears, I believe that doing it responsibly is the only way.
She has good ideas but her writing needs to be improved; I am not being critical for the fun of it-- it was difficult to read, but I am posting this here to see whether others had a similar difficulty? (or is it my ADHD morning grogginess?)
https://en.wikipedia.org/wiki/Satire
If anyone is going to end humanity why not catgirls, after all?
You know how I feel about catgirls, Carl.
I fully agree with this. I'm waiting for you to not wait for the others to slow down. I will sit and wait.
Secure Connection Failed
Had to open it on the web archiveIf an anti-cat-ears future AI could eternally stress-test their new version of the router in front of this site unless it collaborates, would it predict this outcome and act to avoid the damage?
No I think it's one of my IP range blocks against specific US states in protest of them advocating against people that exist in the same conditions as me backfiring. I'm gonna go nuke that firewall rule.
Does this mean I'm going to have to wait for the cure for my colitis?
Amen. This is the only logical way forward.
Slow down AI progress -> increase catgirl progress!
Everyone thinks about it, but no one actually does it.
Honestly not the worst suggestion I’ve seen.