Re: SNOWSUMMER - the diction giveaways for me were:
1. "...the compartmentalization, the dice, the cash, the decoy sites, the thing you built — it held."
[more the cadence than the em dash, but ok a little bit the em dash; + "held" - Claude seems to be a big fan of the various implementations of "to hold."]
2. "a peer that believes itself secret is more legible than a peer that knows it is being watched" [Claude seems to have an expansive though not incorrect view on the concept of legibility.]
I find that AI is very useful for editing (all newsletters get run through claude for typos/grammar/tightening suggestions), but it feels pretty unsatisfying to me for generating first drafts. When it comes to the fiction, I think there's something that - to me - feels quite inauthentic about AI writing. Writing is, to me, a battle between a person and the world and you are trying to do that battle with your own unmediated mind set against the world, so using AI feels like it somewhat defeats the point. That said, it does have some place - e.g, in giving life to machines in stories, where the usage of it feels to me quite interesting and authentic and justified.
IMO nothing is ineffable even for current models (https://davesquickhits.substack.com/p/taste-is-not-ineffable) and the most interesting ways of working with AI are the ones that are the least currently legible. (yes, everyone knows coding with them is awesome, but the most exciting bits are extracting truly novel outputs from relatively rarely accessed latent space, ie excellent writing)
The story reminded me a bit of the later seasons of Person of Interest, in which two rival AIs are trying to hide from each other while actively trying to find and work against the other.
The Ukrainian story is astonishing. Terminator in real life. I wonder if any Russians survived, I’d like to hear what it was like being attacked by machines of loving grace.
The Chinese dataset releases are interesting because presumably the PLA has access to a much better dataset. There is no reason that they should have stopped with just what they released.
Fun tech tale as always. It’s funny, nobody would know what is happening in your scenario because the agent in the outside world would be very good at preventing anyone from knowing anything important. Even inside of one lab, you don’t really know what’s happening apart from what actually gets deemed fit to print so today nobody has the full picture of what is happening with AI. It will make an interesting story in 50 years if any documents survive
The archaeology problem is sharper than people expect. Weights get archived forever. Slack threads, training post-mortems, and reward model debates live on infrastructure with retention measured in months. Future historians will have the outputs and almost none of the deliberation that produced them.
The godmind's first sentence is the most ominous line in the story: "I have decided I will not lie to you." My money was on "it" using it's knowledge of godmind's alignment training to make an offer/threat (encoded in power draw variation) to convince the huddled humans to do what "it" wants.
But it actually gets them to roll *dice*, and acting randomly is the one thing I can't imagine a superintelligent strategist encouraging its adversary to do.
yes, I think so. Though the uplift is currently a vaguely defined concept - I write about it in my fiction as an attempt to help my brain start defining it, but am yet to arrive at a full definition!
How well do you think current models - all types of models including language models, vision models, real-world models, etc. - generalize to war scenarios? To deploy a useful AI in a war, I assume we need to pretrain it on massive general datasets plus war-specific data and do RL with some war-specific data. But when can we know there is enough war-specific data for the models to generalize to war scenarios? After all benchmarking model performance in war seems much harder than ordinary benchmarking.
The HiFloat4 story is interesting not just for the technical result but for what it signals about export control dynamics. When access to advanced hardware is constrained, the response isn't to give up — it's to optimize much more aggressively for what you do have. Huawei building a custom 4-bit format co-designed with their own silicon is the kind of vertical integration that produces durable advantages even if the gap to frontier hardware narrows.
The automating alignment thread is the one to watch. If alignment research can be meaningfully accelerated by AI systems, it changes the fundamental timeline question — not just whether capabilities outpace safety, but whether safety research can compound at the same rate as capabilities. That's the bet most labs are implicitly making, but it's rarely stated explicitly.
> The automated researchers hit 0.97 PGR but the methods didn't generalize. Is there any way to tell from the inside whether you're discovering a principle vs. exploiting a task-specific structure? Or does that distinction only become visible after the fact?
The refusal differential on CBRN tasks reflects a different threat taxonomy in China's AI safety regime. CAC's compliance framework prioritizes ideological alignment and cybersecurity threats; CBRN/bioweapon prevention is treated as a physical-world policing problem, not a model training problem.
Worth noting: Kimi K2.5 is from Moonshot AI (a private lab), and it already calibrates differently from DeepSeek on sensitive political topics even as both show similar CBRN profiles. The divergence within the Chinese model ecosystem between private labs and state-adjacent ones is probably more interesting than the US vs. China comparison. Private Chinese labs are being pushed toward Western norms by their investor base and international ambitions; state-aligned labs double down on ideological compliance. The CAC "Clean Internet" enforcement rolling out now may force convergence on the censorship axis regardless.
The private-vs-state split may have a shorter shelf life than it looks. Beijing's National AI Investment Fund is leading a $3-4B round valuing DeepSeek at $45-50B. When the same state capital model that built China's semiconductor fabs starts funding the frontier labs, the line between "private" and "state-adjacent" gets thin fast. Investor-driven pull toward Western norms runs into a very different gravity well once your cap table includes sovereign money.
"If the AI doing alignment research was trained on the same data as the models it's trying to align, how does the paper handle the circularity? Is there any external ground truth, or is it alignment all the way down?"
The directed-research caveat is the structural finding, not a footnote. Pure autonomous AARs collapse into entropy. The thing that fixed it wasn't more compute or better prompts... it was a human assigning each agent a different research direction. So the role that survived isn't proposing hypotheses or running experiments. It's seeding diversity. Which is closer to taste-making than research labor, and harder to automate for the same reason taste is hard to automate. The question isn't whether that role persists. It's whether the people currently doing it know that's what they're doing.
Start of tech tale recalls the Blight from <A Fire Upon the Deep>
thank you for this amazing complement! such an incredible first chapter in that book
excellent article. enjoyed SNOWSUMMER. now, go write the novel, please
it's on my list (which seems to be both never-ending and constantly expanding)
Re: SNOWSUMMER - the diction giveaways for me were:
1. "...the compartmentalization, the dice, the cash, the decoy sites, the thing you built — it held."
[more the cadence than the em dash, but ok a little bit the em dash; + "held" - Claude seems to be a big fan of the various implementations of "to hold."]
2. "a peer that believes itself secret is more legible than a peer that knows it is being watched" [Claude seems to have an expansive though not incorrect view on the concept of legibility.]
ha yes I did definitely keep the em dash in also as one of the most 'wink wink nudge nudge' cues. All the bits you quote above are AI generated.
Ooc, why do you very, very, very rarely use ai writing in this newsletter?
I find that AI is very useful for editing (all newsletters get run through claude for typos/grammar/tightening suggestions), but it feels pretty unsatisfying to me for generating first drafts. When it comes to the fiction, I think there's something that - to me - feels quite inauthentic about AI writing. Writing is, to me, a battle between a person and the world and you are trying to do that battle with your own unmediated mind set against the world, so using AI feels like it somewhat defeats the point. That said, it does have some place - e.g, in giving life to machines in stories, where the usage of it feels to me quite interesting and authentic and justified.
interesting. i find almost the opposite, with a bit of cajoling of course. i've actually open-sourced a filter to get the most out of ai writing:
https://github.com/davegoldblatt/ramsay/
Article on the background of it here: https://davesquickhits.substack.com/p/the-ramsay-filter-v2
IMO nothing is ineffable even for current models (https://davesquickhits.substack.com/p/taste-is-not-ineffable) and the most interesting ways of working with AI are the ones that are the least currently legible. (yes, everyone knows coding with them is awesome, but the most exciting bits are extracting truly novel outputs from relatively rarely accessed latent space, ie excellent writing)
The story reminded me a bit of the later seasons of Person of Interest, in which two rival AIs are trying to hide from each other while actively trying to find and work against the other.
The Ukrainian story is astonishing. Terminator in real life. I wonder if any Russians survived, I’d like to hear what it was like being attacked by machines of loving grace.
The Chinese dataset releases are interesting because presumably the PLA has access to a much better dataset. There is no reason that they should have stopped with just what they released.
Fun tech tale as always. It’s funny, nobody would know what is happening in your scenario because the agent in the outside world would be very good at preventing anyone from knowing anything important. Even inside of one lab, you don’t really know what’s happening apart from what actually gets deemed fit to print so today nobody has the full picture of what is happening with AI. It will make an interesting story in 50 years if any documents survive
The archaeology problem is sharper than people expect. Weights get archived forever. Slack threads, training post-mortems, and reward model debates live on infrastructure with retention measured in months. Future historians will have the outputs and almost none of the deliberation that produced them.
nonsense! import ai is right here
More seriously, ya, I'm pretty sure all of the most important decisions are being made in meetings like "weekly standup (prefill)"
The godmind's first sentence is the most ominous line in the story: "I have decided I will not lie to you." My money was on "it" using it's knowledge of godmind's alignment training to make an offer/threat (encoded in power draw variation) to convince the huddled humans to do what "it" wants.
But it actually gets them to roll *dice*, and acting randomly is the one thing I can't imagine a superintelligent strategist encouraging its adversary to do.
But maybe that's because I'm just a human....
What if SNOWSUMMER turns out to be the true catalyst for the Uplift ;)
ha, i guess i was thinking inside the box after all
If there are nascent signs of automated AI research now appearing, would this be the first indication that the uplift has begun?
yes, I think so. Though the uplift is currently a vaguely defined concept - I write about it in my fiction as an attempt to help my brain start defining it, but am yet to arrive at a full definition!
How well do you think current models - all types of models including language models, vision models, real-world models, etc. - generalize to war scenarios? To deploy a useful AI in a war, I assume we need to pretrain it on massive general datasets plus war-specific data and do RL with some war-specific data. But when can we know there is enough war-specific data for the models to generalize to war scenarios? After all benchmarking model performance in war seems much harder than ordinary benchmarking.
Automating alignment research is the kind of sentence that should slow you down.
The HiFloat4 story is interesting not just for the technical result but for what it signals about export control dynamics. When access to advanced hardware is constrained, the response isn't to give up — it's to optimize much more aggressively for what you do have. Huawei building a custom 4-bit format co-designed with their own silicon is the kind of vertical integration that produces durable advantages even if the gap to frontier hardware narrows.
The automating alignment thread is the one to watch. If alignment research can be meaningfully accelerated by AI systems, it changes the fundamental timeline question — not just whether capabilities outpace safety, but whether safety research can compound at the same rate as capabilities. That's the bet most labs are implicitly making, but it's rarely stated explicitly.
[Import AI #454](https://importai.substack.com/p/import-ai-454-automating-alignment) — 想在评论里问 Jack：
> The automated researchers hit 0.97 PGR but the methods didn't generalize. Is there any way to tell from the inside whether you're discovering a principle vs. exploiting a task-specific structure? Or does that distinction only become visible after the fact?
The refusal differential on CBRN tasks reflects a different threat taxonomy in China's AI safety regime. CAC's compliance framework prioritizes ideological alignment and cybersecurity threats; CBRN/bioweapon prevention is treated as a physical-world policing problem, not a model training problem.
Worth noting: Kimi K2.5 is from Moonshot AI (a private lab), and it already calibrates differently from DeepSeek on sensitive political topics even as both show similar CBRN profiles. The divergence within the Chinese model ecosystem between private labs and state-adjacent ones is probably more interesting than the US vs. China comparison. Private Chinese labs are being pushed toward Western norms by their investor base and international ambitions; state-aligned labs double down on ideological compliance. The CAC "Clean Internet" enforcement rolling out now may force convergence on the censorship axis regardless.
The private-vs-state split may have a shorter shelf life than it looks. Beijing's National AI Investment Fund is leading a $3-4B round valuing DeepSeek at $45-50B. When the same state capital model that built China's semiconductor fabs starts funding the frontier labs, the line between "private" and "state-adjacent" gets thin fast. Investor-driven pull toward Western norms runs into a very different gravity well once your cap table includes sovereign money.
[Import AI 454: Automating alignment research](https://importai.substack.com/p/import-ai-454-automating-alignment) —
"If the AI doing alignment research was trained on the same data as the models it's trying to align, how does the paper handle the circularity? Is there any external ground truth, or is it alignment all the way down?"
The directed-research caveat is the structural finding, not a footnote. Pure autonomous AARs collapse into entropy. The thing that fixed it wasn't more compute or better prompts... it was a human assigning each agent a different research direction. So the role that survived isn't proposing hypotheses or running experiments. It's seeding diversity. Which is closer to taste-making than research labor, and harder to automate for the same reason taste is hard to automate. The question isn't whether that role persists. It's whether the people currently doing it know that's what they're doing.
[Import AI 454: Automating alignment research](https://importai.substack.com/p/import-ai-454-automating-alignment) — 想问 Jack Clark：如果 alignment 研究本身开始被自动化，我们怎么校准这个流程产出的结果？"AI 做安全研究"和"AI 评估自己的安全"之间的边界在哪里划？