by liberal japonicus
I an reaching the point where I measure out my life with 5 day open threads. How about y’all?
"This is the Voice of Moderation. I wouldn't go so far as to say we've actually SEIZED the radio station . . . "
"This is the Voice of Moderation. I wouldn't go so far as to say we've actually SEIZED the radio station . . . "
by liberal japonicus
I an reaching the point where I measure out my life with 5 day open threads. How about y’all?
So sorry, cleek.
Back to LLMs (all links removed). I thought this interesting:
Einstein, Churchill, and AI
What LLMs Can’t Do
Ian Leslie
Aug 15, 2026
I am not very fond of arguments over what AI can or cannot do, since they tend to devolve into arguments over definitions. For instance, it used to be said that AI can’t reason. Now that it’s solving maths problems which have stumped our brightest mathematicians for decades, the sceptics are saying ah, but that’s not what we meant by reasoning. On the other side, AI boosters address every capability gap with a “Not yet. The technology will obviously continue to improve until it solves whatever shortcoming you’re pointing to.” Well, maybe, but this is a frustratingly unfalsifiable claim which can be wheeled out to demolish any sceptical argument, a literal deus ex machina.
So I appreciated this new paper from a Google DeepMind researcher called Tom Zahavy, which is – unusually for a paper from an AI lab – all about what AI, at least in its current form, cannot do. Its title is “LLMs can’t jump” (Ron Shelton’s movie surely has one of the best and most generative titles of all time). In short, it’s about why LLMs are unlikely to make scientific breakthroughs on their own.
The paper argues that there is a fundamental limit to their capacity for invention. LLMs can ingest tons of data and identify the patterns and rules which govern a particular domain. They can also work their way through a series of logical steps to solve specific, highly complex problems. What they can’t do is make a creative “jump” – the sudden insight that enables a human scientist to go beyond the available data.
Zahavy gives what is possibly the most famous documented example of such a leap. In 1907 (the same year Pablo Picasso re-invented art), Albert Einstein was working at the patent office in Bern and also on a new theory of the universe which yoked together space and time. While pondering how “relativity” would work for someone speeding up or slowing down, he had what he later described as “the happiest thought of my life”.
Einstein imagined a man falling from the roof of a house, or standing in a free-falling elevator. What would that feel like? It would feel weightless. If he was holding an apple and let go of it, the apple would “float” alongside him. From the falling man’s point of view, then, it could be said there was no gravity in his pocket of space. That weightlessness, Einstein realised, would be exactly the same if the man were floating in deep space, far from any planet’s gravitational pull.
In a sense, then, falling, not standing, is the man’s natural state. When he stands, the earth is preventing him from falling. His weight is the earth pushing back. From there, Einstein eventually arrived at the idea that objects don’t fall because gravity is pulling them, as Newton believed, but because mass changes the shape of space and time, and objects move along the paths which that shape creates.
If you don’t fully understand this, that’s OK, neither do I. What matters, for our purposes, is that this epochal breakthrough was based on very little data. It’s not like early twentieth century astronomy was generating lots of conflicting information about gravity and acceleration and space. An 1907 LLM trained on Newtonian rules and fed data on planetary movements would have received barely a squeak of an error signal. There was a tiny unexplained wobble in Mercury’s orbit, which most scientists explained by reference to an unseen planet. Otherwise, Newton had it covered.
But Einstein was unsatisfied. Newtonian gravity and Maxwellian electromagnetism worked in completely different ways, and Einstein couldn’t accept that the universe ran on two separate rulebooks. The universe was one, he just knew it – and this led him to make his conceptual leap.
Back to LLMs. Zahavy notes that Einstein’s insight was rooted in physical experience – in knowing what it was like to be a body subject to gravity. It had to be felt because it could not yet be expressed in abstractions – in words or mathematical symbols. Newtonian science offered no logical explanation for his imaginary scenario. Einstein himself described his thought process like this: ”The words or the language, as they are written or spoken, do not seem to play any role in my mechanism of thought.”
LLMs are built out of words and symbols. They don’t have physical experience of the world. They are cathedrals of pure abstraction. If Einstein had an LLM, it could have helped him work through the consequences of his discovery and to make predictions, which involved a huge amount of mathematical work. He would have appreciated that; he wasn’t a very enthusiastic mathematician. But the LLM couldn’t have made the foundational leap for him. No amount of pattern-matching and deduction would have got it there.
Zahavy points out that many scientific breakthroughs did not make sense in the light of available data. They had to be intuited or imagined into being. That in turn has depended, as unscientific as it sounds, on what the scientist wants to believe – on some kind of aesthetic or moral demand they are making of the world. Great scientists often have deep, supra-scientific convictions which operate like search heuristics, pushing them to look for explanations that can’t be spotted in the data.
He mentions Kepler’s neoplatonic belief in the centrality of the sun. We might add Faraday’s discovery of the electromagnetic field (then given mathematical form by Maxwell). That sprang, in part, from Faraday’s Christian belief that since nature was the orderly creation of God, apparently separate forces should exhibit underlying unity. Similarly, Einstein had a profound philosophical conviction that the universe is a unified whole. He wanted that to be true, which pushed him to discover that it is true.
To design an AI capable of invention, Zahavy says, technologists will need systems that “do not just simulate the world but hold strong beliefs or priors about how that world should be structured…”. Note that “should”. Can a machine have a passionate conviction or belief? Would that count as a form of “intelligence” or is that word inadequate to describe the thought process of an Einstein? Zahavy doesn’t explore those questions, but for now, this kind of cognition remains uniquely human.
C.P. Snow’s collection of biographical essays, Variety of Men, includes vivid portraits of Einstein and Churchill, both of whom he had met. He detected certain likenesses. Neither did very well at school, and both developed a deep and abiding conviction that they were different, with something unique to contribute to the world. Once settled on a belief or idea, both were “unbudgeable”; Einstein quietly, Churchill loudly.
In a brilliant passage on Churchill, Snow draws a sharp distinction between judgement and insight. In leadership terms, good judgement means being able to grasp the complexity of a situation and assess the trade-offs. Churchill, says Snow, was not good at this. In fact, most contemporaries agreed that his judgement was “seriously defective”. Here’s Snow:
Churchill had a very powerful mind, but a romantic and unquantitative one. If he thought about a course of action long enough, if he conceived it alone in his own inner consciousness and desired it passionately, he convinced himself that it must be possible. Then, with incomparable invention, eloquence, and high spirits, he set out to convince everyone else that it was not only possible but the only course of action open to man. Unfortunately, the brute facts of life were not always so malleable as his listeners.
Churchill’s obsessive pursuit of certain ideas led to blunders like Gallipoli and his vain attempts to prevent Indian independence. But it was this same obsessiveness which led to his greatest achievement. When all the smart people with good judgment in the British state were proposing that Hitler be accommodated, Churchill alone thought otherwise. Snow again:
Judgement is a fine thing, but it is not all that uncommon. Deep insight is much rarer. Churchill had flashes of that kind of insight, dug up from his own nature, independent of influences, owing nothing to anyone outside himself. Sometimes it was a better guide than judgement. In the ultimate crisis, when he came to power, there were times when judgment itself could, though it did not need to, become a source of weakness. When Hitler came to power, Churchill did not use judgment, but one of his deep insights: this was absolute danger. There was no easy way round.
You often hear it said that “judgement” is where humans will retain an edge over machines. But “insight” might be an even wider moat, and if the AI doomsters are right then it will become very necessary at some point.
Yann LeCun, former chief AI scientist at Meta, now an AI startup founder, was recently asked what he wanted his legacy to be. He said “Increasing the amount of intelligence in the world.” With more intelligence, he continued, there will be less human suffering, more rational decisions, more understanding of the world and the universe. But more intelligence, in the sense he means, wouldn’t have produced Einstein’s leap or Churchill’s insight.
Were Einstein and Churchill obviously more intelligent than their peers? No. Einstein was the greatest scientist of the twentieth century but not because he had a higher IQ than Planck or Bohr (he acknowledged that many of his peers were better at maths). Churchill’s visionary leadership wasn’t the product of rational analysis. Physics and politics are very different realms and Churchill and Einstein had very different kinds of mind, but they both had this ability to “dig up from their own nature” insights that nobody else had grasped, and which weren’t in the data.
Interesting piece from Leslie. Many “yes!” moments from me, but also a few “no, dammit!” moments where I thought he was missing something deeper, or anthropomorphizing in ways that misrepresented the actual situation:
Well, it’s not. I’d argue that the LLM did not, in fact, solve any unsolved problem because it did not set out to solve the problem knowing that there was a gap in our collective knowledge, nor is is able to understand and decide that it had found a correct solution, and know how that solution could be applied. It has no world and no experience. The researchers set it to solving the problem and prompted the parameters. There was no moment of understanding or synthesis for the bot.
I’m reminded of the technicians that were attempting to find the cause of the anomalous signal in a radio telescope, who isolated every possible functional answer for how a telescope could generate this noise itself and ruled them all out, thus proving the presence of the background cosmic radiation predicted by the Big Bang theory.
That was not a discovery prompted by insight on their part. They had not set out to discover cosmic background radiation. They had tried, and failed, to eliminate a noise source that was interfering with the theoretically optimal operation of a radio telescope. The recognition and discovery happened when they submitted their results to scientists who could make the intuitive connection between their work and the theoretical existence of CBR. Without the scientists, they would have just had a radio telescope with an unknown noise problem.
Yes!
No! Or at least this is anthropomorphizing them and imagining an independence, and awareness, and an understanding that they do not have. They do not live in a world, and cannot therefore abstract anything from the world to intuit anything higher order.
No. Einstein discovered that his theory based on that conviction solved problems with the existing models, and gave us a means of generating new problems that we could use to test the validity of his conviction in other circumstances to see how well his convictions held. I think we have two different species of “truth” at work here.
And just how, exactly, is the AI supposed to form these strong beliefs about a world that it does not live in in any meaningful way? What constitutes a “passion” in a binary assembly? Can a passion be prompted?
Yes, in the sense that he is a self-directed being living in a world and forming judgments about that world, shaped by experience, that can be tested to shape his understanding of that world.
No! Churchill, like the LLMs, is working from a dataset and is highly dependent on “influences,” in that his language and society are both acquired and maintained by collective consensus. He can indeed intuit conceptual jumps that did not exist before he makes them, but those insights are not independent of influences, they are just novel approaches that have the potential to reshape consensus. The insight is still predicated upon a world and a society that pre-exist, and co-exist, whose influences permeate his every thought.
This is, I think, the core disagreement between the AI enthusiasts and those of us who are skeptics. The AI does interact with the natural world. All it can do, all it will ever be able to do without a complete redesign, is respond to specific queries from those who do.
Take an obvious (and frankly worrisome) hypothetical. Suppose an AI is fed a query: prove the Donald Trump is the greatest President ever. It will put together a line of plausible sounding bull to that end. Given a query substituting “worst” for “greatest”, it will produce equally plausible sounding bull.
Neither will be particularly constrained by the fact that whatever statistics it uses might be invented out of whole cloth. If it needs inflation to be low, it’s got plenty of sources (starting with Trump himself) who say that it is. But the AI has no reality check of shopping every week.