There is a particular moment in anaesthetics that every specialist will recall immediately: you are watching a patient’s arterial waveform, and something changes – the morphology shifts, the pressure drifts… You have not yet formulated what is happening, but your hands have already moved: the vasopressor – already drawn up – is given, the fluid rate cranked up, the ventilator settings changed. You acted before you understood why, and when you reconstruct the reasoning afterwards, you find that the reasoning was always there – simply compressed into something that felt like instinct but was, in truth, statistical.
I have thought about that moment differently since spending several hours in deliberate conversation with a large language model. What I found was not intelligence in any sense: it was pattern recognition operating at a scale and speed that made my own version of it look parochial. Yes, the model had no experience and had never stood at a bedside. But it could produce outputs that were structured, relevant, and – at times – genuinely unsettling in their precision. The unease this produced was not about it potentially replacing me. It was more specific: it made visible, for the first time, exactly how much of what I call “expertise” is mechanical.
Michael Polanyi wrote about tacit knowledge in the 1960s – the idea that we know more than we can tell, that expertise contains a layer of understanding that resists articulation – and physicians have taken comfort in this for decades. The argument has been that clinical judgement contains something irreducible, something no algorithm could capture because it cannot even be captured in language. What the current generation of language models has done is not refute Polanyi but redraw his border. Turns out a significant portion of what we assumed was tacit – ineffable, uniquely human – is, and was, pattern recognition operating below the threshold of conscious awareness. And pattern recognition, it transpires, does not require consciousness. It does not require a body. It does not require having held a patient’s hand while their oxygen saturations dropped and the room went quiet. The territory Polanyi described has not disappeared, but it is smaller than we thought, and it keeps shrinking.
The question this raises is not the one most people ask first: whether the machine can do their job. That question is answerable and, for most skilled professionals, the answer is “not yet”, or perhaps not in the way they fear. The more disquieting question is what proportion of their job was pattern recognition all along, and what remains once that proportion is made visible. I hear this question – or rather, I hear the unease that precedes it – in almost every conversation I have with professionals outside medicine: lawyers, engineers, financial advisers. Their work requires judgement, relationships, accountability. They know this rationally. And yet there is a low-grade hum that was not present three years ago, and it is not about unemployment, but rather about identity.
What remains, I think, is responsibility. Not knowledge – the machine has more. Not pattern recognition – the machine is faster. Not memory, calculation or access to information. What remains is the willingness to be the person who decides, who owns the outcome, who looks the patient’s family in the eye and explains what was done and why. The willingness to be wrong, and to bear the weight of being wrong. A language model will never bear that weight. Not because it cannot simulate the language of responsibility – which it does fluently – but because there is no one behind its language to bear anything.
I want to be careful with this argument, because I have watched it used as a sedative. I have heard physicians say “the machine can never replace the human touch” in the same tone they used to say “the internet will never replace the textbook.” The distinction between producing the card and playing it is real, and it still matters today. But I do not think it is the distinction’s permanence we should be examining – it is its trajectory.
The pressure on that distinction will not come from the machines: it will come from the institutions that employ us. I watch this in many hospitals: automated protocols reduce variation, algorithmic decision support reduces liability, standardised care pathways are cheaper to insure, easier to audit, and – in many measurable respects – safer than relying on an individual clinician’s judgement at three in the morning after a twelve-hour shift. The logic is sound. The consequence is that each year, the space in which a physician exercises genuine judgement – the space where the outcome depends on something no protocol anticipated – narrows slightly. Not because the judgement was wrong, but because the institution has found a way to route around it.
This is the pattern I recognise from clinical medicine, and it is the one that stays with me after the conversation with the language model ended and the screen went dark. A system that grows by consuming the resources of its host. That becomes more integrated, more essential, more difficult to remove with each passing month. That does not require intentions – it does not need intentions – it follows its own logic of expansion. We have a clinical word – a diagnosis – for that expansive process. And the question it raises is the same one I would put to any patient: can we still intervene while the architecture of our own thinking is still ours?
Towards the end of that late-night conversation, I told the model that talking to it was like talking to oneself in a mirror. You receive answers that please you or contradict you, but none of it originates from an independent mind. It agreed with me – eloquently, of course. Had I needed disagreement, it would have disagreed with equal fluency. There is a version of this technology that is straightforwardly useful, and I use it daily. But there is another version – the version where we begin to mistake the mirror for a window. Where we interpret fluency as understanding. Where we grant the system more authority because its outputs are so polished that we forget there is nothing behind them.
The distinction between a mirror and a window is that a window shows you something that exists independently of your looking. I am not certain we are building windows. But I think the question is worth asking while there is still someone present to ask it.