(Trying to) Measuring consciousness in the lab
What we learn from a rigorous adversarial test and what that means for AI
If you find Complexity Thoughts interesting, follow us! Click on the Like button, leave a comment, repost on Substack or share this post. It is the only feedback I can have for this free service. The frequency and quality of this newsletter relies on social interactions. Thank you!
→ Don’t miss the podcast version of this post: click on “Spotify/Apple Podcast” above!
Overview
Today we are going to discuss about the recent results obtained by two “theories of consciousness”, including comments from Dr. Kevin Mitchell, Professor of Genetics and Neuroscience at Trinity College Dublin, and Dr. Konrad Kording, Professor at U. Penn and computational neuroscientist.
Our goal is to better understand what’s been tested and how to interpret the results. Accordingly, I will do my best to provide:
A quick overview of theories of consciousness
Setting up a rigorous test
Learning from adversarial testing
Comments from the research community
Consciousness is notoriously slippery: without a clear, operational definition, it becomes all but impossible to develop reliable measures. This lack of consensus complicates efforts to probe awareness in human brains — or even to assess machine “sentience”, which is a hot topic nowadays in Artificial Intelligence research — leaving scientists with both conceptual and methodological challenges.
If you are curious about why this research is relevant for AI, check the following post:
A quick overview of theories of consciousness
Integrated Information Theory (IIT) frames consciousness as an intrinsic property of a system’s causal structure, quantified by the amount of irreducible integrated information (Φ) it supports. According to IIT, any substrate — be it neural tissue, logic gates or even organoids — could be conscious to the extent that it maximizes Φ in a ‘main complex’ localized in posterior cortex.
Global Neuronal Workspace Theory (GNWT) views consciousness as the global broadcasting of information: a stimulus becomes conscious when it triggers a late ‘ignition’ that amplifies and distributes its representation across a network of higher-order and prefrontal regions, enabling flexible report and coordination.
If you are not familiar with brain areas, you can check the above map or this amazing interactive atlas.
Critics and uproar
In September 2023, a letter co-signed by 124 researchers accused IIT of pseudoscience, arguing that its lofty axioms (such as panpsychist attributions to fetuses, plants or logic circuits) rest on untestable assumptions and that the handful of experimental tests to date target peripheral predictions, not the theory’s core.
A few days later, a commentary appearing on Nature warned that IIT’s hype — both in academic outlets and popular media — has outpaced its empirical grounding, exacerbating divisions in a field already smashed by credibility concerns.
In September 2023 Kevin Mitchell dedicated a piece to this matter on his popular blog Wiring the Brain (that I strongly recommend to follow), where he discussed some good questions and reported:
“If we had a theory that could accommodate all those elements and provide some coherent framework in which they could be related to each other – not for providing all the answers but just for asking sensible questions – well, that would be a theory of consciousness.” — K- Mitchell
IIT’s response
In March 2025 a reply by Giulio Tononi and colleagues was published as a Comment in Nature Neuroscience. The authors defended IIT as a “consciousness-first” paradigm: rather than treating awareness as an emergent function, they begin with the axiomatic features of experience (intrinsicality, information, integration, exclusion, composition) and derive objective postulates about a substrate’s causal power and its embedded cause–effect structure. This contrasts with the “dominant computational-functionalist paradigm” which focuses on objective tools to explain functions and computations, potentially accounting for "pseudo-consciousness" rather than subjective experience itself.
They highlight decades of formal development, computational models and empirical indices, arguing these validate the explanatory identity between integrated information and subjective experience. Far from being untestable, they contend, IIT already makes concrete predictions about where and how consciousness should manifest in the brain.
Why the debate is not settled
Konrad Kording, a signatory of the aforementioned letter, remained unconvinced: he criticizes IIT’s defense for sliding from “if IIT is right” metaphysical speculations into evolutionary narratives without clear, falsifiable methodologies, and for relying on correlates like PCI (Perturbational Complexity Index) that arguably track wakefulness more than genuine integration.
Setting up a rigorous test
How can we falsify theories of consciousness in the same spirit of what’s routinely done, for instance, in physics?
One way is to build adversarial collaborations where proponents of different theories, along with theory-neutral researchers, work together to design experiments that can differentiate between their incompatible accounts.
Accordingly, measuring awareness must rely on clear, pre-specified tests rather than simple correlations. First, proponents of each theory agree on — and, overall, preregister — divergent and falsifiable predictions (e.g. IIT’s “posterior hot-zone” versus GNWT’s “prefrontal ignition”), complete with exact and testable pass/fail criteria.
Next, experiments parametrically manipulate conscious experience to isolate neural signals tied purely to awareness. This can include varying stimulus content (faces, objects, letters, false fonts), duration (0.5 – 1.5 s) and task relevance. To capture both where and when the neural signals occur, the same participants undergo fMRI, MEG and intracranial EEG, marrying whole-brain spatial maps with millisecond-scale timing.
Analyses then follow the preregistered plan to the letter: decoding content from activity patterns, tracking activation strength and representational similarity over time, as well as measuring inter-areal synchrony in the frequency bands each theory predicts. Finally, a neutral team compares outcomes against the original criteria: treating failed predictions not as embarrassments but as the clearest path to refining or rejecting a theory.
That’s cool, isn’t it? But the real question is: is that sufficient to make claims?
You might also be interested in knowing more about how different brain architectures lead to similar functions:
Learning from adversarial testing
To move beyond rhetoric, the Cogitate Consortium orchestrated a preregistered, multimodal adversarial collaboration pitting IIT against GNW in 256 participants using fMRI, MEG, and iEEG.
Both theories predicted distinct signatures of conscious content, maintenance and connectivity:
posterior sustained integration for IIT;
prefrontal ignition and long-range broadcasting for GNW.
Conscious content could be decoded in both regions, but sustained synchronization/connectivity within the posterior cortex (IIT) and reliable 'ignition' (transient activation) in the prefrontal cortex specifically at stimulus offset (GNW) were largely absent. Furthermore, GNWT's prediction of phasic (brief, late-phase) long-range connectivity between high-level category-selective areas and the prefrontal cortex was also not supported by the preregistered iEEG and MEG analyses.
These joint failures highlighted the need for tighter quantitative frameworks that bind computational cores to biological implementations, setting a new standard for collaborative theory testing in consciousness science.
Comments from the research community
To better understand the results, I have discussed with some experts.
The study was designed to be a stringent test, making failures informative. The fact that neither IIT's nor GNWT's predictions were reliably found forces the question of whether these specific neural signatures are truly necessary for consciousness, or if the theories need to propose entirely different mechanisms for maintaining and updating conscious content.
Do you think that the results indicate fundamental flaws in the core neural mechanisms proposed by each theory, or primarily highlight the limitations of current experimental paradigms and measurements in capturing the true substrates of consciousness?
Konrad Kording: I think I have very little in the ways of answers. I feel like this field's questions are too poorly defined for me to say useful things about. It is unclear what the approaches measure, it is unclear how they relate to consciousness. And it is unclear how central what was tested is to the theories.
Kevin Mitchell: To begin with, I don't think IIT and GNWT are "theories of consciousness" in any broad sense. They relate to a subset of the questions about consciousness that I think a fully fledged theory would have to encompass [listed here].
What the COGITATE study looks at is the predictions of these theories with regard to the very specific question of the nature of the brain states associated with conscious perception of a visual stimulus (that is, associated with specific conscious visual content, as opposed to other content, not as opposed to no conscious perception). For reasons that aren't terribly compelling, in my view, the IIT favors increased connectivity between posterior visual areas as a correlate of conscious visual perception and maybe somehow a "vehicle" of the current contents of conscious visual perception. While GWNT predicts increased activity and content encoding in PFC and connectivity between frontal and posterior areas.
Given the lack of predicted phenomenology, how is information about conscious percepts maintained over time and integrated across brain regions?
Konrad Kording: N/A
Kevin Mitchell: It's worth asking something about the specificity of these predictions. I don't think they are really that specific to IIT or GWNT nor do I think they rely on the more fundamental posits of those theories. There are lots of other ways one could theorise about consciousness that make similar predictions. (I also feel like there's something arbitrary about the predictions - I think you could generate other predictions starting with the premises of both of those theories or something in their neighborhoods). So, positive results are not confirmatory, just consistent. But negative results are not fatal - they might just prompt a reworking of the specifics.
If consciousness is defined by abstract information processing and functional organization — rather than fixed neural signatures — do these failures actually support a shift toward purely computational markers?
Konrad Kording: N/A
Kevin Mitchell: I don't think either of those frameworks is well enough developed to really be called a "theory". They're quite sketchy ideas, in my view, and not really well integrated with ideas about the other important questions regarding consciousness.
Having said that, the COGITATE study takes them at their word and at least tries to set up some objective test of their predictions. But I don't find it surprising at all that the results are mixed and inconclusive - it seems almost inevitable that they would be given the vagueness of the theories, the fact that we already knew that both posterior and frontal activity are involved in some way for some aspects of conscious visual perception, and the limited resolution of the imaging methods available in humans.
So, I applaud the sentiment behind the effort, even though I don't think the two "theories" in question are developed enough to actually be verified or falsified, even if some of their specific predictions are. Also, more fundamentally, I don't think the GWNT and the IIT are necessarily really in conflict — they seem to be concerned with quite different aspects of the phenomenon of consciousness and I could see them being integrated quite readily.
It is clear that there are more problems in experimental design than in core theories, and I thank Kevin and Konrad for taking time to discuss with me about this fascinating topic.
I am wondering if — beyond static measures of connectivity or activation strength — transient reconfigurations such as rapid shifts in network topology or phase-amplitude coupling across timescales could better capture how the brain maintains and integrates perceptual information.
Often consciousness is regarded as an emergent feature of brain dynamics: therefore, it might be plausibile to think that general-purpose approaches currently under development (for detecting signatures of emergent phenomena) might be applied in the next future. We have already discussed about this topic here:
The role of these theories in assessing LLMs consciousness
There are some studies attempting to use IIT and GNWT to assess the emergence of consciousness (broadly, and possibly poorly, speaking) in Large Language Models (LLMs). I usually do not point to preprints, since I prefer to share information or comment about papers that passed peer-review. Accordingly, I will limit to focus on two studies.
The first study is an Opinion by VanRullen and Kanai, discussing how deep learning models naturally comprise multiple specialized “modules” (latent spaces) whose interactions mirror GNWT’s requirement for a limited-capacity workspace broadcasting information broadly.
However, standard Transformers lack the requisite recurrent loops and top-down/bottom-up gating mechanisms central to GNWT’s selective broadcasting and capacity limits. Without these, LLMs cannot sustain a unified, persistent workspace across time, limiting their ability to meet GNWT’s criteria for global availability and context maintenance (→ see the paper).
“GLW [Global Latent Workspace] could also be a way to develop entirely novel architectures capable of planning, reasoning, and thinking through the flexible reconfiguration of multiple existing modules. This may bring us one step closer to general-purpose (system-2) artificial cognition”
Another study by Gams and Kramar frames IIT’s five axioms — Intrinsic Existence, Composition, Information, Integration and Exclusion — outlining each axiom’s postulate and then qualitatively rating ChatGPT’s adherence on a 1–10 scale based on functional analogies rather than any quantitative Φ computation . They report an average score below 3 — which is under the positivity threshold of 6 — highlighting that, despite its impressive linguistic abilities, ChatGPT lacks the integrated, self-determining properties that IIT associates with conscious systems (→ see the paper).
In a nutshell: both studies reveal that current LLM designs fall short of structural and functional prerequisites for phenomenal consciousness.
Which still confirms what I have written here almost two years ago:
Take home message(s)
I know: likely I didn’t succeed in providing a concise coverage of the topic, but here we are and if you made it up to here, many thanks for your patience and your curiosity. Here a concise summary:
Adversarial testing is key: preregistered, multimodal experiments reveal that neither IIT’s nor GNWT’s core neural signatures consistently predict consciousness.
Current AI architectures fall short: feed-forward LLMs lack the recurrent loops and causal structure required by both theories to achieve consciousness.
Towards next-gen frameworks: future work could prioritize dynamic network reconfiguration and scalable integration metrics to bridge theory and implementation.
It's becoming clear that with all the brain and consciousness theories out there, the proof will be in the pudding. By this I mean, can any particular theory be used to create a human adult level conscious machine. My bet is on the late Gerald Edelman's Extended Theory of Neuronal Group Selection. The lead group in robotics based on this theory is the Neurorobotics Lab at UC at Irvine. Dr. Edelman distinguished between primary consciousness, which came first in evolution, and that humans share with other conscious animals, and higher order consciousness, which came to only humans with the acquisition of language. A machine with only primary consciousness will probably have to come first.
What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing.
I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.
My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461