Navigating the transformative potential of generative AI: a complex systems perspective

Unraveling the impact and emergent features of powerful AI-based systems

Recently, I had the chance to have a cold drink in a sunny Venice with two friends and colleagues: Alex Arenas — professor at the Universitat Rovira i Virgili — and Ricard Solè — professor at the Universitat Pompeu Fabra and the Santa Fe Institute. A rare opportunity to discuss about complex systems and hot topics, such as the most recent developments in AI. One of the most stimulating discussions I had in months.

In fact, a heated debate — involving scientists, philosophers, and influential exponents of the high-tech ecosystem — is ongoing worldwide. The two orthogonal perspectives concern the potential impact of AI on the future of humankind. This is related to the famous technological singularity problem, which I will discuss in a follow-up.

The first perspective can be summarized by the view of Geoffrey Hinton — who recently left Google to raise awareness about the risks of AI, especially large language models. Hinton fears that AI could manipulate or harm unprepared humans.
The second perspective can be summarized by the view of Yann LeCun — Meta's chief AI scientist — who believes that machines will surpass human intelligence through a positive development, “ushering in a new era of enlightenment” (ndr: see, e.g., the discussion about superhuman AI and decision-making).
A more agnostic (third) perspective is the one by Yoshua Bengio — professor at the University of Montreal and scientific director of the Montreal Institute for Learning Algorithms — who acknowledges the risks while emphasizing the need for rational debate and action.

To the best of my knowledge, from a complexity science perspective I think that we are still far from that kind of AI able to autonomously make decisions to harm humans. Nevertheless, I am concerned that malicious usage of AI-power systems — such as Large Language Models — can have dangerous, if not catastrophic, consequences for our society, from global public health to international political relationships. In fact, such systems can be used — by humans — as a new type of weapons.

In the following, I will quickly discuss:

• some of the Pro and mostly the Cons of deploying powerful AI-based systems

• the scientific grounds of why most scientists and practitioners think that “large enough” AI systems can exhibit emergent features

Overall, I am convinced that the debate is meaningful and well needed, although it highlights two fundamental limitations:

1. about our knowledge of how such large-scale AI systems function;

2. about our understanding of how to classify, identify and characterize emergent phenomena in the empirical world.

In the meanwhile, I have found that the view of Melanie Mitchell and David Krakauer is the one more aligned with my current understanding of this matter:

Problems that require enormous quantities of historically encoded knowledge where performance is at a premium will continue to favor large-scale statistical models like LLMs, and those for which we have limited knowledge and strong causal mechanisms will favor human intelligence. — Source: Melanie Mitchell and David C. Krakauer

In fact, I am still convinced that humans — particularly their behavior in response to societal risks — pose the most significant threats to humankind.

Pro and Cons of deploying powerful AI-based systems

On the one hand, an ethical and wise use of AI can have highly positive impact on humankind, by enhancing our educational and healthcare systems, as well as to increasing our productivity. These ones are only a few emblematic examples, but the potential benefits from adopting such systems are uncountable.

On the other hand, AI can be irresponsibly (mis-)used — by humans with malicious intent — to hurt individuals, communities or whole countries. For instance, AI can be deployed to degrade our information ecosystem, by injecting forged textual, audio and video contents that appears likely to be “true”. This simple fact could be used — by humans — to systematically manipulate large-scale communication in online social platforms, where automation is easier to implement and control by third parties. Generating and disseminating sophisticated deepfakes is not only a technological problem to face: its effects, such as the spread of misinformation and disinformation, are able to undermine trust in media and public figures, sustain conflicts or societal unrest. After all, we have already seen (and still see) some effects of the so-called “infodemics” during events of high societal impact, such as the COVID-19 pandemic (see here for the WHO call or here for quantitative analysis) and the war in Ukraine (see here or here for further details).

An infodemic is too much information including false or misleading information in digital and physical environments during a disease outbreak. It causes confusion and risk-taking behaviours that can harm health. It also leads to mistrust in health authorities and undermines the public health response. An infodemic can intensify or lengthen outbreaks when people are unsure about what they need to do to protect their health and the health of people around them — Source: WHO

This is not the only potential treat to our societal system as we know it. AI-powered systems might have the ability to collect, analyze, and process vast amounts of personal data, produced every second by humans and shared — for free, and often under relaxed policies — through the Internet. This capability can be misused — again, by humans — and lead to widespread surveillance and invasion of privacy. For instance, governments or organizations could exploit AI to monitor and control populations, suppress dissent, or manipulate public opinion. Overall, this application has the potential to undermining fundamental human rights and freedoms.

Finally, another highly concerning application relates to the development and deployment of autonomous weapons systems. If AI-powered machines were used to make lethal decisions and engage in warfare without human oversight, it is plausible to assume that consequences might be devastating. In fact, such automated systems could escalate conflicts with unquantifiable impact on the future of humankind.

Regardless of the likelihood of each scenarios, the unregulated deployment of new AI systems lacking a systematic ethical foundation could have catastrophic consequences. Such concerns led some AI scientists and influential people to call for a six-months pause from further developing systems like ChatGPT, igniting a heated debate between those ones who do not see hints of potential mass extinction and those ones who do. For an academic discussion, see this recent PNAS piece (open-access here) by Melanie Mitchell and David C. Krakauer, summarizing the main arguments of both sides.

The challenge for the future is to develop new scientific methods that can reveal the detailed mechanisms of understanding in distinct forms of intelligence, discern their strengths and limitations, and learn how to integrate such truly diverse modes of cognition. — Source: Melanie Mitchell and David C. Krakauer

Overall, it cannot be denied that massive AI-systems such as ChatGPT and its future developments will have a huge impact on human activities, at many levels.

What ChatGPT means for human intelligence | New Scientist Weekly podcast 181, with Melanie Mitchell, professor of complexity at the Santa Fe Institute.

From some perspective, it might be desirable not to pause AI development, as argued by Yann LeCun.

However, the discussion is far more complex.

Hinton believes that AI is able to “confabulate”, a cognitive activity which is very human. Consequently, the next step for machines could be to set up their own subgoals and pursue them at any cost.

Well, here’s a subgoal that almost always helps in biology: get more energy. So the first thing that could happen is these robots are going to say, ‘Let’s get more power. Let’s reroute all the electricity to my chips.’ Another great subgoal would be to make more copies of yourself. Does that sound good? — Geoffrey Hinton

I believe that intelligent machines will usher in a new renaissance for humanity, a new era of enlightenment. I completely disagree with the idea that machines will dominate humans simply because they are smarter, let alone destroy humans. […] Even within the human species, the smartest among us are not the ones who are the most dominating, and the most dominating are definitely not the smartest. We have numerous examples of that in politics and business. — Yann LeCun

I hear people who denigrate these fears, but I don’t see any solid argument that would convince me that there are no risks of the magnitude that Geoff thinks about. Excessive fear can be paralyzing, so we should try to keep the debates at a rational level. — Yoshua Bengio

But why Hinton and colleagues believe that the new-generation AI-powered systems are able to perform some cognitive activities typical of humans?

AI and emergent features

Many scientists, aligned with the opinions of Hinton, assume that AI will soon exhibit emergent abilities.

A system exhibits an emergent phenomenon if such a phenomenon cannot be directly deduced or anticipated from the full knowledge of system’s unit.

A typical example of emergence is the synchronized lighting (phenomenon) of fireflies (system’s units): when fireflies synchronize, they light up at the same time or in a coordinated pattern.

Another typical emergent phenomenon is the flocking behavior observed in birds:

See ComplexityExplained for more details about different aspects of complexity, including emergence.

Decades of complexity science have demonstrated that it is difficult to define what complexity is and to distinguish between what is emergent from what it is not. Consequently, the word “emergence” is one of the most (mis-)used nowadays (see Jensen’s book for an introduction, or read a concise introduction in one of my recent reviews).

Some researchers define as emergent the abilities exhibited by large AI models which are not exhibited by smaller models. Consequently, it is not possible to predict emergent abilities just by extrapolating the ones observed in smaller models, while it’s plausible that by deploying larger AI models more emergent abilities will be observed.

Honestly, I think that a major issue against this argument is that it is not yet possible to distinguish between features that are genuinely emerging and features that are statistically acquired thanks to a broader parameter space. In fact, if we train a model with 1 billion parameters to perform a task, it is plausible that a larger model — e.g., trained with 10 billion parameters — will outperform the previous one and will have room for new tasks.

Further readings: phenomenological and theoretical (understanding deep learning requires rethinking generalization and surprises in high-dimensional ridgeless least squares interpolation)

Another study has recently challenged the idea of emergent abilities in large language models (LLMs). The authors argue that observations may be a result of the researcher's choices of metrics rather than an inherent property of the model family on a specific task. They suggest that previous claims of emergent abilities in LLMs might be misleading and induced by the way researchers analyze the models. However, it is worth remarking that their paper does not deny the possibility of LLMs displaying emergent abilities, but rather questions the validity of previous claims.

Is a large scale enough to guarantee emergent phenomena?

Likely this is the question right now. To gain some hints, we can briefly discuss the impact of scale on some complex systems of high interest.

First, let us consider the genome. It is like the blueprint of an organism, containing the genetic information necessary for its growth and development. It is plausible to assume that “the larger the genome, the larger the amount of genetic information and the higher the complexity of an organism”. We should first agree on how to define the complexity of an organism and, overall, how to quantify it. Uncountable studies have been published to this aim. One simple possibility is to use the number of genes. The figure below shows the tree of life based on completely sequenced genomes, where each entry along the circle is an organism and the blue bars encode the corresponding number of genes. Surprisingly, in humans (indicated by the red dot) we find about 22,800 genes, a number comparable with the Rattus norvegicus and the Danio rerio, smaller than Mus musculus (~25,300) and Arabidopsis thaliana (~26,000), and much smaller than Oryza sativa (~37,500). Following our assumption, some plants and some rodents are more complex than us.

Tree of life with genome sizes as outer bars (in number of genes, not the total quantity of DNA). Figure by Gringer, Public Domain.

Second, let us consider the brain. It is reasonable to assume that “the larger the brain size the better cognitive abilities”. Also in this case, it is not entirely clear what it means to have a “more powerful” brain or how to measure it (in terms of neurons? The size of prefrontal cortex? Specific specialized circuits? …), hindering our ability to characterize the corresponding emergent properties. A direct comparison of the brain size for a variety of organisms shows that, in fact, humans have a large brain:

However, the figure does not include data about organisms such as the sperm whale (brain size: 7-9 kg) or the elephant (brain size: 4.5-5 kg), which should be smarter than us according to our initial assumption. The point is fairly more complex, and recent studies (see also here and here for further details) show that the absolute number of neurons

is a better predictor of performances in some cognitive task:

With 86–100 billion neurons, humans have the largest number of neurons: larger than expected with respect to their body mass, even among primates and mammals. At variance with the genome, in this case the scale argument might hold, after all.
Or maybe it’s still not clear what intelligence is.

The overall point is that emergent abilities are still poorly understood even for organisms. It is a societal duty to discuss about potential emergent abilities in AI-based machines, but the debate should be based on scientific grounds. To this aim, we need to (i) enhance our understanding to classify, identify and characterize emergent phenomena in the empirical world; and (ii) deepen our knowledge of how such large-scale AI systems function.