Complexity Thoughts: Issue #23
Unraveling complexity: building knowledge, one paper at a time
If you find Complexity Thoughts, click on the Like button, leave a comment, repost on Substack or share this post. It is the only feedback I can have for this free service.
The frequency and quality of this newsletter relies on social interactions. Thank you!
Dall-e 3 representation of this issue’s content
Foundations of network science and complex systems
Network representations of attractors for change point detection
A common approach to monitoring the status of physical and biological systems is through the regular measurement of various system parameters. Changes in a system’s underlying dynamics manifest as changes in the behaviour of the observed time series. For example, the transition from healthy cardiac activity to ventricular fibrillation results in erratic dynamics in measured electrocardiogram (ECG) signals. Identifying these transitions—change point detection—can be valuable in preparing responses to mitigate the effects of undesirable system changes. Here, we present a data-driven method of detecting change points using a phase space approach. Delay embedded trajectories are used to construct an ‘attractor network’, a discrete Markov-chain representation of the system’s attractor. Once constructed, the attractor network is used to assess the level of surprise of future observations where unusual movements in phase space are assigned high surprise scores. Persistent high surprise scores indicate deviations from the attractor and are used to infer change points. Using our approach, we find that the attractor network is effective in automatically detecting the onset of ventricular fibrillation (VF) from observed ECG data. We also test the flexibility of our method on artificial data sets and demonstrate its ability to distinguish between normal and surrogate time series.
Model scale versus domain knowledge in statistical forecasting of chaotic systems
Chaos and unpredictability are traditionally synonymous, yet large-scale machine-learning methods recently have demonstrated a surprising ability to forecast chaotic systems well beyond typical predictability horizons. However, recent works disagree on whether specialized methods grounded in dynamical systems theory, such as reservoir computers or neural ordinary differential equations, outperform general-purpose large-scale learning methods such as transformers or recurrent neural networks. These prior studies perform comparisons on few individually chosen chaotic systems, thereby precluding robust quantification of how statistical modeling choices and dynamical invariants of different chaotic systems jointly determine empirical predictability. Here, we perform the largest to-date comparative study of forecasting methods on the classical problem of forecasting chaos: we benchmark 24 state-of-the-art forecasting methods on a crowdsourced database of 135 low-dimensional systems with 17 forecast metrics. We find that large-scale, domain-agnostic forecasting methods consistently produce predictions that remain accurate up to two dozen Lyapunov times, thereby accessing a long-horizon forecasting regime well beyond classical methods. We find that, in this regime, accuracy decorrelates with classical invariant measures of predictability like the Lyapunov exponent. However, in data-limited settings outside the long-horizon regime, we find that physics-based hybrid methods retain a comparative advantage due to their strong inductive biases.
DomiRank Centrality reveals structural fragility of complex networks via node dominance
From a simple model perspective, Γ(t) can be interpreted as the evolving fitness of the individuals in a population subject to competition
Determining the key elements of interconnected infrastructure and complex systems is paramount to ensure system functionality and integrity. This work quantifies the dominance of the networks’ nodes in their respective neighborhoods, introducing a centrality metric, DomiRank, that integrates local and global topological information via a tunable parameter. We present an analytical formula and an efficient parallelizable algorithm for DomiRank centrality, making it applicable to massive networks. From the networks’ structure and function perspective, nodes with high values of DomiRank highlight fragile neighborhoods whose integrity and functionality are highly dependent on those dominant nodes. Underscoring this relation between dominance and fragility, we show that DomiRank systematically outperforms other centrality metrics in generating targeted attacks that effectively compromise network structure and disrupt its functionality for synthetic and real-world topologies. Moreover, we show that DomiRank-based attacks inflict more enduring damage in the network, hindering its ability to rebound and, thus, impairing system resilience. DomiRank centrality capitalizes on the competition mechanism embedded in its definition to expose the fragility of networks, paving the way to design strategies to mitigate vulnerability and enhance the resilience of critical infrastructures.
Biological Systems
A typical problem in complexity science is to devise an optimal strategy to explore an environment while exploiting its resources. The following papers propose interesting solutions, including mode-switching behavior and ergodic information harvesting, which deserves further exploration. The second paper is especially intriguing, since the reconciles the metabolic cost of motion with information theory.
Mode switching in organisms for solving explore-versus-exploit problems
We discovered that the velocity distributions that emerge from the interplay between exploratory movements and goal-directed control are broad-shouldered across taxa, and that this distinctive distribution of movements is robustly modulated by sensory salience
Trade-offs between producing costly movements for gathering information (‘explore’) and using previously acquired information to achieve a goal (‘exploit’) arise in a wide variety of problems, including foraging, reinforcement learning and sensorimotor control. Determining the optimal balance between exploration and exploitation is computationally intractable, necessitating heuristic solutions. Here we show that the electric fish Eigenmannia virescens uses a salience-dependent mode-switching strategy to solve the explore–exploit conflict during a refuge-tracking task in which the same category of movement (fore-aft swimming) is used for both gathering information and achieving task goals. The fish produced distinctive non-Gaussian distributions of movement velocities characterized by sharp peaks for slower, task-oriented ‘exploit’ movements and broad shoulders for faster ‘explore’ movements. The measures of non-normality increased with increased sensory salience, corresponding to a decrease in the prevalence of fast explore movements. We found the same sensory salience-dependent mode-switching behaviour across ten phylogenetically diverse organisms, from amoebae to humans, performing tasks such as postural balance and target tracking. We propose a state-uncertainty-based mode-switching heuristic that reproduces the distinctive velocity distribution, rationalizes modulation by sensory salience and outperforms the classic persistent excitation approach while using less energy. This mode-switching heuristic provides insights into purposeful exploratory behaviours in organisms, as well as a framework for more efficient state estimation and control of robots.
Tuning movement for sensing in an uncertain world
While animals track or search for targets, sensory organs make small unexplained movements on top of the primary task-related motions. While multiple theories for these movements exist—in that they support infotaxis, gain adaptation, spectral whitening, and high-pass filtering—predicted trajectories show poor fit to measured trajectories. We propose a new theory for these movements called energy-constrained proportional betting, where the probability of moving to a location is proportional to an expectation of how informative it will be balanced against the movement’s predicted energetic cost. Trajectories generated in this way show good agreement with measured trajectories of fish tracking an object using electrosense, a mammal and an insect localizing an odor source, and a moth tracking a flower using vision. Our theory unifies the metabolic cost of motion with information theory. It predicts sense organ movements in animals and can prescribe sensor motion for robots to enhance performance.
Caenorhabditis elegans transfers across a gap under an electric field as dispersal behavior
This study reports that electrical interactions occur among different species!
Interactions between different animal species are a critical determinant of each species’ evolution and range expansion. Chemical, visual, and mechanical interactions have been abundantly reported, but the importance of electric interactions is not well understood. Here, we report the discovery that the nematode Caenorhabditis elegans transfers across electric fields to achieve phoretic attachment to insects. First, we found that dauer larvae of C. elegans nictating on a substrate in a Petri dish moved directly to the lid through the air due to the electrostatic force from the lid. To more systematically investigate the transfer behavior, we constructed an assay system with well-controlled electric fields: the worms flew up regardless of whether a positive or negative electric field was applied, suggesting that an induced charge within the worm is related to this transfer. The mean take-off speed is 0.86 m/s, and the worm flies up under an electric field exceeding 200 kV/m. This worm transfer occurs even when the worms form a nictation column composed of up to 100 worms; we term this behavior “multiworm transfer.” These observations led us to conclude that C. elegans can transfer and attach to the bumblebee Bombus terrestris, which was charged by rubbing with flower pollen in the lab. The charge on the bumblebee was measured with a coulomb-meter to be 806 pC, which was within the range of bumblebee charges and of the same order of flying insect charges observed in nature, suggesting that electrical interactions occur among different species.
Neuroscience
Decomposing neural circuit function into information processing primitives
Cognitive functions arise from the coordinated activity of many neural populations. It is challenging to measure how specific aspects of neural dynamics translate into operations of information processing, and, ultimately, cognitive functions. An obstacle is that simple circuit mechanisms -such as self-sustained or propagating activity and nonlinear summation of inputs- do not directly give rise to high-level functions. Nevertheless, they already implement simple transformations of the information carried by neural activity. Here, we propose that distinct neural circuit functions, such as stimulus representation, working memory or selective attention stem from different combinations and types of low-level manipulations of information, or information processing primitives. To test this hypothesis, we combine approaches from information theory with computational simulations of neural circuits involving interacting brain regions that emulate well-defined cognitive functions (canonic ring models and a large-scale connectome-based model). Specifically, we track the dynamics of information emergent from dynamic patterns of neural activity, using suitable quantitative metrics to detect where and when information is actively buffered ("active information storage"), transferred ("information transfer") or non-linearly merged ("information modification"), as possible modes of low-level processing. We find that neuronal subsets maintaining representations in working memory or performing attention-related gain modulation are signaled by their involvement in operations of information storage or modification, respectively. Thus, information dynamics metrics, beyond detecting which network units participate in cognitive processing, also promise to specify how and when they do it, i.e., through which type of primitive computation, a capability that may be exploited for the analysis of experimental recordings.
Human behavior
A synthesis of evidence for policy from behavioural science during COVID-19
Scientific evidence regularly guides policy decisions1, with behavioural science increasingly part of this process2. In April 2020, an influential paper3 proposed 19 policy recommendations (‘claims’) detailing how evidence from behavioural science could contribute to efforts to reduce impacts and end the COVID-19 pandemic. Here we assess 747 pandemic-related research articles that empirically investigated those claims. We report the scale of evidence and whether evidence supports them to indicate applicability for policymaking. Two independent teams, involving 72 reviewers, found evidence for 18 of 19 claims, with both teams finding evidence supporting 16 (89%) of those 18 claims. The strongest evidence supported claims that anticipated culture, polarization and misinformation would be associated with policy effectiveness. Claims suggesting trusted leaders and positive social norms increased adherence to behavioural interventions also had strong empirical support, as did appealing to social consensus or bipartisan agreement. Targeted language in messaging yielded mixed effects and there were no effects for highlighting individual benefits or protecting others. No available evidence existed to assess any distinct differences in effects between using the terms ‘physical distancing’ and ‘social distancing’. Analysis of 463 papers containing data showed generally large samples; 418 involved human participants with a mean of 16,848 (median of 1,699). That statistical power underscored improved suitability of behavioural science research for informing policy decisions. Furthermore, by implementing a standardized approach to evidence selection and synthesis, we amplify broader implications for advancing scientific evidence in policy formulation and prioritization.
Socioeconomic reorganization of communication and mobility networks in response to external shocks
In the context of crises, it is customary to examine specific behavioral traits individually. However, as we show, to comprehensively uncover the effects of nonpharmaceutical interventions, it is necessary to view across multiple behavioral dimensions. Here, we analyze mobile phone communication data to investigate the dynamics of network segregation patterns of the same set of people both in terms of mobility and of social communication during the initial wave of COVID-19 in Sierra Leone. Interestingly, we find opposite trends in the network segregation dynamics, characterized overall by simultaneous increase in mobility segregation and reduction in social network segregation. Our results underscore the significance of data-driven studies going beyond single-axis approaches to assess the impact of emergency policies.
Bio-inspired computing & Machine learning
Structural plasticity for neuromorphic networks with electropolymerized dendritic PEDOT connections
this strategy could reinforce the analogy with biology for implementing multiple computing mechanisms that are hard to realize with conventional hardware, such as long time constant during learning or representation of various traces of information for learning implementation
Neural networks are powerful tools for solving complex problems, but finding the right network topology for a given task remains an open question. Biology uses neurogenesis and structural plasticity to solve this problem. Advanced neural network algorithms are mostly relying on synaptic plasticity and learning. The main limitation in reconciling these two approaches is the lack of a viable hardware solution that could reproduce the bottom-up development of biological neural networks. Here, we show how the dendritic growth of PEDOT:PSS-based fibers through AC electropolymerization can implement structural plasticity during network development. We find that this strategy follows Hebbian principles and is able to define topologies that leverage better computing performances with sparse synaptic connectivity for solving non-trivial tasks. This approach is validated in software simulation, and offers up to 61% better network sparsity on classification and 50% in signal reconstruction tasks.
Brain organoid reservoir computing for artificial intelligence
Brain-inspired computing hardware aims to emulate the structure and working principles of the brain and could be used to address current limitations in artificial intelligence technologies. However, brain-inspired silicon chips are still limited in their ability to fully mimic brain function as most examples are built on digital electronic principles. Here we report an artificial intelligence hardware approach that uses adaptive reservoir computation of biological neural networks in a brain organoid. In this approach—which is termed Brainoware—computation is performed by sending and receiving information from the brain organoid using a high-density multielectrode array. By applying spatiotemporal electrical stimulation, nonlinear dynamics and fading memory properties are achieved, as well as unsupervised learning from training data by reshaping the organoid functional connectivity. We illustrate the practical potential of this technique by using it for speech recognition and nonlinear equation prediction in a reservoir computing framework.
Neural networks need the right representations of input data to learn. Here we ask how gradient-based learning shapes a fundamental property of representations in recurrent neural networks (RNNs)—their dimensionality. Through simulations and mathematical analysis, we show how gradient descent can lead RNNs to compress the dimensionality of their representations in a way that matches task demands during training while supporting generalization to unseen examples. This can require an expansion of dimensionality in early timesteps and compression in later ones, and strongly chaotic RNNs appear particularly adept at learning this balance. Beyond helping to elucidate the power of appropriately initialized artificial RNNs, this fact has implications for neurobiology as well. Neural circuits in the brain reveal both high variability associated with chaos and low-dimensional dynamical structures. Taken together, our findings show how simple gradient-based learning rules lead neural networks to solve tasks with robust representations that generalize to new cases.
How can modern science leverage machine-learning predictions in a statistically principled way? … This manuscript presents prediction-powered inference, a framework that achieves the best of both worlds: extracting information from the predictions of a high-throughput machine-learning system and guaranteeing statistical validity of the resulting conclusions
Prediction-powered inference is a framework for performing valid statistical inference when an experimental dataset is supplemented with predictions from a machine-learning system. The framework yields simple algorithms for computing provably valid confidence intervals for quantities such as means, quantiles, and linear and logistic regression coefficients without making any assumptions about the machine-learning algorithm that supplies the predictions. Furthermore, more accurate predictions translate to smaller confidence intervals. Prediction-powered inference could enable researchers to draw valid and more data-efficient conclusions using machine learning. The benefits of prediction-powered inference were demonstrated with datasets from proteomics, astronomy, genomics, remote sensing, census analysis, and ecology.
The applications are remarkable, featuring proteomics, galaxy classification, gene expression, deforestation analysis, health insurance, income analysis, plankton counting.
Why the simplest explanation isn’t always the best
Worth reading! Start from this paper:
Phantom oscillations in principal component analysis
Dimensionality reduction simplifies high-dimensional data into a small number of representative patterns. One dimensionality reduction method, principal component analysis (PCA), often selects oscillatory or U-shaped patterns, even when such patterns do not exist in the data. These oscillatory patterns are a mathematical consequence of the way PCA is computed rather than a unique property of the data. We show how two common properties of high-dimensional data can be misinterpreted when visualized in a small number of dimensions.
and then follow-up with the commentary: Why the simplest explanation isn’t always the best