AIAny

Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton

Measures why complexity in closed systems rises then falls while entropy only climbs, using a coffee-and-cream cellular automaton. The key result: only interacting particles produce a transient complexity peak; non-interacting ones never do.

Introduction

Entropy in a closed system only ever goes up, yet the universe was structureless at the Big Bang and will be structureless again once everything disperses — so "interestingness" clearly behaves differently from entropy. This paper takes that vague intuition and tries to pin a number on it, asking a question most thermodynamics treatments dodge: what exactly rises and then falls as a system equilibrates?

Key Findings
  • Apparent complexity, defined operationally. The authors model mixing coffee and cream as a 2D cellular automaton and measure the Kolmogorov complexity of a coarse-grained (blurred) snapshot of its state — a quantity they name "apparent complexity," distinct from the entropy that just keeps climbing.
  • Interaction is the whole story. Analytically, non-interacting particles never reach high apparent complexity; the tendrils-and-swirls phase only appears when particles interact. Structure is not a free byproduct of mixing — it requires coupling.
  • A complexity bump, not a plateau. Numerically, the interacting system's apparent complexity peaks at a value comparable to the width of the "coffee cup" before collapsing as equilibrium arrives, reproducing the rise-and-fall curve the paper set out to capture.
  • An open challenge, stated honestly. Proving the observed peaking behavior analytically is left as an explicit open problem, not papered over.
How It Works

Coarse-graining is the conceptual pivot: at full resolution a mixed state is as random as an unmixed one, so complexity must be measured on a blurred view. That blurring is what makes "the cream is half-swirled" register as more complex than either pure separation or uniform brown — a small modeling choice that does most of the work.

Great Fit / When to Skip

Great fit if you care about defining and quantifying complexity, self-organization, or the arrow of time, and want a concrete toy model rather than hand-waving. Look elsewhere if you came for a finished theorem — the central behavior is shown numerically and posed as a conjecture, not proven — or if you expected a machine-learning method: despite living in an AI directory, this is a physics-and-information-theory paper at heart.

Information

  • Websitear5iv.labs.arxiv.org
  • OrganizationsMIT, Caltech
  • AuthorsScott Aaronson, Sean M. Carroll, Lauren Ouellette
  • Published date2014/05/27

More Items

Systematically studies how language and vision interact during unified multimodal pretraining, identifies mechanisms that enable modality synergy versus competition, demonstrates the benefit of early joint training, and derives efficient pretraining recipes validated at scale.

Presents Metis, a prototype memory foundation model that embeds a persistent native memory state into the backbone so historical experience is compressed and accessed via memory attention. Key features: forward-only, gradient-free online memory updates; memory-specific mid-training objectives; and a dual text/code memory design.

Empirically studies how transformer-based native multimodal pre-training scales under fixed compute, deriving compute- and data-allocation power laws and an efficiency frontier for model size, token count, and data mixture; evaluates cross-modal transfer and multimodal in-context learning.