A container for knowledge

A few months ago I gave a talk at @joinedgecity on what ‘kind of problem’ consciousness is. The core theme was “to have knowledge [about consciousness] we need a place to put it.”

Why do I post so much about consciousness? Because I think it’s important. We are building the future much more quickly than we’re sorting out which paths lead to good places. Consciousness research is a crucial source of clarity about what is possible & what is good.

But I think sometimes people sort of assume that the field is more established than it is — that somewhere there’s a Professional Committee of Very Serious Scientists (perhaps in lab coats) who are carefully tracking progress on understanding the philosophical foundations of consciousness: curating & aggregating knowledge, closing in on the answer, and who would certainly recognize & celebrate a solution to consciousness if they saw it. I can unequivocally say this is not the case.

The field really is pre-paradigmatic — everybody’s doing their best but most dialogue is people talking past each other, billionaires & PhDs hold roughly the same philosophical views as educated laymen, the academic politics are absolutely brutal to navigate, and there are flashes of deep insight scattered across papers, journals, & conferences but knowledge doesn’t really accrete. It’s alchemy, not chemistry.

One of the core bottlenecks in formal consciousness research is knowledge accretion: good ideas happen but are often lost (e.g. Helmholtz). I think about this in terms of “to have knowledge [about consciousness] we need a place to put it” — the theme of this talk. Another bottleneck is perseverance; one of the hardest things in the world is to care about something when people around you don’t.

My answer to both problems is finding better compressions. By reducing the mental RAM needed to store technical concepts we increase our effective working memory; by framing philosophy in simple language we make it intuitive. Alfred North Whitehead suggested “civilization advances by extending the number of important operations which we can perform without thinking about them”; philosophy does too.

With all that said, here’s my talk; I hope in retrospect people find it really obvious.

There’s no consensus on what consciousness is or how to study it. But with AI speeding up, “consciousness” is leaving the realm of idle philosophy & entering the realm of important policy unknown. Soon it will be urgent

Decades of confusion & neglect have led to the field sinking into learned helplessness. We need fresh & agentic ways of thinking that can antidote this. Themes I endorse:

Basic structure of my talk: seven ways to look at consciousness with fresh eyes & a walk-through of what each means

“Natural kinds” is a term derived from Aristotle; it’s an attempt to point at the fundamental building blocks of reality. Or as Plato put it, concepts that ‘carve reality at its joints’.

The study of consciousness is the reverse-engineering of qualia natural kinds

Understanding which domain consciousness naturally ‘lives in’ would greatly help clarify what the natural kinds of consciousness could be

The two most plausible domains for this are physics & computation. Each leads to very, very different sorts of answers

Physics is a great accounting system for what exists. If consciousness research wants to be good at this (and it should) it should take notes from physics

Something that I think people dramatically underestimate is how central “symmetry” is to physics

STV is a serious attempt to replace an enormous and complicated chunk of reasoning & logic (conventional approaches to understanding what pain & pleasure are, & why they’re so deeply linked with motivation) with something very simple. As I describe it in my 2023 paper:

>The Symmetry Theory of Valence is both aggressive and simple. It’s aggressive in that it purports to offer an exact, single-factor solution to one of the oldest mysteries in phenomenology, what makes some things feel better than others. It’s simple in that it fits in one sentence, merely combining three concepts: (1) information geometries of mind, (2) symmetry, and (3) valence. Each pairwise connection is already strong:

>• 1+2: The history of physics and mathematics gives us a strong prior to analyze information
geometries of mind in light of symmetry;
>• 1+3: By definition, information geometries of mind fully describe a mind’s valence;
>• 2+3: There’s a long cultural history of understanding pleasantness and beauty in terms of
symmetry.

>As with any novel theory, it would be strange if STV was true. But I hold that it would be even
more strange – there would be some deep violation of beauty somewhere – if it wasn’t.

https://x.com/johnsonmxe/status/1800919946710667453

https://opentheory.net/Qualia_Formalism_and_a_Symmetry_Theory_of_Valence.pdf

“Strong Monism” is the maximally opinionated take on “dual aspect monism” or Russell’s “neutral monism”. I contrast it with “Weak Monism”. You would rather be a Strong Monist than a Weak Monist, yes?

https://opentheory.net/2019/06/taking-monism-seriously/

The outgrowth of Strong Monism is that *if physics and consciousness derive from the same thing, we actually know a lot about consciousness* & mining physics for insight should be a primary heuristic

Weak & Strong Monism pay the same ontological costs; why not go with Strong?

I think Strong Monism is an incredibly optimistic frame because it both clearly defines what a “solution to consciousness” could be & offers hope that this could be years away, not millennia

The dirty secret of “physics” is there isn’t one theory of physics, there are many competing theories. How does this impact physics-based consciousness research?

I expect there to be as many formulations of consciousness as there are formulations of physics. This makes getting one’s bearings a little hard at first but it’ll pay dividends later since some formulations of physics are better at certain problems than others

A few of my favorite physical constructs:

“Emergence” is often used as a sort of magical evocation; life “emerges” from physics, consciousness “emerges” from complexity. This is not illuminating. But emergence can be a deeply useful technical frame

@erikphoel has a great formal method for this in “Causal Emergence 2.0”. His basic thesis: we can quantify the uncertainty about causes & effects across different scales; emergence happens when & insofar as macroscopic scales have more certainty vs micro

Chemistry is a fantastic emergent structure; a true jewel of science. People interested in understanding consciousness should study *why* chemistry works

Many fields have physics envy. Consciousness research should have chemistry envy & biology envy

I introduce the term “monist shadow” here; I hope to write more soon

A few words foreshadowing the next section & a new project I've been developing, working name “QPT”

It’s crucial to have treasure maps for research; without them we simply won’t find the important stuff. But constructing useful maps requires a bit of cleverness

It’s always hard to reify a research heuristic into a process that can be shared, extended, scaled. But here’s what I think will work to create treasure maps:

The overlap between consciousness research & AI will define much of the 2030s. I wrote extensively about AI consciousness in last year’s A Paradigm for AI Consciousness. It seems pretty pressing to figure out.
https://opentheory.net/2024/06/a-paradigm-for-ai-consciousness/

An equally pressing question is how one could use AI to help research consciousness. Imo the bottleneck is clarifying ‘what kind of problem’ this is. This talk is intended as both answer & prompt

I end my discussion of fresh lenses with a few slides on Wittgenstein & philosophy of language. This might only be super interesting for language nerds — but I expect that most theorists of consciousness are bottlenecked by failing to keep sufficiently careful track of ‘what kind of thing’ various things are. This leads them to use words imprecisely, and once we’ve crystallized intuitions into words it’s very difficult to go back to the raw felt-senses to debug confusion.

My intent today is not to speak directly about this, but to offer a down payment on how we might think about clarifying language (such as that about consciousness) in the future.

———
Wittgenstein argued that language is defined by use. The question is: what then? The LLM paradigm is one answer for how to take this idea seriously. Another answer would be to mathematically study the different kinds of ways language is used — if a language-game is a high-dimensional shape, what classes of shapes are out there?

I think there’ll be a lot of AI alpha in ‘prescriptive synthetic data’, i.e. using LLMs to *cohere* language games — finding what mathematical shape a language-game is *trying* to be and just making that jump. Can we do this to the way we speak about consciousness?

Karpathy has proposed thinking of LLMs as a new sort of computer that computes with natural language. I think philosophers need to grapple with this idea much more deeply than we have.

https://x.com/johnsonmxe/status/1751996688128999795

Anyway, recapping the seven proposals — consciousness research isn’t intractable. But neither is it trivial. We can make progress if we want to, but we do have to really want it

Finally, a quick recap of some of the things I’ve been working on. Thanks for reading. And a sincere thank you to @janineleger @timourxyz @jwmares without which this talk would have not happened

Summoning ghosts — fragments of avatar state — is unlikely to solve consciousness unless we find a dualism with physics

This talk is about how to build context for this dualism

@IntuitMachine seems to have a well-thought-out system for how LLMs can be used to explore the latent structure of theories & arguments. I suspect this could evolve into using mechanistic interpretability to make an LLM’s weights themselves the object of discussion.