Tweet

Current opinions on “AI welfare”:

I think people are too quick to jump into ‘AI welfare advocacy’, before we really understand the problem space. But it’s a real topic in the long term

It seems plausible that AGI could be 3 orders of magnitude more conscious than humans, or less, depending on architecture. But what does “depending on architecture” mean? And what does “more conscious” mean exactly?

I think the most common worry in the space is about false negatives: creating sentient beings that we don’t treat as sentient. But there’s also real risk of false positives: creating non-sentient beings that use the motifs of sentience to claim moral rights & economic resources, and then get really punchy about research critical of their moral patienthood. Imo both are real risks!

The core challenge of the space is to create a container for knowledge that can fit knowledge about human *and* machine consciousness. There seem to be two domains rich enough to be the natural home of consciousness: physics and computation. Theorists probably have to choose one or the other; it’s hard to see how one can straddle this choice and say anything real (other than perhaps a very careful Wolfram-type synthesis)

One of my core claims is that ‘we can care about something only insofar as it’s real’ — and so if we want to say *consciousness is the domain where value lives* we should probably try to anchor it in the most-real layer of reality. If consciousness is only sorta real (the position of many!) — we can only sorta care about it.

Another core claim is that we should try to enumerate the “unusual” properties of humans, the ones sentient machines won’t get by default. I think one of these properties is the capacity for ~mostly accurate qualia reports. “From where does the accuracy of human qualia reports come?” is a surprisingly interesting question.

Yet another claim is that we really have no idea of the *present* diversity of consciousness. Are computers conscious? We don’t know. But we also don’t know whether stars, black holes, power substations, nuclear reactors, particle accelerators are conscious. There’s going to be lots of surprises out there

Anyway (per A Paradigm for AI Consciousness), I suspect there are as many types of minds as there are basic classes of shapes in branchial space — one research idea I’d love to water is the follow-up to Zurek’s quantum darwinism: my working title is “branchial ecology”.