Rethinking - and reframing - superintelligence - Harvard Gazette

"The debate over artificial superintelligence (ASI) can tend toward extremes, with predictions that it will either save humanity or destroy it. E. Glen Weyl has a different perspective: Superintelligence is already here, and it has been for thousands of years. Speaking Nov. 19 at the Berkman Klein Center, Weyl, an economist with Microsoft Research and co-author of the book "Radical Markets," urged listeners to think about superintelligence in the context of James Lovelock's "Gaia Hypothesis," i.e., as a collective self-regulating system that encompasses all living things and the natural environment."

""When we separate digital systems from people, we make them dangerous because they don't have the feedback to maintain homeostasis, and also they are not useful because they're not integrated into production processes and human participation." In fact, Weyl said, "Superintelligence is already all around us: corporations, democracies, religions, cultures - all of these things manifest capabilities that humans do not have on their own. We think of artificial superintelligence as something unique. Instead, I'm suggesting that it is another way that allows us to map human social relationships and thereby extend them.""

Superintelligence can be understood as a collective, self-regulating phenomenon embodied in systems like corporations, democracies, religions, and cultures that extend human capabilities. The Gaia framing treats superintelligence as an integrated system linking living things and the environment. Framing artificial superintelligence solely as isolated, autonomous machines leads to engineering choices that separate digital systems from people. Separated systems lack human feedback and homeostatic controls, making them both dangerous and unproductive. Integrating digital systems with human participation and production processes restores feedback, maintains stability, and leverages collective social relationships to augment capacity and cohesion.

Read at Harvard Gazette