Open-Endedness, Multi-Agent Learning and Existential Risk

Describing Things in Different Ways

So we’ve gone very quickly from a new frontier of AI research (general AI and open-ended learning), back to a somewhat older field of math (game theory). But it’s “math on steroids” now, because we’re doing it on super-computers (as compared to what they had in the 1950s), and can thus represent far more complex games/environments, and build more sophisticated models of the agent as compared to the simplistic “homo economicus” paradigm. So it’s certainly time for a new iteration on this research.

(From DeepMind tutorial slides on Multi-Agent Learning.)

Perhaps a thing is simple if you can describe it fully in several different ways, without immediately knowing that you are describing the same thing. — Richard Feynman

Feynman said this during his Nobel Prize acceptance speech, about how there ended up being many equivalent ways to describe a quantum theory of electricity and magnetism. In that same speech, he mentions that as an undergraduate he read a book on this subject by Dirac, which ended with, “It seems that some essentially new physical ideas are here needed.” This strikes me as quite similar to this summary-slide from a recent DeepMind talk on multi-agent learning:

We don’t really know when or how to compose losses. There’s real thinking to be done.

Moving Beyond Objective Functions

We cannot use objective functions as a way of specifying all types of behavior that we might want to describe. Some behaviors may be so complex that it would just be hard to write down an objective function to describe them. For other behaviors, such an objective function simply does not exist.

Image borrowed from figure 4 of the alpha-Rank paper.

Societal Implications

There are many crucial questions we need to answer as a society. As our species evolves and advances, are all these adaptations ultimately positive and sustainable? Or are we pushing ourselves towards evolutionary collapse? Will capitalism end up with a few winners and the rest losers, who die off? Should we try to prevent climate catastrophe, or should we march right towards it — expecting that our economic and technological advancements will provide some solution (despite also being the cause!)?

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store