Run Towards Fully-General Tech Companies

They Just Might Save Us

While the rise of Autonomous AGI (AAGI) could be a very dangerous scenario / arms race, there’s a step before AAGI (and even orthogonal to it), which is the use of simpler AI (whether human-level AGI, or even narrow AI) in a controlled, non-autonomous way. Companies which possess such controlled AI would be able to point this capability in any direction they choose. This could produce so-called “fully-general tech companies,” as described in Andrew Critch’s Tech company singularities:

  • A fully general tech company is a technology company with the ability to become a world-leader in essentially any industry sector, given the choice to do so — in the form of agreement among its Board and CEO — with around one year of effort following the choice.

Once we have such powerful technological capabilities within companies, and a newcomer company can become the leader in any sector it chooses within 1 year, my hope is that we might create/imagine new sectors besides the ones we current.

One such sector I’d like to see us create, is a market for new market mechanisms / institutions. [Alternate wording: One such capability would be an ability to discover and implement new Radical Markets.] Take the proposals from Posner and Weyl’s Radical Markets. How do we achieve these?

What are areas where we need new markets?

  • Energy grids
  • Carbon credits
  • Real estate / land usage
  • Paying individuals for their data / attention online
  • Pandemic restrictions
    - compensating businesses that close / limit service during critical periods in order to stop pandemic spread
    - even compensating individuals to stay home / limit contact.
    - could also include a “cost of privacy”, where individuals retain privacy by default (there’s a lot that can be done with aggregate / differentially-private data), but one could get paid extra if they volunteer to sacrifice a bit of privacy to help the network get some critical data)
  • Prediction / Decision Markets
    - Should be easier to bet on any event where you can concretely point to an outcome

Creating a new market is a difficult job today. But in the world of fully-general tech companies, one could imagine AI tools that identify the societal need for particular market mechanisms, advocate for any public policy needed for such markets to take shape (see the AI Policy-Maker), implement the software for such a market to run, build interfaces for customers/users to interact with the market, etc.

Take any issue that we see as a societal problem, and I’ll ask, “where is the market for it?” If it’s indeed a problem, then the claim is that society would be more prosperous overall if we solved it — i.e. there’s a utilitarian advantage to be had. Often, it’s easy enough to see how it’s possible with current technology. So if it’s physically/technically possible, and would bring net benefit to society, why aren’t we doing it? The answer is basically something of the form “the right people/organizations aren’t being incentivized enough,” “they’re being dis-incentivized/blocked,” etc. We’re leaving money on the table in so many places by not fixing this! So, being able to identify situations where this is happening, and come up with mechanisms for trade to close the gap, would be the ultimate super-power for a fully-general tech company to have!

The prospect that AI could do this for us is a compelling one. And it’s a sign that we should be running — sprinting — towards human-level AI that can solve arbitrarily complex intellectual tasks. Things like reading huge volumes of knowledge, designing market structures aimed to have some positive effect on society, writing (or better, negotiating) legislation to achieve the needed regulatory structures (along with explanations / justifications of its proposals), writing software to run the actual mechanics of markets, etc. If AI could do all these things, then a fully-general tech company could point all those tools at a particular problem it cares about and start making impact!

Andrew Critch’s main hope is that we reach an age of fully-general tech companies (FGTCs) before we reach AGI or AAGI — and then we have a chance to regulate those companies (or convince them in other ways) not to build AAGI. My hope is that we also use those companies to solve major world problems — to design not only the “purely technological” solutions, but the social ones as well, citing again the Radical Markets idea:

  • that economic and political systems are systems/technologies like any other, to be built and improved over time.
  • seeing economic and political systems less as ideologies where we must find the “right” one, and more as technologies to be built and improved over time



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store