DeepMind Gato and the Long, Uncertain Road To Artificial General Intelligence – The Wire Science

Photo: Possessed Photography/Unsplash


  • Last month, DeepMind, a subsidiary of tech giant Alphabet, rocked Silicon Valley when it announced Gato, arguably the most versatile AI model in existence.
  • To some computer experts, it’s proof that the industry is on the cusp of reaching a much-anticipated, much-hyped milestone: Artificial General Intelligence (AGI).
  • This would be huge for humanity. Think of everything you could achieve if you had a machine that could be physically adapted for any purpose.
  • But a host of experts and scientists have argued that something fundamental is missing from the grand plans to build Gato-like AI into full-fledged AGI machines.

Last month, DeepMind, a subsidiary of tech giant Alphabet, rocked Silicon Valley when it announced Gato, arguably the most versatile artificial intelligence model in existence. Billed as a “generalist agent”, Gato can perform over 600 different tasks. It can control a robot, caption images, identify objects in images, and more. It is probably the most advanced AI system in the world that is not dedicated to a single function. And for some computer experts, it’s proof that the industry is on the cusp of hitting a long-awaited, much-anticipated milestone: Artificial General Intelligence.

Unlike regular AI, artificial general intelligence (AGI) would not require massive amounts of data to learn a task. While ordinary artificial intelligence must be pre-trained or programmed to solve a specific set of problems, a general intelligence can learn through intuition and experience.

In theory, an AGI could learn everything a human being can do, if given the same access to information. If you place an AGI on a chip and then put that chip in a robot, the robot can learn to play tennis the same way you or I can: by swinging a racket and getting a feel for the game. That does not necessarily mean that the robot is conscious or capable of cognition. It wouldn’t have any thoughts or emotions, it would just be really good at learning how to do new tasks without human help.

This would be huge for humanity. Think of all you could achieve if you had a machine with the intellectual capacity of a human and the loyalty of a trusted canine companion – a machine that could be physically modified for any purpose. That is AGI’s promise. It’s C-3PO without the emotions, Lt Commander Data without the curiosity, and Rosey the Robot without the personality. In the hands of the right developers, it could embody the idea of ​​human-centric AI.

But how close is AGI’s dream really? And does Gato really bring us closer?

For one group of scientists and developers (I call this group the “Scaling-Uber-Everything” crowd, using a term coined by world-renowned AI expert Gary Marcus), Gato and similar systems based on transformer models of deep learning has already given us the blueprint for building AGI. Essentially, these transformers use massive databases and billions or trillions of adjustable parameters to predict what will happen in a sequence.

The Scaling-Uber-Everything crowd, including household names such as OpenAI’s Ilya Sutskever and the University of Texas at Austin’s Alex Dimakis, believe that transformers will inevitably lead to AGI; all that’s left is to make them bigger and faster. Like Nando de Freitas, a member of the team that created Gato, recently tweeted: “It’s all about scale now! The game is over! It’s about making these models bigger, more secure, more compute efficient, faster sampling, smarter memory…” De Freitas and the company understand that they need to create new algorithms and architectures to support this growth, but they also seem to believe that an AGI will come naturally as we continue to make models like Gato bigger.

Call me old-fashioned, but when a developer tells me he plans to wait for an AGI to magically emerge from the mass of big data like a mudfish from the primordial soup, I tend to think they’re taking a few steps to skip. Apparently I’m not the only one. A host of experts and scientists, including Marcus, have argued that something fundamental is missing from grandiose plans to build Gato-like AI into full-fledged, generally intelligent machines.

I recently explained my thinking in a trilogy of essays for The next web‘s Neural Vertical, where I’m an editor. In short, an important principle of AGI is that it must be able to obtain its own data. But deep learning models, such as transformer AIs, are little more than machines designed to make inferences about the databases already delivered to them. They are librarians and as such they are only as good as their training libraries.

A general intelligence could theoretically figure things out even if it had a small database. It would be intuitive for the methodology to accomplish its task based on nothing more than its ability to choose what external data was and was not important, like a human deciding where to focus his attention.

Gato is cool and nothing beats that. But essentially it’s a smart package that arguably presents the illusion of a general purpose AI through expert use of big data. For example, the massive database likely contains data sets built on the entire content of websites such as Reddit and Wikipedia. It’s amazing that people have been able to do so much with simple algorithms by forcing them to parse more data.

In fact, Gato is such an impressive way to fake general intelligence that I wonder if we might be barking at the wrong tree. Many of the tasks Gato is capable of today were once thought of as something only an AGI could do. It feels like the more we achieve with regular AI, the harder it seems to build a general agent.

For those reasons, I am skeptical that only deep learning is the way to AGI. I think we need more than bigger databases and extra parameters to tweak. We need a whole new conceptual approach to machine learning.

I think humanity will eventually succeed in the quest to build AGI. My best guess is that we’ll be knocking on AGI’s door sometime around the early to mid 2100s, and if we do, we’ll find that it looks very different from what DeepMind’s scientists envision.

But the beauty of science is that you have to show your work, and right now DeepMind is doing just that. It has every chance to prove me and the other naysayers wrong.

I really hope it works.

Tristan Greene is a futurist who believes in the power of human-centric technology. He is currently the editor of Neural, the futuristic vertical of The Next Web.

This article was first published by dark.

Leave a Comment

Your email address will not be published.