AI research at Google

Last update: Aug 10, 2012

Google X Lab, also referred to as Google X or Google[x], is a secretive part of Google that experiments with ambitious future technologies. Its purpose is to develop products that have the potential to make a large positive impact on society. The full extent of the research is not public knowledge but it's thought to include projects relating to artificial intelligence (AI) and robotics.

The Google founders have long been interested in AI. One venture capitalist said that every time he discusses the future with Google CEO Larry Page, Larry argues that Google will eventually become an artificial intelligence. Indeed, Larry and co-founder Sergey Brin have both said that their ultimate aim is for Google Search to become AI-complete. That is, the machine will be just as intelligent as a human.

In 2006 an internal company memo said that Google wants to have the world's top AI research laboratory. It is possible that Google X Lab forms a part of this. AI is certainly a big part of Google in general. This is clear from the many AI-related jobs they have on offer.

Contents    (hide)

Latest news

Google image recognition

August 8, 2012 - Jeff Dean of Google reveals that he has been collaborating with the speech team at Google to use deep networks in speech recognition. He says that Google are currently applying these networks to other applications too, such as image recognition, language modeling, and machine translation. It is not clear whether this technology is already in production, or in development only. Deep networks, also known as deep belief networks, are a type of artificial neural network. Jeff Dean and colleagues at Google published a paper about their research in this area two months ago (see news entry below ↓). See also the Scholarpedia article about deep belief networks.

Google detected face

June 26, 2012 - Google reveals that they've built a 9-layer neural network that can learn to detect faces using only unlabeled images. The model has 1 billion connections and the dataset has 10 million 200x200 pixel images randomly downloaded from the internet. It was trained on a cluster of 1,000 machines (16,000 cores) for three days. The results show it's possible to train a face detector without having to label images as containing a face or not. The detector is robust to translation, scaling, and out-of-plane rotation. It's also sensitive to other high-level concepts such as cat faces and human bodies. The network obtained 15.8% accuracy in recognizing 20,000 object categories, a leap of 70% relative improvement over the previous state-of-the-art. See the article by the New York Times and the science paper on arXiv: arxiv.org/abs/1112.6209.

May 15, 2012 - Google has published two new photos from inside their data centers. These were revealed in the Story of Send animation about how Gmail is routed. We can see the servers are mounted on wheeled racks with 20 servers per rack. It is unknown whether this is the standard production configuration in all Google data centers. This has little to do with the X Lab directly, but the info helps us better estimate Google's total computational capacity, and thus how close Google is to creating a machine which that the computational power of a human brain.

Inside a Google data center
Inside a Google data center
Google server racks
Google server racks

Google X Lab history

Technology journalist MG Siegler claims he first heard about Google X in March 2011 although he never wrote about it at the time. The lab was first mentioned in public by a Business Insider article in June 2011. The only details given at that time were that the lab exists, that it does secret research into special projects, and that it's lead by Sergey Brin. Business Insider heard about it from journalist and Google expert Steven Levy as well as other undisclosed sources. The lab first entered mainstream public awareness via a New York Times article in November 2011. Since then various Google employees have openly acknowledged the lab's existence. Some list Google X as their employer on their Google profile pages (see the list below).

Google X Lab details

The following list of details comes from an article in the New York Times. They, in turn, were sourced from unnamed Google employees.

  • The lab is at an undisclosed location in the Bay Area, California
  • Two offices: logistics on the Mountain View campus, and the robotics lab at a secret location
  • Sergey Brin, Google's co-founder, is deeply involved in the lab, possibly as its head
  • Larry Page, Google's other co-founder, worked on Google X before becoming Google CEO in April 2011
  • Investment is very small compared to investments made in Google's core businesses
  • Google X Lab is run mysteriously, clandestine, secret even from most Google employees

Sebastian Thrun was interviewed by Charlie Rose in April 2012. At the start of the interview Sebastian is named as the founder and head of Google X. He says that the purpose of the lab is to work on "moonshot" ideas and turn them into products that have an impact on society. Top scientists and engineers are hired and given all the time and resources they need, free from pressure to publish research papers. Watch the video of the interview.

Projects

Project Glass - a research program to build an augmented reality head-mounted display. The first demo resembles a pair of normal eyeglasses where the lens is replaced by a head-up display. Product testing began in April 2012. It's been reported that the glasses could be available to the public by the end of 2012. They would cost about the same as current smartphones. See the Wikipedia article, the Google+ page, and the YouTube video.

Google driverless car

Driverless Car - this project is currently being led by Google engineer Sebastian Thrun, director of the Stanford Artificial Intelligence Laboratory and co-inventor of Google Street View. Thrun's team at Stanford created the robotic vehicle Stanley which won the 2005 DARPA Grand Challenge and its US$2 million prize from the United States Department of Defense. The team developing the system consisted of 15 engineers working for Google, including Chris Urmson, Mike Montemerlo, and Anthony Levandowski who had worked on the DARPA Grand and Urban Challenges. The U.S. state of Nevada passed a law allowing driverless cars. This went into effect on March 1, 2012, and the Nevada Department of Motor Vehicles issued the first license for a self-driven car in May 2012. Wikipedia article: Google driverless car

Google Assistant / Majel - a voice recognition and comprehension system to be used as a virtual assistant in Android phones. This is Google's equivalent to Apple's Siri. The project was known as "Majel" before March 2012. After that it became "Google Assistant". The product had been expected to launch as part of Android in Q4 2012. In fact, it may have been at least partially rolled into the Jelly Bean Android 4.1 release at the end of Q2 2012. Majel wasn't necessarily developed within Google X, it might simply have been a standard Google research project. Indeed, the delineation between X Lab and standard research is possibly blurred anyway. Wikipedia page: Assistant (Google).

Other products and research

According to the NY Times article, Larry and Sergey came up with a list of 100 shoot-for-the-stars ideas, including:

  • Robotic avatars, telepresence, for telecomuters and meetings
  • Projects involving the Web of Things
  • Web-connected light bulb communicates with Android - was said to be unveiled by end of 2011, but no sign of it as of August 2012
  • Space elevator - a long-held dream of the Google founders

People involved

Lab is filled with roboticists and electrical engineers, hired from places like Microsoft, Nokia Labs, Stanford, M.I.T., Carnegie Mellon, and New York University. They include:

  • Sebastian Thrun - a leader at Google[x], expert in robotics and artificial intelligence, teaches computer science at Stanford, involved in inventing the driverless car.
  • Andrew Ng - Stanford professor, specializes in applying neuroscience to artificial intelligence to teach robots and machines to operate like people.
  • Johnny Chung Lee - specialist in human-computer interaction, came to Google[x] from Microsoft in 2011 after helping develop Microsoft's Kinect, now working on Web of Things.
  • Steve Lee - Director at Google, lead product management for Google[x]. Previously led product management for Google mobile maps and location services.
  • Will Patrick - Product marketing.
  • Simon Prakash - Senior Director of Q&R at Google[x]. Worked for Apple prior to 2012.
  • Brandon Badger - Product manager at Google[x], previously involved in Google Books and YouTube Sports.
  • Erik Wolsheimer

Google founders' quotes on artificial intelligence

Larry Page
Larry Page, Google CEO and co-founder
Sergey Brin
Sergey Brin, Google co-founder

It has been well-known for many years that the Google founders, Larry Page and Sergey Brin, are very interested in artificial intelligence. They have often talked about making Google Search artificially intelligent so that it understands exactly what information you're seeking. They've also speculated on interfacing all of the world's information directly to your brain.

Google are also sponsors and founding partners of the Singularity University. In 2008 they helped set it up with more than $250,000 in donations (ref). Larry Page is also personally funding a research effort to reverse engineer the brain of a worm and recreate it as a detailed computer simulation.

Here are some quotes where the Google founders talk about artificial intelligence:

"Artificial intelligence would be the ultimate version of Google. The ultimate search engine that would understand everything on the Web. It would understand exactly what you wanted, and it would give you the right thing. We're nowhere near doing that now. However, we can get incrementally closer to that, and that is basically what we work on." - Larry Page, October 2000 (source)

"Google will fulfill its mission only when its search engine is AI-complete. You guys know what that means? That's artificial intelligence." - Larry Page, May 2002 (The Big Switch, page 212)

"The ultimate search engine would understand exactly what you wanted when you typed in a query, and it would give you the exact right thing back, in computer science we call that artificial intelligence. That means it would be smart, and we're a long way from having smart computers." - Larry Page, November 2002 (source)

"HAL had a lot of information, could piece it together, could rationalize it. Hopefully it would never have a bug like HAL did where he killed the occupants of the spaceship. But that [level of artificial intelligence] is what we're striving for, and I think we've made it a part of the way there." - Sergey Brin, November 2002 (source)

"If you had all the world's information directly attached to your brain, or an artificial brain that was smarter than your brain, you'd be better off." - Sergey Brin, 2004 (source)

"Every time I talk about Google's future with Larry Page, he argues that it will become an artificial intelligence." - Steve Jurvetson, January 2005 (source)

"We are not scanning all those books to be read by people, we are scanning them to be read by an AI." - Unidentified Google engineer, October 2005 (source)

"One of our big goals in search is to make search that really understands exactly what you want, understands everything in the world. As computer scientists, we call that artificial intelligence." - Larry Page, October 2005 (source)

"The ultimate search engine would understand everything in the world. It would understand everything that you asked it and give you back the exact right thing instantly. You could ask 'what should I ask Larry?' and it would tell you." - Larry Page, May 2006 (source)

"People always make the assumption that we're done with search. That's very far from the case. We're probably only 5% of the way there. We want to create the ultimate search engine that can understand anything. Some people could call that artificial intelligence." - May 2006 (source)

"Google wants to be the best in search. To reach that goal Google wants to have the world's top AI research laboratory." - Google internal company paper, October 2006 (source)

"One of my favourite things is artificial intelligence, but it has gotten a very bad rap, but my prediction is that when AI happens it's going to be a lot of computation and not so much clever algorithms but just a lot of computation. My theory is that if you look at your programming, your DNA, it's about 600 megabytes compressed, so it's smaller than any modern operating system, smaller than Linux or Windows or anything like that, your whole operating system, that includes booting up your brain. So your program algorithms probably aren't that complicated, it's probably more about the overall computation. We have some people at Google who are trying to build artificial intelligence and to do it on a large scale to make search better. Very few [other] people are working on this, and I don't think it's as far off as people think." - Larry Page, February 2007 (source)

Google's computational power vs. a human brain

Google 40 petaflops

Larry Page has been quoted as saying that "when artificial intelligence happens it will be not so much clever algorithms but just a lot of computation". So how much computational power does Google have, including all of its servers worldwide? Google keeps this information secret, but we can make some estimates.

As of January 2012 it was estimated that Google has around 1.8 million servers housed in its data centers worldwide. See the analysis here: Google data centers and server count

The computational power of each server is also not explicitly revealed by Google. But in a paper released in 2009 it was suggested that each server contained two dual-core processors. We don't know what the processor specifications are, but we do know that Google tends to use cheap, low-performance hardware and lots of it. If we make an educated guess that each processor runs at 2 GHz, and we know the number of processors per server, and the number of servers, then we can make an estimate of the total computational power.

It turns out that the Google platform has more computational power than the world's largest supercomputer. A lot more. This is a fair comparison too because Google does in fact view their platform not as a collection of data centers, but as a single supercomputer. The total power of the platform, however, is not yet sufficient to run a complete human brain simulation - supposing that the same simulation architecture were used as in the Blue Brain Project.

The Google platform continues to grow, however. It is thought that they are adding another 500,000 servers in 2012. The system is designed to be able to cope with a total of 10 million servers. In a few years time the Google platform should be sufficiently powerful to run a complete simulation of a human brain, including displays of intelligence and self-awareness.

Google AI robot leaked/hoaxed via Reddit

On December 5, 2011, someone posted anonymously to reddit claiming to be a disgruntled former employee of the Google X Lab. See the discussion thread: I recently left Google X. The original post by J32PMXR was deleted after two hours, but here's a saved copy of what they wrote:

reddit and Google X Lab

"This is in total violation of the NDA, but I don't care anymore. Sue me.

The central focus of Google X for the past few years has been a highly advanced artificial intelligence robot that leverages the underlying technology of many popular Google programs. As of October [2011] (the last time I was around the project), the artificial intelligence had passed the Turing Test 93% of the time via an hour long IM style conversation. IM was chosen to isolate the AI from the speech synthesizer and physical packaging of the robot.

The robot itself isn't particularly advanced because the focus was not on mechanics, but rather the software. It is basically a robotish looking thing on wheels. Speech recognition is somewhat better than what you would get with normal speech input, mostly because of the use of high quality microphones and lip-reading assistance.

I have had the chance to interact with the robot personally and it is honestly the most amazing thing that I have ever seen. I like to think of it like Stephen Hawking because it is extremely smart and you can interact with it naturally, but it is incapable of physically doing much. There is a planned phase two for development of an advanced robotics platform."

Follow up comments by J32PMXR say the robot has a suite of sensors including optical, laser, infrared, ultrasonic, and depth cameras. It can supposedly lip read, although that might be restricted to detecting emotion such as a smile. Most of the processing is done onboard, internet is used for external information. A stated goal is to enable intelligent conversation with your mobile phone, similar to Apple's Siri but more advanced.

Replies to the posting express skepticsm, but they also admit that Google is likely working in a similar direction. Another former Google employee replies that the idea of using a robot is non-Googly. They say Google would approach AI via statistical methods and by throwing massive computing power at the problem. Larry Page has hinted at this approach himself, see the quotes above.

Concensus on the reddit discussion is that this was probably a hoax. It's possible/probable that Google does indeed have some kind of AI-enabled robot in its lab. But the chances seem very low that a group of staff were fired and that one of them posted about it in public.

Solve for X

Solve for X logo

Solve for X is a forum to encourage and amplify technology-based moonshot thinking and teamwork. The forum was announced by Google in February 2012. Initially, much of the blogosphere and mainstream media reported this as being the same as Google X. Although the two projects are obviously related insofar as they both concern high technology, are both sponsored by Google, and both contain the letter "X" in their name, they're not actually the same thing. See the forum's official website and Google+ page.