These are only a few examples demonstrating that the best A.I. programs can be unreliable when faced with situations that differ, even to a small degree, from what they have been trained on. The errors made by such systems range from harmless and humorous to potentially disastrous: imagine, for example, an airport security system that won’t let you board your flight because your face is confused with that of a criminal, or a self-driving car that, because of unusual lighting conditions, fails to notice that you are about to cross the street.
Even more worrisome are recent demonstrations of the vulnerability of A.I. systems to so-called adversarial examples. In these, a malevolent hacker can make specific changes to images, sound waves or text documents that while imperceptible or irrelevant to humans will cause a program to make potentially catastrophic errors.
The possibility of such attacks has been demonstrated in nearly every application domain of A.I., including computer vision, medical image processing, speech recognition and language processing. Numerous studies have demonstrated the ease with which hackers could, in principle, fool face- and object-recognition systems with specific minuscule changes to images, put inconspicuous stickers on a stop sign to make a self-driving car’s vision system mistake it for a yield sign or modify an audio signal so that it sounds like background music to a human but instructs a Siri or Alexa system to perform a silent command.
These potential vulnerabilities illustrate the ways in which current progress in A.I. is stymied by the barrier of meaning. Anyone who works with A.I. systems knows that behind the facade of humanlike visual abilities, linguistic fluency and game-playing prowess, these programs do not — in any humanlike way — understand the inputs they process or the outputs they produce. The lack of such understanding renders these programs susceptible to unexpected errors and undetectable attacks.
What would be required to surmount this barrier, to give machines the ability to more deeply understand the situations they face, rather than have them rely on shallow features? To find the answer, we need to look to the study of human cognition.
Our own understanding of the situations we encounter is grounded in broad, intuitive “common-sense knowledge” about how the world works, and about the goals, motivations and likely behavior of other living creatures, particularly other humans. Additionally, our understanding of the world relies on our core abilities to generalize what we know, to form abstract concepts, and to make analogies — in short, to flexibly adapt our concepts to new situations. Researchers have been experimenting for decades with methods for imbuing A.I. systems with intuitive common sense and robust humanlike generalization abilities, but there has been little progress in this very difficult endeavor.
A.I. programs that lack common sense and other key aspects of human understanding are increasingly being deployed for real-world applications. While some people are worried about “superintelligent” A.I., the most dangerous aspect of A.I. systems is that we will trust them too much and give them too much autonomy while not being fully aware of their limitations. As the A.I. researcher Pedro Domingos noted in his book “The Master Algorithm,” “People worry that computers will get too smart and take over the world, but the real problem is that they’re too stupid and they’ve already taken over the world.”
The race to commercialize A.I. has put enormous pressure on researchers to produce systems that work “well enough” on narrow tasks. But ultimately, the goal of developing trustworthy A.I. will require a deeper investigation into our own remarkable abilities and new insights into the cognitive mechanisms we ourselves use to reliably and robustly understand the world. Unlocking A.I.’s barrier of meaning is likely to require a step backward for the field, away from ever bigger networks and data collections, and back to the field’s roots as an interdisciplinary science studying the most challenging of scientific problems: the nature of intelligence.
Melanie Mitchell is Professor of Computer Science at Portland State University and External Professor at the Santa Fe Institute. Her book, “Artificial Intelligence: A Guide for Thinking Humans,” will be published in 2019 by Farrar, Straus, and Giroux.