Artificial intelligence is no longer just evolving nomenclature in IT. Everyone is taking interest. With the mainstream press and bloggers from every corner weighing in, it is worth taking stock of the nomenclature and learning how to differentiate three overly used key terms: artificial intelligence (AI), machine learning, and deep learning. The simplest way to think of their relationship is to visualize them as a concentric model (as depicted in the figure below) with AI — the idea that came first and has since been evolving — having the largest area. This is followed by machine learning, which blossomed later and is shown as a subset of AI. Inside both of those is deep learning, which is just one class of machine learning algorithms but one that is currently driving today’s AI explosion.
AI has been part of our thoughts and slowly evolving in academic research labs since a group of computer scientists first defined the term at the Dartmouth Conferences in 1956 and provided the genesis of the field. In the long decades since, AI has alternately been heralded as an all-encompassing Holy Grail and thrown into technology’s bit bucket as a mad conception of overactive academic imaginations. In reality, however, until around 2012, its reach was limited to advanced technological companies, governments, and research agencies, feeding into both perceptions.
Since then, AI has broken away from the hypothetical and into real-world business solutions. Much of them have to do with the wide availability of GPUs (graphics processing units), which make parallel processing ever faster, cheaper, and more powerful. The ascent of AI also has to do with the simultaneous one-two punch of practically infinite storage and a deluge of data of every stripe, including images, video, audio, text, transactions, and geospatial data.
Deep learning, a type of machine learning within the field of AI, has also caught the attention of business. Deep learning breaks down tasks in ways that make all kinds of machine assists seem possible — even likely. Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction.
Autonomous vehicles, enhanced preventive healthcare, even better movie recommendations are all here today or on the horizon because of deep learning. Both deep learning and AI are here to stay and will be pervasive in our everyday life.
The Relationships Between AI, Machine Learning, and Deep Learning
To understand these fields better, we first need to understand the relationship deep learning has with machine learning, neural networks, and artificial intelligence.
Generally speaking, deep learning is a more welcoming label for what’s been known as neural networks. “Deep” refers to the depth of the network, or the number of layers. A neural network can be very shallow with few layers. In the 1980s, most neural networks were a single layer because of the cost of computation and availability of data.
Neural networks are inspired by the structure of the human brain, specifically the cerebral cortex. At a basic level, the representation is that of a perceptron, the mathematical model of a biological neuron. As in the cerebral cortex, several layers of interconnected perceptrons can exist in the network.
One of the popular neural networks is the multilayer perceptron. The first layer is called the input layer. Each node in this layer accepts an input and then passes its output as the input to each node in the next layer. Generally, there are no connections between nodes in the same layer. The last layer produces the outputs. We call the middle part the “hidden layer.” These neurons have no connection to the outside (e.g. input or output) and are only activated by nodes in the previous layer. One way to think of deep learning is as the technique for learning in neural networks that uses multiple layers of abstraction to solve problems such as image and speech recognition, natural language understanding, sentiment analysis, language translation, and genomics.
Machine learning is thought of as a subset of AI, whereas deep learning is a specialized type of machine learning. Machine learning involves statistical learning that doesn’t know the answers up front. Instead, the algorithm will learn using training data, verify the success of its results, and adjust its approach accordingly. There are three general classes of machine learning: supervised learning, unsupervised learning, and reinforcement learning.
With supervised learning, an algorithm uses a labeled dataset to infer the desired results. A label is the data item you’re trying to predict. This process can require a large volume of data and resources since the data needs to be labeled manually.
Supervised learning is great for making predictions using classification and regression techniques. For example, let’s say that we are a mobile carrier and want to determine the churn rate of our customers. If we had historical data that described customer behavior along with churn status, we could use supervised machine learning.
In unsupervised learning, there aren’t any predefined or corresponding answers. The goal is to discover hidden patterns in the data. Unsupervised learning usually involves clustering techniques to group customers by behavior, or dimension reduction to transform data into a different form. While supervised learning can be useful, we often need to resort to unsupervised learning and then return to supervised learning based on what we discover.
Reinforcement learning differs from the two above in that it requires feedback but not labels. It is a computer program to learn a strategy (taking optimized actions) in an environment to maximize a certain reward, such as driving a vehicle or playing a game. It was the spotlight when Google’s AlphaGo – an AI program beat the 18-time world champion of the game of Go in 2016.
Almost all the value today surrounding deep learning is through supervised learning or learning from labeled data. One reason that deep learning has taken off so rapidly is that it is very good at supervised learning. The well-designed network structure of deep learning is able to learn features (this ability is called feature learning or feature representation) efficiently from labels. But scale is an important consideration with deep learning as well. As we construct larger neural networks and train them with increasingly more data, their performance continues to improve. This is generally different from other machine learning techniques that reach a plateau in performance. For most flavors of the old generations of learning algorithms, performance will plateau. Deep learning, on the other hand, is the first class of algorithms that is scalable and whose performance just keeps getting better as you feed the algorithms more data. This effect is visualized in the figure below.
Why Deep Learning
Source: Andrew Ng
Computers have long had methods for recognizing objects inside of images. Computer vision has been a primary beneficiary of deep learning and now rivals humans on many image recognition tasks.
Facebook has had great success with identifying faces in photographs by using deep learning. Its recognition accuracy is considered game-changing. Asked whether two facial photos show the same person, a human being previously unfamiliar with the faces will get it right 97.53% of the time. Software developed by researchers at Facebook can score 97.25% on the same experiment, regardless of variations such as the background, pose, lighting, and surrounding objects.
Speech recognition is another area that’s been impacted by deep learning. Spoken languages are so vast and ambiguous, but deep learning is able to handle this diversity. Baidu, a leading search engine in China, has developed a voice recognition system that is faster and more accurate than humans at producing text on a mobile phone in both Mandarin and English.
Google is now using deep learning to manage the energy at the company’s data centers. They’ve cut their energy needs for cooling by 40%. That translates to about a 15% improvement in power usage efficiency for the company and hundreds of millions of dollars in savings.
The Future of Deep Learning
Early on in the advance of deep learning, unsupervised methods had a catalytic effort in raising interest. Since then, it has been overshadowed by the success of purely supervised learning. Many believe, however, that unsupervised learning will become far more important in the long term. Human and animal learning is largely unsupervised as we discover the structure of the world by observing it, not by being told the name of every object.
Want to see artificial intelligence, machine learning, and deep learning in action? Download our white paper “Tactical Intelligence Derived from Social Media Sources,” which puts some of the principles discussed here to the test in the real world.
Qi Zhao is vice president of analytics at Opera Solutions.