Here at BETSOL, we’re big on machine learning, so it’s always exciting to see a great consumer-oriented application of it hit the mainstream. This post will dive into the machine learning technology you can use right now in Gmail’s “Smart Reply” feature.
In case you haven’t seen it when you read emails in a Gmail app, you are offered three auto-reply options. At first glance, these might seem like basic canned responses. But as you start using them, you realize there is something much more sophisticated going on.
Upon choosing a Smart Reply, Gmail starts a reply email and auto-inserts your selected text. You’re free to adjust the text, add to it, or just simply hit Send.
After using it for a while, the Auto Reply begins to improve. They will pick up your personal writing style and voice. Are you a bubbly person who likes to use exclamation marks? Gmail will catch on.
(source: https://www.blog.google/products/gmail/save-time-with-smart-reply-in-gmail/)
This technology first appeared in Google’s Inbox app. It quickly became responsible for 12% of all replies, which is incredible (Google Blog).
Auto-Reply Technology
Typically rules-based programming would simply be insufficient to deal with the complexities and subtleties of communication, which is why machine learning becomes the requisite technology to something as seemingly simple as creating a short email response.
Google utilizes a “Deep Neural Network” to generate the email replies.
These systems generalize better, and handle completely new inputs more gracefully than brittle, rule-based systems ever could.
Specifically, Smart Reply utilizes one neural network to break down and encode the received email. A separate network predicts responses.
Machine Learning Vectors
The goal of these networks is to identify and create “vectors” for communication. Rather than specifically handling each individual word, the systems come up with a direction (vector) to understand the intent. In this way, “how are you doing,” “how’s it going,” and “doing well” fall along a similar vector. They have the same intent.
The sender’s intent can then be matched up with a matching intent vector that’s put together in your unique style.
This vector, which Geoff Hinton calls a “thought vector,” captures the gist of what is being said without getting hung up on diction — for example, the vector for “Are you free tomorrow?” should be similar to the vector for “Does tomorrow work for you?” (Google Research)
(source: https://research.googleblog.com/2015/11/computer-respond-to-this-email.html)
What’s most exciting is that these encoders and vectors were learned, not programmed.
This leads me to a few questions…
Is Google learning me or am I settling for Google?
The more I’ve used smart replies, the less I’m sure who is really responsible for the language. Sure, Google has picked up on my style and types of responses. But many times, I’ve settled for a response that is about right, but not exactly what I would have sent.
So it appears my replies are becoming some merged “Justin/Google” voice and style.
Calling the mobile OS “Android” seems more and more fitting every day.
Where does the Gmail machine learning lead?
As this technology progresses, you can easily imagine a Smart Reply sentence becoming a paragraph. Or a detailed reply. Taken to its logical conclusion, will I even need to be part of the communication process?
Want to learn more?
- Deep Learning: https://en.wikipedia.org/wiki/Deep_learning
- Smart Reply Research: https://research.googleblog.com/2017/05/efficient-smart-reply-now-for-gmail.html
- Enterprise Machine Learning for Project Estimates: https://www.betsol.com/2017/05/machine-learning-accurately-estimate-it-project/
Tags:
