Dr. Rich Sutton

Throughout his career Dr. Richard Sutton has balanced pure and applied research in academia and in industry.

Dr. Richard Sutton is the principal founder of the field of the field of reinforcement learning. The ideas behind Sutton’s foundational work are intuitive, easy to implement, and have widespread applicability, including in seemingly disparate domains such as artificial intelligence, neuroscience, psychology, and control theory.

To get the complete picture of the impact of Sutton’s innovation, one must look to the large number of publications, research projects, and deployed applications in all of these areas.

The idea of having computers learn through reinforcement goes back to at least the 1950s (and is an important concept for how humans learn). However, not until Sutton’s PhD thesis (

Temporal Credit Assignment in Reinforcement Learning, 1984) and seminal paper (“Learning to Predict by the Method of Temporal Differences”, 1988) did the idea began to receive widespread attention. Tesauro (IBM) used Sutton’s ideas to build his super-human backgammon-playing program

(1992). This was the first major deployed application that demonstrated how powerful Sutton’s methods could be. Perhaps Rich Sutton’s tour de force is his 1998 textbook, Reinforcement Learning: An Introduction(co-authored by his PhD supervisor Andy Barto). This book has stood the test of time and continues to be heavily cited (27,086 as of January 7, 2018, according to Google Scholar), despite its age (18 years, an eon in the field of computing science). This book has helped educate multiple generations of researchers interested in machine learning. The book has been used in hundreds of undergraduate and graduate courses around the world. The second edition has recently been completed and will appear in print in a few months.

One measure of Sutton’s impact is to look at the number of citations to papers on reinforcement learning. During the 1980s there were a few dozen references per year (Google Scholar). Starting with the 1988 paper, interest in the area took off. By 1997, there averaged 1,000 papers per year and today it is roughly 7,000 per year. Lifetime, Sutton has a total of 56,332 citations and an h-index of 63 (January 7, 2018).

There are many real-world applications based on Rich Sutton’s ideas. These include computer game playing, robots learning to play soccer, teaching a four-legged robot to walk, controlling a helicopter in flight, routing vehicles to avoid traffic congestion, dialogue systems, and market trading systems.  DeepMind opened their first international office in Edmonton, in recognition of the pivotal role that reinforcement learning and Dr. Sutton’s research has played in their company’s success.

Prof. Sutton has been elected a Fellow of the Royal Society of Canada in 2016. In 2018, Prof. Sutton has been awarded with CAIAC Life Time Achievement Award and became Fellow of CAIAC.