Profis

Approx. 0

Assumes 6hrs/wk (work at your own pace)

Built by
Join thousands of students

Start Free Course

Start Free Course
Free
You get
Instructor videos
Learn by doing exercises and view PROJEKT instructions

Course Summary

You should take this course if you have an interest in machine learning and the desire to engage with it from a theoretical perspective. Through a combination of classic papers and more recent work, you will explore automated decision-making from a computer-science perspective. You will examine efficient algorithms, where they exist, for single-agent and multi-agent planning as well as approaches to learning near-optimal decisions from experience. At the end of the course, you will replicate a result from a published paper in reinforcement learning.

Why Take This Course?

This course will prepare you to participate in the reinforcement learning research community. You will also have the opportunity to learn from two of the foremost experts in this field of research, Profs. Charles Isbell and Michael Littman.

Prerequisites and Requirements

Before taking this course, you should have taken a graduate-level machine-learning course and should have had some exposure to reinforcement learning from a previous course or seminar in computer science (students who have completed CS 7641 will be well prepared for this course).

Additionally, you will be programming extensively in Java during this course. If you are not familiar with Java, we recommend you review Udacity's Intro to Java Programming course materials to get up to speed beforehand.

See the Technology Requirements for using Udacity.

What Will I Learn?

Projects

P4: Train a Smartcab to Drive

A smartcab is a self-driving car from the not-so-distant future that ferries people from one arbitrary location to another. In this project, you will use reinforcement learning to train a smartcab how to drive.

Syllabus

  • Reinforcement Learning Basics
  • Introduction to BURLAP
  • TD Lambda
  • Convergence of Value and Policy Iteration
  • Reward Shaping
  • Exploration
  • Generalization
  • Partially Observable MDPs
  • Options
  • Topics in Game Theory
  • Further Topics in RL Models

Instructors & Partners

instructor photo

Charles Isbell

Charles Isbell is a Professor and Senior Associate Dean at the School of Interactive Computing at Georgia Tech. His research passion is artificial intelligence, particularly on building autonomous agents that must live and interact with large numbers of other intelligent agents, some of whom may be human. Lately, he has turned his energies toward adaptive modeling, especially activity discovery (as distinct from activity recognition), scalable coordination, and development environments that support the rapid prototyping of adaptive agents. He is developing adaptive programming languages, and trying to understand what it means to bring machine learning tools to non-expert authors, designers and developers. He sometimes interacts with the physical world through racquetball, weight-lifting and Ultimate Frisbee.

instructor photo

Michael Littman

Michael Littman is a Professor of Computer Science at Brown University. He also teaches Udacity’s Algorithms course (CS215) on crunching social networks. Prior to joining Brown in 2012, he led the Rutgers Laboratory for Real-Life Reinforcement Learning (RL3) at Rutgers, where he served as the Computer Science Department Chair from 2009-2012. He is a Fellow of the Association for the Advancement of Artificial Intelligence (AAAI), served as program chair for AAAI's 2013 conference and the International Conference on Machine Learning in 2009, and received university-level teaching awards at both Duke and Rutgers. Charles Isbell taught him about racquetball, weight-lifting and Ultimate Frisbee, but he's not that great at any of them. He's pretty good at singing and juggling, though.

instructor photo

Chris Pryby

Chris Pryby is a Course Developer for the Georgia Tech Online Master's Degree program and is passionate about working to build the future of online education. He has bachelor's degrees from UGA in mathematics and computer science and a doctorate in mathematics from Georgia Tech. Chris has also been active in martial arts since 2011, and he holds a first-degree black belt in hapkido.