Carview!

HOME
ABOUT
- RESULTS
- differences
- BENEFITS
- HISTORY
- TEAM
- LOCATION
- FACILITIES
- BANKING
- MEMBERSHIPS
- APPROVALS
- LICENCES
- SUPPLIERS
- SPONSORSHIPS
- MEDIA
- PRIVACY
AUCTIONS
SHIPPING
FEES
- TS REWARDS
TOOLS
guides
FAQ
CONTACT
- CONNECT

VEHICLES
BRAND
- JAPANESE CARS
  - DAIHATSU
  - EUNOS
  - FORD
  - HONDA
  - ISUZU
  - LEXUS
  - MAZDA
  - MITSUBISHI
  - MITSUOKA
  - NISSAN
  - SUBARU
  - SUZUKI
  - TOYOTA
- GERMAN CARS
- AMERICAN CARS
- BRITISH CARS
- ITALIAN CARS
- FRENCH CARS
- SWEDISH CARS
- KOREAN CARS
TYPE
- mobility
- VENDING
- instruction
- TAXIS
- AMBULANCES
- FIRE ENGINES
- HEARSES
- LIMOUSINES
- COMMERCIAL
CLASS
FUEL
TRUCKS
minitrucks
- DAIHATSU
- HONDA
- MAZDA
- MITSUBISHI
- NISSAN
- SUBARU
- SUZUKI
- DUMP
- CRANE
- CAMPER
- REFRIGERATED
- 4WD
- NEW
BUSES
MOTORHOMES
- YAHOO!
- RAKUTEN
- DEALER

PARTS
- FREE REPORT
- PARTS CONTAINERS
- PARTS SYSTEMS
- PARTS PROTECTION
- BODY SHELLS
- DISMANTLING
- ONLINE PARTS
- NEW PARTS
- INTERIOR PARTS
- EXTERIOR PARTS
  - BONNETS
  - BUMPERS
  - GRILLES
  - FENDERS
  - DOORS
  - TRUNKS
  - SPOILERS
  - LIGHTS
  - EMBLEMS
  - CAMERAS
- ENGINES
- TRANSMISSIONS
- WHEELS & TYRES
  - WHEELS
  - TYRES
CUTS
PERFORMANCE PARTS
TRUCK PARTS
MOTORBIKE PARTS
- MOTORBIKE ENGINES
- MOTORBIKE ACCESSORIES

MOTORBIKES
MARINE
FORKLIFTS
MACHINERY
AGRICULTURAL
OTHER
COUNTRY
- AUSTRALIA
- CANADA
- KENYA
- MYANMAR
- NEW ZEALAND
- PAKISTAN
- TANZANIA
- UNITED STATES

CARVIEW

MOTORHOMES

Select Language

HTTP/2 200 server: GitHub.com content-type: text/html; charset=utf-8 last-modified: Sat, 13 Sep 2025 21:52:19 GMT access-control-allow-origin: * etag: W/"68c5e793-16f33" expires: Mon, 29 Dec 2025 19:13:20 GMT cache-control: max-age=600 content-encoding: gzip x-proxy-cache: MISS x-github-request-id: CADA:234FE9:94A991:A69D2B:6952D077 accept-ranges: bytes age: 0 date: Mon, 29 Dec 2025 19:03:20 GMT via: 1.1 varnish x-served-by: cache-bom-vanm7210023-BOM x-cache: MISS x-cache-hits: 0 x-timer: S1767035000.972915,VS0,VE205 vary: Accept-Encoding x-fastly-request-id: 8f9b6ef79942f8704955055bf9a71be13d37b47e content-length: 18536 Home

Home
Media
Research
Papers
Talks
Students
Service

Khimya Khetarpal

I am a Research Scientist at Google Deepmind and an Affiliate Faculty Member of (Mila). I earned my Ph.D. in Computer Science from Reasoning and Learning Lab at McGill University and Mila, where I was advised by Doina Precup. I am broadly interested in artificial intelligence and reinforcement learning.

Research Summary: I am interested in the capability of AI agents to understand and develop broadly intelligent behavior. My research focuses on how agents can efficiently represent the world's knowledge, plan with it, and adapt to changes over time through learning and interaction. See my research page for more details.

Email / CV / GitHub / Google Scholar / Twitter / YouTube

DeepMind December 2022 - Now

Research Scientist
Microsoft Research Fall 2021, Winter 2022

Research Intern [Host: Katja Hofmann, Harm van Seijen]
DeepMind Summer 2021

Research Scientist Intern [Host: Satinder Singh, Tom Zahavy]
DeepMind Fall 2019

Research Scientist Intern [Host: Doina Precup]
McGill University 2017 - Now

Ph.D. in Computer Science
Intel 2016 - 2017

Perceptual Computing Engineer
Univerity of Florida 2014 - 2016

Masters in Computer Engineering
IIT Kanpur 2013 - 2014

Research Associate
Robert Bosch 2011-2012

Embedded Software Development
VIT University 2007 - 2011

B.Tech in Electronics & Communication Engineering

Highlights and News

5/08/2025 Invited Talk on Navigating the Affordance Landscape for Continual Agent Adaptation for CoLLAs Early-Career Spotlight Program.

26/04/2025 New blog post on New in ML: A Guide for Navigating the ML Conference Scene.

21/04/2025 Long Range Navigator (LRN): Extending robot planning horizons beyond metric maps to appear at CoRL (2025) . LRN presents a method that extends planning horizon using an affordance representation mapping camera images to `affordable' frontiers, and optimizing for goal alignment.

21/01/2025 Cracking the Code of Action: A Generative Approach to Affordances for Reinforcement Learning to appear at ICLR Third Workshop on Deep Learning for Code.

21/01/2025 A Unifying Framework for Action-Conditional Self-Predictive Reinforcement Learning to appear at AISTATS International Conference on Artificial Intelligence and Statistics (AISTATS).

1/11/2024 A Unifying Framework for Action-Conditional Self-Predictive Reinforcement Learning to be presented as an Oral at NeurIPS 2024 Workshop: Self-Supervised Learning - Theory and Practice.

26/09/2024 Two papers to appear at NeurIPS 2024. Details coming soon.

1/09/2024 Attending the Dagstuhl Seminar on Explainable AI for Sequential Decision Making.

04/25/2024 Our work on Disentangling the Causes of Plasticity Loss in Neural Networks is accepted at Conference on Lifelong Learning Agents - CoLLAs 2024. See my talk slides.

12/16/2023 Contributed Talk on "POMRL" at "NeurIPS Generalization in Planning Workshop".

11/3/2023 Invited Talk at "UW Robotics Colloqium" hosted by Abhishek Gupta and Josh Smith at University of Washington.

9/30/2023 2 papers to appear in the "NeurIPS 2023 GenPlan Workshop".

7/25/2023 Our work on "Discovering Object-Centric Generalized Value Functions From Pixels" published at ICML 2023.

7/18/2023 "POMRL: No-Regret Learning-to-Plan with Increasing Horizons" published at the Journal of Transactions of Machine Learning Research (TMLR) .

6/30/2023 Recognized as an Expert Reviewer at "Transactions on Machine Learning Research" journal.

5/23/2023 Invited talk at "Upper Bound, 2023"; AI Week in Edmonton hosted by Amii.

5/12/2022 Started working as a Research Scientist at Google, Deepmind.

4/10/2022 Successfully defended my PhD dissertation 🎉

15/09/2022 "Towards continual reinforcement learning: A review and perspectives" accepted for publication at the Journal of Artificial Intelligence Research (JAIR) .

1/09/2022 I am co-organizing a NeurIPS 2022 workshop on All Things Attention: Bridging Different Perspectives on Attention. Consider submitting your work.

24/01/2022 I am interning with Harm van Seijen at Microsoft Research, Montreal in joint collaboration with Ida Momennejad, Sam Devlin, and Katja Hofmann .

28/09/2021 "Temporally Abstract Partial Models" accepted to NeurIPS, 2021.

20/08/2021 I will be interning with Katja Hoffman at Microsoft Research, Cambridge starting Nov 2021.

20/05/2021 I will be interning with Satinder Singh & Tom Zahavy at Deepmind in London starting June 2021.

23/2/2021 I am co-organizing two ICRL 2021 workshops on A Roadmap to Never-ending Reinforcement Learning & Rethinking ML Papers. Consider submitting your work.

15/2/2021 Invited talk on "Towards Continual Reinforcement Learning: A Review and Perspectives" at RIKEN Center for Advanced Intelligence Project- Approximate Bayesian Inference Team, Japan.

12/1/2021 "Learning Robust State Abstractions for Hidden-Parameter Block MDPs" accepted to ICLR, 2021.

2/12/2020 "Self-Supervised Attention-Aware Reinforcement Learning" accepted to AAAI, 2021. Also to appear at NeurIPS Workshop on Object Representations for Learning and Reasoning. Congrats to my fabulous mentee Haiping Wu.

2/12/2020 "Variance Penalized On-Policy and Off-Policy Actor-Critic" accepted to AAAI, 2021.

13/11/2020 Invited talk at Northeast Reinforcement Learning and Decision Making Symposium (NERDS).

8/9/2020 Invited to participate in the Rising Stars 2020 EECS workshop. Check out my participant profile here. An Academic Career Workshop for Women, hosted virtually at UC Berkeley.

17/7/2020 Our research on "What can I do here? A Theory of Affordances in RL" is featured in the MIT Technology Review.

2/7/2020 I am thrilled to serve on the Editorial Board for our new Mila Blog. Check it out!

1/6/2020 Our paper on "What can I do here? A Theory of Affordances in RL" has been accepted at the International Conference on Machine Learning (ICML) 2020. See full paper here.

14/5/2020 I gave a talk on "A Theory of Affordances in RL" at the Reinforcement Learning and Artificial Intelligence lab meeting at University of Alberta, invited by Martha White.

6/1/2020 Our work on Value Preserving State Action Abstractions has been accepted for presentation at AISTATS, 2020.

11/10/2019 Our work on Options of Interest: Temporal Abstraction with Interest Functions has been accepted at AAAI, 2020. A preliminary draft of the ideas in this work was published in the AAAI, 2019 Student Abstract proceedings

13/9/2019 I taught an invited lecture on “Introduction to RL” at the MILA/IVADO summer school on Deep Learning, Montreal, 2019

29/7/2019 I am currently a PhD Research Intern at DeepMind , Montreal working with Doina Precup and the HRL team.

6/6/2019 I am co-organizing a Lifelong Learning: A Reinforcement Learning (LLARLA) Workshop to be held during RLDM, 2019 .

24/3/2019 I am co-organizing a Workshop on Multi-Task and Lifelong Reinforcement Learning to be held during ICML-19 in Long Beach, USA . We have an exicting line up of speakers. Consider submitting your work.

10/26/2018 Our work on Learning Options with Interest Functions, has been selected for poster presentation in the AAAI-19 Student Abstract and Poster Program to be held during AAAI-19 in Honolulu, Hawaii

10/15/2018 I have been chosen to be a presenter at the 24th AAAI/SIGAI Doctoral Consortium (DC) at AAAI-19 in Honolulu, Hawaii

6/20/2018 Our work on 'Safe Option-Critic: Learning Safety in the Option-Critic Architecture' Accepted in ICML 2018 Workshop on Adaptive Learning Agents. Paper is now on arxiv!

6/16/2018 Our work on 'Attend Before you Act: Leveraging human visual attention for continual learning' Accepted in ICML 2018 Workshop on LifeLong Learning. Paper coming out soon!

5/13/2018 I will be one of the Teaching Assistants at the AI For Social Good Summer School from May 14-June 21. Super excited to be a part of this initiative.

2/28/2018 Invited talk on Introduction to Computer Vision & Reinforcement Learning at ITTT (Informative Talks on Technical Topics) organized by the McGill IEEE Student Branch.

11/13/2017 Volunteered as a mentor at the Women in Innovation and Artificial Intelligence Breakfast organized as the kick-off event of the McGill Innovation Week in the Fall of 2017.

10/1/2017 Organized the weekly Fall-17 AI Safety RL Lab Reading Group to get started with discussions and initial reading on related literature.

9/15/2017 Joined the Reasoning and Learning lab at McGill University as a 1st year Ph.D. student. Attended the ACM Canadian Women Celebration of Women in Computing

6/1/2017 Left job at Intel. I will be joining Ph.D. in CS program at the McGill University in Fall 2017.

5/1/2017 My application for Reinforcement Learning Summer School (RLSS) 2017 has been accepted. 225 out of 1130 applicants were selected.

4/14/2017 My application for International Computer Vision Summer School (ICVSS) 2017 has been accepted. 150 out of 430 applicants were selected.

03/27/2017 Manuscript entitled “Creating Segments and Effects on Comics by Clustering Gaze Data” has been accepted for publication in the Transactions on Multimedia Computing Communications and Applications [ACM TOMM]

02/27/2017 Invited Talk on Learning Visual Representations at the ASU Active Perception Group Seminar Series at the Arizona State University, Tempe.

08/03/2016 Invited Talk at the Women in Deep Learning event at the Deep Learning Summer School, Montreal

06/09/2016 As of June-27-2016, I start working as a Software Engineer in the Perception Computing Group-IOTG at Intel Corporation, Arizona.

05/11/2016 My application for CIFAR-CRM 2016 Deep Learning Summer School has been accepted.

04/22/2016 Paper on 'A Preliminary Benchmark of Four Saliency Algorithms On Comic Art' Accepted in ICME-MMArt Workshop 2016 .

09/23/2015 Attended RoboBusiness Expo 2015 at San Jose from Intel.

02/05/2015 I will be spending Summer & Fall of 2015 as a Perception Computing Software Intern at Intel.

12/10/2014 My robot AUMORO built in the course IMDL- Fall 2014 was covered in the WUFT news on Robot Demo Day .

Media

A concept in psychology is helping AI to better navigate our world

MIT Technology Review , Hao, K. (July, 2020)

UF Engineers Display Intelligent Machines At Robot Demo Day

WUFT News , Whitson, C. (December, 2014)

The Gainesville Sun , Finger, D. (December, 2014)

Research

My research aims to (1) understand intelligent behavior that bridges both action and perception grounded in theoretical foundations of reinforcement learning, and (2) build AI agents to efficiently represent the world knowledge, plan with it, and adapt to changes over time through learning and interaction. I approach this with the following research directions:

Selective Attention for Fast Adaptation and Robustness

Humans adapt robustly in complex and dynamic environments by using selective attention, which is then aggregated in a representation. How can do we enable more efficient learning of such representations to guide behavior?

Papers
Attend Before You Act: Leveraging human visual attention for Continual Learning [ICML Workshop 2018]
Options of Interest: Temporal abstraction with interest functions [AAAI 2020]
Self-Supervised Attention-Aware Reinforcement Learning [AAAI 2021]

Learning Abstractions and Affordances

Interactive behavior requires dynamically tracking and updating action possibilities, i.e. ``affordances'' (defined as set of state and actions that achieve desired consequences). How can AI agents learn to represent and reason about their environment through this lens?

Papers
What can I do here? A theory of Affordances in Reinforcement Learning [ICML 2020]
Temporally Abstract Partial Models [NeurIPS 2021]
The Paradox of Choice: On the Role of Attention in HRL [NeurIPS Workshop, 2022]
Toward Human-AI Alignment in Large-Scale Multi-Player Games [Preprint, 2024]

Discovery and Continual Reinforcement Learning

For practical applications, to what extent can RL agents transfer what was learned from previous experience to new situations on-the-fly and adapt to non-stationarity, with the ability to discover more complex capabilities on top of those already developed?

Papers
Continual Reinforcement Learning: A Review and Perspectives [JAIR 2022]
POMRL: No-Regret Learning-to-Plan with Increasing Horizons [TMLR 2023]
Discovering Object-Centric Generalized Value Functions From Pixels [ICML 2023]

Papers

Representative papers are highlighted.

For a complete list, please see Google Scholar. /

	Self-Predictive Representations for Combinatorial Generalization in Behavioral Cloning Daniel Lawson, Adriana Hugessen, Charlotte Cloutier, Glen Berseth+, Khimya Khetarpal+ Workshop on Reinforcement Learning Beyond Rewards: Ingredients for Developing Generalist Agents (RLC) 2025. paper \| website
	Cracking the Code of Action: A Generative Approach to Affordances for Reinforcement Learning Lynn Cherif, Flemming Kondrup, David Venuto, Ankit Anand, Doina Precup, Khimya Khetarpal Third Workshop on Deep Learning for Code (ICLR), 2025. paper

	Long Range Navigator (LRN): Extending robot planning horizons beyond metric maps Matthew Schmittle, Rohan Baijal, Nathan Hatch, Rosario Scalise, Mateo Guaman Castro, Sidharth Talia, Khimya Khetarpal, Siddhartha Srinivasa, Byron Boots Conference on Robot Learning (CoRL) 2025. paper \| website \| video
	A Unifying Framework for Action-Conditional Self-Predictive Reinforcement Learning Khimya Khetarpal, Zhaohan Daniel Guo, Bernardo Avila Pires, Yunhao Tang, Clare Lyle, Mark Rowland, Nicolas Heess, Diana Borsa, Arthur Guez, Will Dabney Self-Supervised Learning - Theory and Practice Workshop, Neural Information Processing Systems (NeurIPS), 2024. (Oral) AISTATS, 2025. paper
	Toward Human-AI Alignment in Large-Scale Multi-Player Games Sugandha Sharma, Guy Davidson, Khimya Khetarpal, Anssi Kanervisto, Udit Arora, Katja Hofmann, Ida Momennejad, Wordplay: When Language Meets Games @ ACL, Workshop 2024 paper
	Disentangling the Causes of Plasticity Loss in Neural Networks Clare Lyle, Zeyu Zheng, Khimya Khetarpal, Hado van Hasselt, Razvan Pascanu, James Martens, Will Dabney Conference on Lifelong Learning Agents (CoLLAs), 2024. paper \| talk
	Forecaster: Towards Temporally Abstract Tree-Search Planning from Pixels Thomas Jiralerspong, Flemming Kondrup, Doina Precup, Khimya Khetarpal, GenPlan Workshop, Neural Information Processing Systems (NeurIPS), 2023. paper
	Discovering Object-Centric Generalized Value Functions From Pixels Somjit Nath, Gopeshh Subbaraj, Khimya Khetarpal, Samira Ebrahimi Kahou International Conference on Machine Learning (ICML), 2023. paper
	POMRL: No-Regret Learning-to-Plan with Increasing Horizons Khimya Khetarpal, Claire Vernade, Brendan O'Donoghue, Satinder Singh, Tom Zahavy Transactions on Machine Learning Research (TMLR), 2023. (Expert Reviewer Certification.) GenPlan Workshop, Neural Information Processing Systems (NeurIPS), 2023. paper \| talk \| code
	The Paradox of Choice: Using Attention in Hierarchical Reinforcement Learning Andrei Nica, Khimya Khetarpal, Doina Precup All Things Attention Workshop, Neural Information Processing Systems (NeurIPS), 2022. paper \| code \|
	Temporally Abstract Partial Models Khimya Khetarpal, Zafarali Ahmed, Gheorghe Comanici, Doina Precup Neural Information Processing Systems (NeurIPS), 2021 paper \| code \| talk
	Towards Continual Reinforcement Learning: A Review and Perspectives Khimya Khetarpal, Matthew Riemer, Irina Rish, Doina Precup Journal of Artificial Intelligence Research (JAIR), 2022 paper
	Learning Robust State Abstractions for Hidden-Parameter Block MDPs Amy Zhang, Shagun Sodhani, Khimya Khetarpal, Joelle Pineau International Conference on Learning Representations (ICLR), 2021 paper \| bibtex \| webpage
	Sequoia: A Software Framework to Unify Continual Learning Research Fabrice Normandin, Florian Golemo, Oleksiy Ostapenko, Pau Rodriguez, Matthew D Riemer, Julio Hurtado, Khimya Khetarpal, Dominic Zhao, Ryan Lindeborg, Timothée Lesort, Laurent Charlin, Irina Rish, Massimo Caccia Workshop on Theory and Foundation of Continual Learning (ICML Workshop), 2021 paper \| framework \| 5 min intro
	Self-Supervised Attention-Aware Reinforcement Learning Haiping Wu, Khimya Khetarpal, Doina Precup Association for the Advancement of Artificial Intelligence (AAAI), 2021 paper \| poster \| code \| talk
	Variance Penalized On-Policy and Off-Policy Actor-Critic Arushi Jain, Gandharv Patil, Ayush Jain, Khimya Khetarpal, Doina Precup Association for the Advancement of Artificial Intelligence (AAAI), 2021 paper \| poster \| code
	What can I do here? A Theory of Affordances in Reinforcement Learning (Featured in MIT Technology Review) Khimya Khetarpal, Zafarali Ahmed, Gheorghe Comanici, David Abel, Doina Precup International Conference on Machine Learning (ICML), 2020 paper \| bibtex \| code \| press

	Options of Interest: Temporal Abstraction with Interest Functions Khimya Khetarpal, Martin Klissarov, Maxime Chevalier-Boisvert, Pierre-Luc Bacon, Doina Precup Association for the Advancement of Artificial Intelligence (AAAI), 2020 paper \| bibtex \| code \| webpage \| poster
	Value Preserving State-Action Abstractions David Abel, Nathan Umbanhowar, Khimya Khetarpal, Dilip Arumugam, Doina Precup, and Michael L. Littman International Conference on Artificial Intelligence and Statistics (AISTATS), 2020 paper \| bibtex \| code
	Learning Generalized Temporal Abstractions across Both Action and Perception (Scholarship Award) Khimya Khetarpal Association for the Advancement of Artificial Intelligence (AAAI), 2019 Doctorial Consortium Track paper \| bibtex
	Learning Options with Interest Functions (3 Minute Thesis Finalist) Khimya Khetarpal, Doina Precup Association for the Advancement of Artificial Intelligence (AAAI), 2019 Student Abstract Track paper \| bibtex \| code \| webpage \| poster
	Variational State Encoding as Intrinsic Motivation in Reinforcement Learning Martin Klissarov, Riashat Islam, Khimya Khetarpal, Doina Precup The Multi-disciplinary Conference on Reinforcement Learning and Decision Making (RLDM), 2019 paper \| bibtex
	Attend before you act: Leveraging human visual attention for continual learning (Best Paper Award- 3rd Place) Khimya Khetarpal, Doina Precup Lifelong Learning: A Reinforcement Learning Approach Workshop (ICML), 2018 paper \| bibtex \| code \| webpage \| poster
	Safe option-critic: Learning safety in the option-critic architecture Arushi Jain, Khimya Khetarpal, Doina Precup Adaptive Learning Agents Workshop, (ICML), 2018. Invited for submission to special issue of The Knowledge Engineering Review (Cambridge University Press journal) paper \| bibtex \| code \| webpage
	Re-evaluate: Reproducibility in evaluating reinforcement learning algorithms Khimya Khetarpal, Zafarali Ahmed, Andre Cianflone, Riashat Islam, Joelle Pineau Reproducibility in Machine Learning Workshop, (ICML), 2018. paper \| bibtex \| code \| poster
	Environments for Lifelong Reinforcement Learning Khimya Khetarpal, Shagun Sodhani, Sarath Chandar, Doina Precup Continual Learning Workshop, Workshop, (NeurIPS), 2018. paper \| bibtex \| poster
	Creating Segments and Effects on Comics by Clustering Gaze Data Ishwarya Thirunarayanan, Khimya Khetarpal, Sanjeev Koppal, Olivier Le Meur, John Shea and Eakta Jain ACM Transactions on Multimedia Computing, Communications, and Applications, (TOMM), 2017. paper \| bibtex \| webpage
	A Preliminary Benchmark Of Four Saliency Algorithms On Comic Art Khimya Khetarpal, Eakta Jain International Workshop on Multimedia Artworks Analysis (MMArt), (IEEE ICME), 2016. paper \| bibtex \| webpage \| talk slides
	Mobile robot navigation using evolving neural controller in unstructured environments AwhanPatnaik, Khimya Khetarpal, Laxmidhar Behera International Conference on – Advances in Control and Optimization of Dynamical Systems, (IFAC), 2014. paper \| bibtex \| code \| video \| talk slides

Representative Talks

Talks

	Disentangling the Causes of Plasticity Loss in Neural Networks The Conference on Lifelong Learning Agents (CoLLAs), Pisa, Italy, Spotlight Talk, May, 2024
	POMRL: No-Regret Learning-to-Plan with Increasing Horizons Upper Bound, Amii, Edmonton, Invited Talk, May, 2023 Meta Paris, Virtual, Invited Talk, Aug, 2023 WEIRD Lab, UW, Seattle, Invited Talk, Apr, 2023
	Bridging State and Action: Towards Continual Reinforcement Learning (PhD Defence) RL Lab, McGill University, Mila, Montreal, October, 2022 slides \| pictures
	Bridging State and Action: Towards Continual Reinforcement Learning RLAI Lab, University of Alberta, Edmonton, Invited Talk, 2022 Microsoft Research, NYC, Invited Talk, 2022 (virtual) Microsoft Research, Montreal, Invited Talk, 2022 (virtual) Brown Robotics Lab, Brown University, Invited Talk, 2022, (virtual) Deepmind , Invited Talk, 2022, (virtual) Google Research , Invited Talk, 2022 (virtual)
	Temporally Abstract Partial Models Neural Information Processing Systems (NeurIPS), 2021 Microsoft Research RL Reading Group, Invited Talk, 2021 paper \| code \| slides \| talk
	Towards Continual Reinforcement Learning: A Review and Perspectives RIKEN Center for Advanced Intelligence Project- Approximate Bayesian Inference Team (Japan), Invited Talk, 2021 paper \| slides \| twitter
	What can I do here? A Theory of Affordances in Reinforcement Learning (Featured in MIT Technology Review) International Conference on Machine Learning (ICML), Virtual Vienna, 2020 Northeast Reinforcement Learning and Decision Making Symposium (NERDS), 2020 talk \| slides \| paper \| twitter
	Options of Interest: Temporal Abstraction with Interest Functions Association for the Advancement of Artificial Intelligence (AAAI), New York, 2020. AAAI spotlight \| DeepMind HRL Meeting \| paper \| twitter
	Learning Generalized Temporal Abstractions across Both Action and Perception Association for the Advancement of Artificial Intelligence (AAAI), Hawaii, 2019 Doctorial Consortium Track, (Mentor: Michael Littman) AAAI DC \| paper \| twitter
	Learning Options with Interest Functions Association for the Advancement of Artificial Intelligence (AAAI), Hawaii, 2019 Student Abstract Track, (3 Minute Thesis Finalist) talk \| webpage \| twitter
	Attend before you act: Leveraging human visual attention for continual learning (Best Paper Award- 3rd Place) Lifelong Learning: A Reinforcement Learning Approach Workshop (ICML), Stockholm, 2018 oral \| webpage \| twitter

Students

Daniel Lawson

PhD

co-advised with Glen Berseth

Lynn Cherif

co-advised with Doina Precup

Somjit Nath

PhD

Thomas Jiralerspong

Haiping Wu

Teaching

	Teaching Assistant, COMP-767 Reinforcement Learning, Winter 2020 Teaching Assistant, COMP-208 Computers in Engineering, Winter 2018 Guest Lecture, Hierarchical RL, Management Studies, 2019 [slides]
	Reinforcement Learning, IVADO Deep Learning Summer School, 2019 [slides]
	Lecturer, Reinforcement Learning, 2020 [slides] Lecturer, Deep Reinforcement Learning, 2019 [slides] Teaching Assistant, 2018 [resources]

Organizational Roles

Reinforcement Learning Conference, RLC 2025, Area Chair

All Things Attention: Bridging Different Perspectives on Attention, NeurIPS 2022

A Roadmap to Never-Ending Reinforcement Learning, Workshop, ICLR 2021

Rethinking ML Papers, Workshop, ICLR 2021

Continual Reinforcement Learning, Un-Workshop WiML, ICML 2020

Lifelong Learning: A Reinforcement Learning Approach (LLARLA), RLDM 2019

Multi-Task and Lifelong Reinforcement Learning Workshop, ICML 2019

Reviewing

Reviewer, JMLR, Journal of Machine Learning Research

Reviewer, TMLR, Transactions on Machine Learning Research

Reviewer, EWRL, European Workshop on Reinforcement Learning('22)

Reviewer, AISTATS ('21), ICLR ('20, '21), NeurIPS ('20, '21, '22)

Reviewer, NeurIPS, Deep RL Workshop ('20), Reproducibility Challenge ('19)

Program Committee, Continual Learning Workshop, NeurIPS 2018

Reviewer, AI for Social Good Workshop, NeurIPS 2018

HOME
ABOUT
AUCTIONS
SHIPPING
FEES
TOOLS
HOW
FAQ
CONTACT

Original Source | Taken Source