Juan Pablo Rodriguez

Interested in planning and automatic goal generation for Reinforcement Learning agents.


Research questions

Published May 16, 2020

The objective for his write-up is to lay out some of the research questions and ideas that most interestes me.

Interesting concepts

  1. Probabilistic generative models.
  2. Program Induction and Program Synthesis
  3. Self-Play (Active learning)
  4. Goals, Plans and Stories.
  5. Sleep-wake cycle in learning.
  6. Transfer learning – Metaphor and analogy learning
  7. Deep Learning
1. Probabilistic generative models

The process of understanding “how the world works?” can be formalized as probabilistic generative model of the world. This type of models supports simulations and therefore planning.

2. Program induction and program synthesis

Coming up with a generative model that explains how the world works is in the domain of program synthesis. We have data-points collected from observation of the world and we have to come up with a program that best explains these observations.

3. Self-play (Active learning)

Self-play allows an agent to come up with novel strategies to address situations that are unknown a priori.

4. Goals, plan and stories

It seems to me that humans don’t have a static reward/cost function. On the other hand, we build stories and set goals that have an implicit function that guide our learning/behavior. Intelligent behavior can be summed up by coming up with goals/plans/stories that lead to an agent learning.

5. Sleep-wake cycle in learning

During “sleep” phase there should be a consolidation of knowledge in our agent (model) similar to what a huma brains does when it sleep.

6. Transfer learning – Metaphor and analogy learning

I believe that our human ability to generalize to out-of -sample observations is possible because of our ability to frame the current unknown situation in terms of a known one using metaphors and analogies. The consequence is that we transfer our knowledge from one task to another through a a story.

7. Deep Learning

Deep learning has proven to be a principled and scalable way to perform patter recognition. I think of deep learning as our core primitive in ideas mentioned above.

This can eventually lead me to a good statement of purpose.