psc's website

Pablo Samuel Castro

Señor Swesearcher at Google

I was born and raised in Quito, Ecuador, and moved to Montreal after high school to study at McGill. I stayed in Montreal for the next 10 years, finished my bachelors, worked at a flight simulator company, and then eventually obtained my masters and PhD at McGill, focusing on Reinforcement Learning under the supervision of Doina Precup and Prakash Panangaden. After my PhD I did a 10-month postdoc in Paris before moving to Pittsburgh to join Google. I have worked at Google since 2012, and am currently a staff research Software Developer in Google Brain in Montreal, focusing on fundamental Reinforcement Learning research, Machine Learning and Creativity, and being a regular advocate for increasing the LatinX representation in the research community. I am also an adjunct professor at Université de Montréal. Aside from my interest in coding/AI/math, I am an active musician and love running (6 marathons so far, including Boston!). If you would like to chat with me, book some time in my public office hours.

Hola, I'm psc

Pablo Samuel Castro

Señor Swesearcher at Google

Recent Posts

The Dormant Neuron Phenomenon in Deep Reinforcement Learning

Albums

PongDay

Selected Publications

On the consistency of hyper-parameter selection in value-based deep reinforcement learning

Mixture of Experts in a Mixture of RL settings

In value-based deep reinforcement learning, a pruned network is a good network

Mixtures of experts unlock parameter scaling for deep RL

Stop Regressing - Training Value Functions via Classification for Scalable Deep RL

Small batch deep reinforcement learning

Minigrid & miniworld - Modular & customizable reinforcement learning environments for goal-oriented tasks

A Kernel Perspective on Behavioural Metrics for Markov Decision Processes

Bigger, Better, Faster, Human-level Atari with human-level efficiency

The dormant neuron phenomenon in deep reinforcement learning

Proto-Value Networks, Scaling Representation Learning with Auxiliary Tasks

Reincarnating Reinforcement Learning, Reusing Prior Computation to Accelerate Progress

The State of Sparse Training in Deep Reinforcement Learning

A general class of surrogate functions for stable and efficient reinforcement learning

MICo, Learning improved representations via sampling-based state similarity for Markov decision processes

The Difficulty of Passive Learning in Deep Reinforcement Learning

Deep Reinforcement Learning at the Edge of the Statistical Precipice

Losses, Dissonances, and Distortions

Revisiting Rainbow, Promoting more insightful and inclusive deep reinforcement learning research

Metrics and continuity in reinforcement learning

Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning

Autonomous navigation of stratospheric balloons using reinforcement learning

Estimating Policy Functions in Payment Systems using Reinforcement Learning

Dopamine, A Research Framework for Deep Reinforcement Learning

GANterpretations

Rigging the lottery, Making all tickets winners

Scalable methods for computing state similarity in deterministic MDPs

A Geometric Perspective on Optimal Representations for Reinforcement Learning

A comparative analysis of expected and distributional reinforcement learning

ML-Jam, Performing Structured Improvisations with Pre-trained Models

Shaping the Narrative Arc, Information-Theoretic Collaborative Dialogue

An Atari Model Zoo for Analyzing, Visualizing, and Comparing Deep Reinforcement Learning Agents

Distributional reinforcement learning with linear function approximation

Inverse reinforcement learning with multiple ranked experts