Como parte de la RIIAA en Quito, di una introducción a los Transformers, que es la arquitectura detrás de avances como GPT-3, Music Transformer, Parti, y muchos otros.

Grabación

Pueden ver la grabación aquí:

Materiales

Aquí pueden acceder a los diferentes materiales que mencioné durante el curso:

• Las diapositivas que usé en el curso
• Write with Transformers de Hugging Face (GPT-2)
• Eleuther GPT-J-6B, que es mucho mejor modelo que el GPT-2 de Hugging Face
• El colab simple sobre bigrams
• El colab de Flax sobre LSTMs
• El excelente the Illustrated Transformer de Jay Alammar, en el cual basé la descripción de Transformers.
• The Annotated Transformer
• El blog post de Parti

The thread I wrote at the start of perf season at Google seemed to resonate with lots of people, so I decided to put a slightly extended version of it in blog-post form.

What is perf?

In brief, “perf” season at Google is when we evaluate our performance over the last few months, in the form of a self-assessment, and our peers provide their assessments on how they perceive our performance. The general purpose of this exercise is to receive feedback on how to grow as an engineer/researcher/employee, but it is also the process through which you can get promoted (by nominating yourself).

The NeurIPS 2021 review period is about to begin, and there will likely be lots of complaining about the quality of reviews when they come out (I’m often guilty of this type of complaint).

I decided to write a post describing how I approach paper-reviewing, in the help that it can be useful for others (especially those who are new to reviewing) in writing high quality reviews.

I’m mostly an RL researcher, so a lot of the tips below are mostly from my experience reading RL papers. I think many of the ideas are applicable more generally, but I acknowledge some may be more RL-specific.

This post is based on this colab.

You can also watch a video where I go through the basics here.

Pueden ver un video (en español) donde presento el material aquí.

Introduction

Reinforcement learning methods are used for sequential decision making in uncertain environments. It is typically framed as an agent (the learner) interacting with an environment which provides the agent with reinforcement (positive or negative), based on the agent’s decisions. The agent leverages this reinforcement to update its behaviour in an aim to get closer to acting optimally. In interacting with the uncertain environment, the agent is also learning about the dynamics of the underlying system.

I made a website where you can

• Draw your own GridWorlds
• Play around with hyperparameters while agent is training
• Transfer values between agents
• “Teleport” the agent to help it during learning

Hope you find it useful and fun!

In my previous post providing tips for interviewing at Google, I included the sentence “If you don’t know anyone at Google, you’ve already applied and haven’t heard back in a while, feel free to send me a note with your CV and I’ll see if there’s something I can do.”

I received a number of requests from people who had applied but never heard back. In most of these cases, I spotted issues with their resumes, which may or may not explain why they never heard back. In an effort to help others who are getting ready to apply for a job, I decided to write a new blog post with tips on how to prepare your resume for application.