If you are interested in dimensionality reduction, chances are that you have come across the Johnson-Lindenstrauss lemma. I learned about it while studying the Linformer paper, which contains a result on dimensionality reduction for the Transformer. Essentially, they prove that self-attention is lo… read more

The transformer For another meeting of our reinforcement/machine learning reading group, I gave a talk on the underlying model of GPT-2 and GPT-3, the ‘Transformer’. There are two main concepts I wanted to explain: positional encoding and attention. During the talk, I found that two things were most… read more

Effective Kan fibrations in simplicial sets As part of the Workshop on Homotopy Type Theory and Univalent Foundations (HoTT/UF), I have contributed a talk on my recent work with Benno van den Berg. The talk is available on youtube. An abstract can be found here and the full paper is now also availab… read more

Slides for chapter 11 of Barto and Sutton For my fortnightly reading group on reinforcement learning, I prepared a talk on chapter 11 of the book by Barto and Sutton. This chapter is on off-policy methods with approximation. The main content consists of some negative results for stochastic semi-grad… read more