I am a senior research scientist at Google DeepMind.
I obtained my PhD in the department of Computer Science at the Technion, where I was fortunate to be advised by Prof. Eran Yahav. My PhD dissertation has been awarded the Reynolds Doctoral Dissertation Award (formerly “SIGPLAN Outstanding Doctoral Dissertation Award”).
Previously, I served 7 years as an officer onboard a missile ship in the Israeli Navy. Later, I completed my BSc summa cum laude at the Computer Science Department at the Technion, as an alumnus of The Rothschild-Technion Scholars Program for Excellence. Between 2014-2016, I worked at Microsoft R&D center in Haifa, developing data security services for the cloud. Between June-September of 2018, I interned at Google New-York, researching neural models for speech recognition.
In addition, I hold a B.A. in Humanities.
I am happily married to Lee and father of Gur 🙂
- October 2023 - CodeBERTScore was accepted to EMNLP’2023!
- September 2023 - Unlimiformer and Self-Refine were accepted to NeurIPS’2023!
- August 2023 - I left CMU and started as a research scientist at Google DeepMind [Tweet]
- May 2023 - A new paper: On the Expressivity Role of LayerNorm in Transformers’ Attention was accepted to Findings of the ACL’2023!
- May 2023 - a new preprint: Unlimiformer: Long-Range Transformers with Unlimited Length Input
- April 2022 - PAL: Program-aided Language Models and Why do kNN-LMs Work? were accepted to ICML’2023!
- April 2023 - a new preprint: Self-Refine: Iterative Refinement with Self-Feedback
- March 2023 - Learning Performance-Improving Code Edits and CodeBERTScore (Spotlight!) will appear in the Deep Learning for Code ICLR’2023 workshop
- February 2023 - a new preprint: Learning Performance-Improving Code Edits
- February 2023 - a new preprint: CodeBERTScore: Evaluating Code Generation with Pretrained Models of Code
- January 2023 - Our DocPrompting paper was accepted to ICLR’2023 as a Spotlight!
- January 2023 - a new preprint: Why do Nearest Neighbor Language Models Work?
- December 2022 - A new demo for PAL!
- December 2022 - I was invited to the explAInable podcast (Hebrew)
- November 2022 - a new preprint: PaL: Program-aided Language Models
- October 2022 - Our paper Language Models of Code are Few-Shot Commonsense Learners was accepted to EMNLP’2022!
- September 2022 - We released a new repository for evaluation of code generation: code-bert-score, along with pretrained models of several programming languages, based on CodeBERT.
- August 2022 - a new preprint:
DocPrompting: Generating Code by Retrieving the Docs
- July 2022 - I released a new HuggingFace 🤗
transformersimplementation of RetoMaton, kNN-language models and kNN-machine translation: https://github.com/neulab/knn-transformers
- June 2022 - I was selected for the ACM SIGPLAN Reynolds Doctoral Dissertation Award (formerly “SIGPLAN Outstanding Doctoral Dissertation Award”)!
- May 2022 - Our RetoMaton paper was accepted to ICML’2022!
- April 2022 - Our PolyCoder paper will appear in ICLR 2022’s DL4Code and PLDI 2022’s MAPS workshops.
- March 2022 - A new preprint: A Systematic Evaluation of Large Language Models of Code
- February 2022 - A new preprint: Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval
- January 2022 - Our paper How Attentive are Graph Attention Networks? was accepted to ICLR’2022!
- WebArena: A Realistic Web Environment for Building Autonomous Agents
- Unlimiformer: Long-Range Transformers with Unlimited Length Input
- Self-Refine: Iterative Refinement with Self-Feedback
- Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Sean Welleck, Bodhisattwa Prasad Majumder, Shashank Gupta, Amir Yazdanbakhsh, Peter Clark
- To appear in NeurIPS’2023
- Online demo: https://self-refine-webgen.herokuapp.com/
- Press: [MarkTechPost] [Medium] [EmergentMind]
- [PDF] [Code] [Website] [Tweet] [BibTex]
- CodeBERTScore: Evaluating Code Generation with Pretrained Models of Code
- On the Expressivity Role of LayerNorm in Transformers’ Attention
- Why do Nearest Neighbor Language Models Work?
- PAL: Program-aided Language Models
DocPrompting: Generating Code by Retrieving the Docs
- Language Models of Code are Few-Shot Commonsense Learners
- Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval (RetoMaton)
- How Attentive are Graph Attention Networks?
- On the Bottleneck of Graph Neural Networks and its Practical Implications
- A Structural Model for Contextual Code Changes
- Adversarial Examples for Models of Code
- Neural Reverse Engineering of Stripped Binaries using Augmented Control Flow Graphs
- Structural Language Models of Code
- Contextual Speech Recognition with Difficult Negative Training Examples
- code2seq: Generating Sequences from Structured Representations of Code
- code2vec: Learning Distributed Representations of Code
- A General Path-Based Representation for Predicting Program Properties
- Learning Performance-Improving Code Edits
- A Systematic Evaluation of Large Language Models of Code (PolyCoder)
- Single-Node Attack for Fooling Graph Neural Networks
- Machine Learning for Programming Language Processing
- Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
- Jonathan Shen, …, Uri Alon, …
- 2022 - Reynolds Doctoral Dissertation Award (formerly “SIGPLAN Outstanding Doctoral Dissertation Award”)
- 2021-2022 – Rothschild Post-Doctoral Fellowship
- 2021-2022 – Fulbright Post-Doctoral Fellowship (declined)
- 2020 – ACM SIGPLAN Research Highlight, “code2vec: Learning Distributed Representations of Code” (POPL’2019)
- 2019 – Jacobs Excellence Scholarship
- 2019 – Department Funding Excellence Scholarship
- 2018 – Department Funding Excellence Scholarship
- 2016 – Excellent Teaching Assistant
- 2016 – Dean’s Excellent Scholarship
- 2016 – Alumnus of the Rothchild-Technion Program for Excellence
- 2015 – SAMBA – CS Excellent Students
- Reviewer: ICLR’2023, NeurIPS ‘2022 (Outstanding Reviewer), TMLR, ICML’2022 (Outstanding Reviewer - top 10%), ICLR’2022 (Highlighted Reviewer), AIPLANS NeurIPS 2021 workshop, ICML’2021 (top 10% Best Reviewers), ICLR’2021, NeurIPS’2020, ICLR’2020
- Program Committee: MAPS’2022, Deep Learning for Code ICLR’22 workshop, PLDI’2021, NeurIPS’2020 CAP workshop, AIDM’20, AIDM’19
- Area Chair: Learning on Graphs ‘2022