Photo of Alexey Svyatkovskiy

Alexey Svyatkovskiy

I am currently a research scientist at Google DeepMind, GenAI. My focus is on RL post-training and synthetic data generation to improve Gemini coding capabilities. Previously, I led an applied research team at Microsoft focusing on code understanding and generation, and automating SWE tasks in IntelliCode and GitHub Copilot.

Earlier in my career, I worked at PPPL on ML for fusion energy science and at CERN's CMS collaboration on fundamental physics problems.

News & Updates

Google Gemini
July 2025

Gemini 2.5 Technical Report Published

Gemini 2.5 Pro achieves SOTA performance on several frontier coding and reasoning benchmarks.

Read the paper on arXiv →
Google Gemini
May 2025

Gemini 2.5 Enhanced Coding Performance

Gemini 2.5 I/O edition has been released, featuring my our work. Significantly improved agentic coding capabilities in IDEs like Cursor. The update brings meaningful improvements for front-end and UI development, code transformation, and sophisticated agentic workflows.

Read the announcement →
April 2025

ICSE 2025: Reinforcement Learning for High-Quality Unit Test Generation

Our paper on using RL from automatic feedback for unit test generation was accepted at the DeepTest 2025 workshop, co-located with ICSE 2025. The work demonstrates how RL techniques can improve the quality and coverage of automatically generated unit tests.

View paper details →
February 2025

Featured in ACM Queue: Program Merge - What's Deep Learning Got to Do with It?

Participated in an interview with ACM Queue discussing our DeepMerge work at Microsoft Research, sharing insights on using machine learning to resolve merge conflicts and the challenges of collaborative programming at scale.

Read on ACM CACM →
December 2024

NeurIPS 2024 in Vancouver

Attending the NeurIPS 2024 in Vancouver, engaging with the latest in machine learning and AI research.

May 2023

ACL 2023: CodeExecutor - Learning to Execute Code with Transformers

Published at ACL 2023. Introduced CodeExecutor, a Transformer model that leverages code execution pre-training and curriculum learning to understand and perform code execution, enabling better semantic comprehension of program behavior.

Read on arXiv →
Google DeepMind
October 2023

Joined Google DeepMind

Started as a Research Scientist at Google DeepMind, focusing on reinforcement learning post-training and synthetic data generation to improve Gemini's coding capabilities.

November 2023

FSE 2023: InferFix - End-to-End Program Repair with LLMs

Published at FSE 2023. Introduced InferFix, a transformer-based program repair framework paired with static analysis (Infer) to fix critical security and performance bugs, achieving 65.6% top-1 accuracy for C# and 76.8% for Java. Deployed at Microsoft for automated bug fixing.

Read on arXiv →
November 2023

FSE 2023: AdaptivePaste - Intelligent Copy-Paste in IDE

Published at FSE 2023. Introduced AdaptivePaste, a learning-based approach to source code adaptation that helps developers adapt variable identifiers when copy-pasting code, achieving 79.8% accuracy and reducing manual adaptation time by nearly half.

Read on arXiv →
October 2023

DeepSpeed4Science Initiative: AI for Scientific Discovery

Published white paper on DeepSpeed4Science initiative, building AI system technologies to accelerate scientific discoveries in structural biology, climate science, and fusion energy. Served as Microsoft liaison for Princeton University collaborations.

Read on arXiv →
October 2023

ICSME 2023 Keynote in Bogotá

Delivered a keynote presentation at the IEEE International Conference on Software Maintenance and Evolution (ICSME 2023) in Bogotá, Colombia, discussing the intersection of AI and software engineering.

Conference website →
November 2022

FSE 2022: Exploring Personalized Models for Code Generation

Published at FSE 2022. Evaluated personalized fine-tuning techniques for code generation including custom fine-tuning, lightweight fine-tuning, and prefix tuning, analyzing trade-offs between compute cost and predictive performance for project-specific models.

Read on arXiv →
November 2022

FSE 2022: CodeReviewer - Automating Code Review with Pre-training

Published at FSE 2022. Introduced CodeReviewer, a pre-trained model utilizing four specialized pre-training tasks for code review, achieving state-of-the-art performance on code change quality estimation, review comment generation, and code refinement across nine programming languages.

Read on arXiv →
September 2022

TSE/FSE 2022: MergeBERT - Neural Merge Conflict Resolution

Published in IEEE Transactions on Software Engineering (TSE) and FSE 2022. Introduced MergeBERT, a transformer-based framework for automated merge conflict resolution achieving 63-68% accuracy with nearly 3x improvement over existing tools, validated with 25 developers on 122 real conflicts.

Read on arXiv →
May 2022

DeepMerge: Learning to Merge Programs

We published in TSE. This paper introduces the first data-driven approach to automatically resolve merge conflicts using deep learning.

Read the paper →
May 2022

ACL 2022: ReACC - Retrieval-Augmented Code Completion

Published at ACL 2022. Introduced a retrieval-augmented framework that leverages external code context through lexical copying and semantic retrieval, achieving state-of-the-art performance on the CodeXGLUE benchmark for Python and Java code completion.

Read on arXiv →
May 2022

MSR 2022: Methods2Test Dataset for Unit Test Generation

Published at MSR 2022. Released Methods2Test, the largest supervised dataset of 780K JUnit test cases mapped to focal methods, extracted from 91K Java open-source projects to enable machine learning research for automated test generation.

Read on arXiv →
April 2022

ICLR 2022: Learning to Complete Code with Sketches

We published at ICLR 2022. Introduced Grammformer, a grammar-guided Transformer model that generates code completions with "holes" in uncertain parts, achieving 10-50% more accurate completions than traditional generative models.

Read on arXiv →
November 2021

EMNLP 2021: eWASH - Long-Range Modeling of Source Code

Published at EMNLP 2021. Introduced eWASH (Extended Window Access by Syntax Hierarchy), a technique for incorporating entire file-level context into fixed-length windows by leveraging syntactic hierarchies, achieving state-of-the-art results on CodeXGLUE benchmarks for code completion and summarization.

Read on arXiv →
December 2021

NeurIPS 2021: CodeXGLUE Benchmark for Code Understanding

Published at NeurIPS 2021 Datasets and Benchmarks Track. CodeXGLUE includes 10 tasks across 14 datasets for code understanding and generation with baseline systems, and has become a widely-used benchmark in the code AI research community.

Read on arXiv →
June 2021

MAPS 2021: Generating Bug-Fixes Using Pretrained Transformers

Published at MAPS 2021 (co-located with PLDI). Introduced DeepDebug, a data-driven program repair approach using sequence-to-sequence learning with denoising pretraining and supervised finetuning, generating 75% more non-deletion fixes than prior state of the art.

Read on arXiv →
May 2021

MSR 2021: Fast and Memory-Efficient Neural Code Completion 🏆 Best Paper Award

Published at MSR 2021 and received the Best Paper Award. Designed a novel reranking neural completion model combining static analysis with granular token encodings, achieving 90% accuracy in top-5 suggestions while consuming just 6 MB of RAM—19x less than previous models.

Read on arXiv →
November 2020

EMNLP 2020: PyMT5 - Multi-Mode Translation of Code and Text

Published at EMNLP 2020. Introduced PyMT5, a T5-based model trained to translate between Python methods and docstrings, achieving 92.1% syntactically correct method generation on a corpus of 26 million Python methods.

Read on arXiv →
September 2020

AthenaTest: Unit Test Generation with Transformers

Introduced AthenaTest and the Methods2Test dataset. The approach generates readable unit tests by learning from real-world test cases, achieving 43.7% focal method coverage and receiving overwhelming developer preference over GPT-3 and EvoSuite.

Read on arXiv →
September 2020

ICLR 2021: GraphCodeBERT - Code Representations with Data Flow

Published at ICLR 2021. Introduced GraphCodeBERT, a pre-trained model for programming language that incorporates semantic-level code structure through data flow, achieving state-of-the-art performance on code search, clone detection, code translation, and refinement.

Read on arXiv →
November 2020

FSE 2020: IntelliCode Compose - Neural Code Generation

Published at the FSE. Introduced IntelliCode Compose, a Transformer-based multi-line code generation system deployed in Visual Studio Code.

Read the paper →
April 2019

Nature: Predicting Disruptions in Fusion Plasmas Through Deep Learning

Published in Nature! Developed a deep learning method to predict disruptive instabilities in tokamak fusion reactors, trained on data from DIII-D and JET tokamaks. The approach demonstrates cross-machine prediction capabilities crucial for ITER and future fusion reactors, opening possibilities for active reactor control and optimization.

Read in Nature →
July 2019

Featured by Oak Ridge National Lab on Fusion AI Research

Oak Ridge Leadership Computing Facility highlighted our fusion disruption prediction work using the Titan and Summit supercomputers. The feature showcased how FRNN leveraged world-class HPC systems to achieve breakthrough performance in predicting plasma instabilities.

Read the feature →
April 2019

Featured by NVIDIA on AI for Clean Fusion Energy

NVIDIA Developer Blog featured our work on converting the FRNN algorithms into production code, highlighting the use of GPU-accelerated parallel processing to predict fusion disruptions within the 30-millisecond time frame required by ITER.

Read on NVIDIA Blog →
Microsoft
May 2018

Joined Microsoft

Started at Microsoft to work on code understanding and generation, developing AI-powered tools to enhance SWE productivity in IntelliCode and GitHub Copilot.

December 2017

AI for Fusion Energy: Plasma Disruption Prediction

Featured on Princeton.edu for developing the Fusion Recurrent Neural Network (FRNN). Our work applied deep learning to predict disruptions in tokamak fusion plasmas, using high-performance computing on Oak Ridge's Titan supercomputer to train FRNN on massive datasets from the JET and DIII-D tokamaks.

Read the feature →
Princeton University
March 2015

Joined Princeton University

Started as a researcher at Princeton Institute for Computational Science and Engineering (PICSciE), applying machine learning to fusion energy challenges and developing deep learning methods for plasma disruption prediction.