1

CommitBERT: Commit Message Generation Using Pre-Trained Programming Language Model

Introduces the model and data that generate a commit message when code diff is given using the pre-trained programming language model about six programming languages (Python, PHP, Go, Java, JavaScript, and Ruby). - [ACL NLP4Prog Workshop 2021](https://nlp4prog.github.io/2021/)

Large Product Key Memory for Pretrained Language Models

Improving accuracy and speed trade-off when finetuning pretrained language models by using large product key memory and mitigating a catastrophic drift with initialization and residual memory. *(I was a research internship at Clova AI while doing this work.)* - [Findings of EMNLP 2020](https://2020.emnlp.org/)