I’m an undergraduate student at Kyunghee University majoring in Computer Engineering, expected to graduate in February 2022. Two sets of work have defined my career interests: Research in NLP and AI Engineering.
Research in NLP
As an AI research engineer, My research has focused on Language Modeling to make a more smart AI model. Specifically, I am interested in building a language model to solve a real-world problem: (1) developing an efficient language model through model light-weighting such as knowledge distillation or augmented memory in the model. (2) Programming language NLP which can help people write code, docstring and commit message. I have the following views regarding research: Research should be based on service products. And transferring from the only R & D to a service product is an essential ability for AI researchers.
As the model gets bigger, engineering’s ability to manage many GPU machines is essential for training. Since I have the industry experience to deal with many GPU machines, I deeply understand how to handle many multi-node GPU with avoiding bottlenecks. I also managed data pipelines efficiently (See my open source project: matorage!) and designed scalable cloud infrastructure to service trained models.
BS in Computer Science, 2015
Kyung Hee University (Leave of absence for two years due to military service in Korea)
Improving accuracy and speed trade-off when finetuning pretrained language models by using large product key memory and mitigating a catastrophic drift with initialization and residual memory. (I was a research internship at Clova AI while doing this work.) - Findings of EMNLP 2020
Brunel AI is a startup that provides ai search products to help people search for patents. It was a good experience to think about AI from a product perspective as the developed model was applied to the actual product and received feedback.
Task : Organization of AI Engineering
NAVER Clova AI is Korea’s leading AI organization. I had studied Korean language modeling in the LaRva team, and through this, I have a know-how about the large-scale modeling in GPU environment.
Also, based on good resources and team members, I was able to achieve great results in various tasks in a short period of time(only 6 months).
Task 1: Pretraining large scale Lanuage Model such as BERT, RoBERTa on distributed GPU environment.
Task 2: Efficient modeling and lightweight model in pretrained language model
Task 3: KorQuAD2.0 leaderBoard 1th(F1/EM:83.54/66.95) in 04/22/2020
Platfarm is a startup that develops products that recommend chat text into emoji. As my first industry experience, I was able to learn about the collaboration culture.
Task : Emoji recommendation in chatting text