Tae Hwan Jung

Tae Hwan Jung

AI Engineer

graykode

I’m an undergraduate student at Kyunghee University majoring in Computer Engineering, expected to graduate in February 2022. Two sets of work have defined my career interests: Research in NLP and AI Engineering.

Research in NLP
As an AI research engineer, My research has focused on Language Modeling to make a more smart AI model. Specifically, I am interested in building a language model to solve a real-world problem: (1) developing an efficient language model through model light-weighting such as knowledge distillation or augmented memory in the model. (2) Programming language NLP which can help people write code, docstring and commit message. I have the following views regarding research: Research should be based on service products. And transferring from the only R & D to a service product is an essential ability for AI researchers.

AI Engineering
As the model gets bigger, engineering’s ability to manage many GPU machines is essential for training. Since I have the industry experience to deal with many GPU machines, I deeply understand how to handle many multi-node GPU with avoiding bottlenecks. I also managed data pipelines efficiently (See my open source project: matorage!) and designed scalable cloud infrastructure to service trained models.

Interests

  • Language Modeling
  • Program Language NLP
  • AI Product Engineering

Education

  • BS in Computer Science, 2015

    Kyung Hee University (Leave of absence for two years due to military service in Korea)

Industry Experience

 
 
 
 
 

AI Research Engineer

Upstage AI

Jul 2021 – Present Yongin, Republic of Korea
 
 
 
 
 

Contract AI Engineer

Brunel AI

Sep 2020 – Jun 2021 Mapo-gu, Republic of Korea

Brunel AI is a startup that provides ai search products to help people search for patents. It was a good experience to think about AI from a product perspective as the developed model was applied to the actual product and received feedback.

Task : Organization of AI Engineering

  • Help transformation into AI tasks that can be realized as real products(For example, which data to collect will be meaningful in the long term or what AI task to solve the problem with)
  • Improve search quality : Developing a model that recommends a patent based on the user’s input query
  • Building infrastructure for data pipeline and model serving
 
 
 
 
 

AI Research Internship

NAVER Clova AI

Dec 2019 – Jun 2020 Seongnam, Republic of Korea

NAVER Clova AI is Korea’s leading AI organization. I had studied Korean language modeling in the LaRva team, and through this, I have a know-how about the large-scale modeling in GPU environment.
Also, based on good resources and team members, I was able to achieve great results in various tasks in a short period of time(only 6 months).

Task 1: Pretraining large scale Lanuage Model such as BERT, RoBERTa on distributed GPU environment.

  • I have dealt with up to 64 V100 GPUs and large amounts of training data in a distributed environment.
  • I have gone through hundreds of pre-training processes and have experienced how to avoid CPU, GPU bottlenecks, how to use half precision, and how to manage large amounts of data.

Task 2: Efficient modeling and lightweight model in pretrained language model

  • We studied how to use Lample’s Product Key Memory(PKM) efficiently, avoiding catastrophic drift in Masked-LM and fine-tuning tasks, and this study was accepted findings of EMNLP 2020.
  • I have know-how about knowledge distillation to lightweight the Korean RoBERTa model

Task 3: KorQuAD2.0 leaderBoard 1th(F1/EM:83.54/66.95) in 04/22/2020

  • KorQuAD 2.0 is a dataset similar to Google’s Natural Question, which is a very difficult problem to find the correct answer on one page of Wikipedia. The answer can be a table, list, or paragraph.
 
 
 
 
 

AI Engineer Internship

Platfarm

Dec 2018 – Feb 2019 Mapo-gu, Republic of Korea

Platfarm is a startup that develops products that recommend chat text into emoji. As my first industry experience, I was able to learn about the collaboration culture.

Task : Emoji recommendation in chatting text

  • Data annotation in unrefined chatting corpus
  • Emoji recommendation about sentiment analysis in chatting text.

Projects

*

nlp-tutorial

Natural Language Processing Tutorial for Deep Learning Researchers text

nlp-roadmap

ROADMAP(Mind Map) and KEYWORD for students those who have interest in learning NLP text

distribution-is-all-you-need

The basic distribution probability Tutorial for Deep Learning Researchers text

ai-docstring

A tool that AI automatically recommends commit messages. text

commit-autosuggestions

A tool that AI automatically recommends commit messages. text

matorage

Matorage is tensor(multidimensional matrix) object storage manager for deep learning framework(Pytorch, Tensorflow V2, Keras)

gpt-2-Pytorch

Simple Text-Generator with OpenAI gpt-2 Pytorch Implementation text

xlnet-Pytorch

Simple XLNet implementation with Pytorch Wrapper text

kubeFL

Federated Learning Infra Architecture on Kubernetes(EKS)

ALBERT-Pytorch

Pytorch Implementation of ALBERT(A Lite BERT for Self-supervised Learning of Language Representations) text

toeicbert

Undergraduate Class Project, TOEIC(Test of English for International Communication) solving using pytorch-pretrained-BERT model text

modelsummary

All Model summary in PyTorch similar to model.summary() in Keras text