Publications

2024

1. PACLIC

   RICoTA: Red-teaming of In-the-wild Conversation with Test Attempts

   Eujeong Choi, Younghun Jeong, Soomin Kim, and Won Ik Cho

   In Proceedings of the 38th Pacific Asia Conference on Language, Information and Computation, Dec 2024

   Paper
2. NAACL

   Tree-of-Question: Structured Retrieval Framework for Korean Question Answering Systems

   Dongyub Lee, Younghun Jeong, Hwa-Yeon Kim, Hongyeon Yu, and 6 more authors

   In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 6: Industry Track), Jun 2024

   Abs Paper

   We introduce Korean language-specific RAG-based QA systems, primarily through the innovative Tree-of-Question (ToQ) methodology and enhanced query generation techniques. We address the complex, multi-hop nature of real-world questions by effectively integrating advanced LLMs with nuanced query planning. Our comprehensive evaluations, including a newly created Korean multi-hop QA dataset, demonstrate our method’s ability to elevate response validity and accuracy, especially in deeper levels of reasoning. This paper not only showcases significant progress in handling the intricacies of Korean linguistic structures but also sets a new standard in the development of context-aware and linguistically sophisticated QA systems.

2022

1. EMNLP

   KOLD: Korean Offensive Language Dataset

   Younghun Jeong, Juhyun Oh, Jongwon Lee, Jaimeen Ahn, and 3 more authors

   In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Dec 2022

   Abs Paper Data

   Recent directions for offensive language detection are hierarchical modeling, identifying the type and the target of offensive language, and interpretability with offensive span annotation and prediction. These improvements are focused on English and do not transfer well to other languages because of cultural and linguistic differences. In this paper, we present the Korean Offensive Language Dataset (KOLD) comprising 40,429 comments, which are annotated hierarchically with the type and the target of offensive language, accompanied by annotations of the corresponding text spans. We collect the comments from NAVER news and YouTube platform and provide the titles of the articles and videos as the context information for the annotation process. We use these annotated comments as training data for Korean BERT and RoBERTa models and find that they are effective at offensiveness detection, target classification, and target span detection while having room for improvement for target group classification and offensive span detection. We discover that the target group distribution differs drastically from the existing English datasets, and observe that providing the context information improves the model performance in offensiveness detection (+0.3), target classification (+1.5), and target group classification (+13.1). We publicly release the dataset and baseline models.

2. AACL

   Assessing How Users Display Self-Disclosure and Authenticity in Conversation with Human-Like Agents: A Case Study of Luda Lee

   Won Ik Cho, Soomin Kim, Eujeong Choi, and Younghun Jeong

   In Findings of the Association for Computational Linguistics: AACL-IJCNLP 2022, Nov 2022

   Abs Paper

   There is an ongoing discussion on what makes humans more engaged when interacting with conversational agents. However, in the area of language processing, there has been a paucity of studies on how people react to agents and share interactions with others. We attack this issue by investigating the user dialogues with human-like agents posted online and aim to analyze the dialogue patterns. We construct a taxonomy to discern the users’ self-disclosure in the dialogue and the communication authenticity displayed in the user posting. We annotate the in-the-wild data, examine the reliability of the proposed scheme, and discuss how the categorization can be utilized for future research and industrial development.

3. CODI@COLING

   Evaluating How Users Game and Display Conversation with Human-Like Agents

   Won Ik Cho, Soomin Kim, Eujeong Choi, and Younghun Jeong

   In Proceedings of the 3rd Workshop on Computational Approaches to Discourse, Oct 2022

   Abs Paper

   Recently, with the advent of high-performance generative language models, artificial agents that communicate directly with the users have become more human-like. This development allows users to perform a diverse range of trials with the agents, and the responses are sometimes displayed online by users who share or show-off their experiences. In this study, we explore dialogues with a social chatbot uploaded to an online community, with the aim of understanding how users game human-like agents and display their conversations. Having done this, we assert that user postings can be investigated from two aspects, namely conversation topic and purpose of testing, and suggest a categorization scheme for the analysis. We analyze 639 dialogues to develop an annotation protocol for the evaluation, and measure the agreement to demonstrate the validity. We find that the dialogue content does not necessarily reflect the purpose of testing, and also that users come up with creative strategies to game the agent without being penalized.

2021

1. NeurIPS

   KLUE: Korean Language Understanding Evaluation

   Sungjoon Park, Jihyung Moon, Sungdong Kim, Won Ik Cho, and 27 more authors

   In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), Oct 2021

   Abs Paper Data Website

   We introduce Korean Language Understanding Evaluation (KLUE) benchmark. KLUE is a collection of eight Korean natural language understanding (NLU) tasks, including Topic Classification, Semantic Textual Similarity, Natural LanguageInference, Named Entity Recognition, Relation Extraction, Dependency Parsing, Machine Reading Comprehension, and Dialogue State Tracking. We create all of the datasets from scratch in a principled way. We design the tasks to have diverse formats and each task to be built upon various source corpora that respect copyrights. Also, we propose suitable evaluation metrics and organize annotation protocols in a way to ensure quality. To prevent ethical risks in KLUE, we proactively remove examples reflecting social biases, containing toxic content or personally identifiable information (PII). Along with the benchmark datasets, we release pre-trained language models (PLM) for Korean, KLUE-BERT and KLUE-RoBERTa, and find KLUE-Roberta-large outperforms other baselines including multilingual PLMs and existing open-source Korean PLMs. The fine-tuning recipes are publicly open for anyone to reproduce our baseline result. We believe our work will facilitate future research on cross-lingual as well as Korean language models and the creation of similar resources for other languages. KLUE is available at https://klue-benchmark.com.

2. CLSW

   Automatic Prediction and Linguistic Interpretation of Chinese Directional Complements Based on BERT Model

   Younghun Jeong, Ming Yue Li, Su Min Kang, Yun Kyung Eum, and 1 more author

   In Chinese Lexical Semantics Workshop, Mar 2021

   Abs Paper Code

   The Chinese directional complement is one of the trickiest concepts for second language learners due to their derivative meanings. In particular, 出来, 起来, 下来, and 下去 are easy to be confused. This study aims to gain grammatical and educational insights for these complements with a neural network model. This study fine-tuned the Chinese BERT model with Chinese directional complement data composed of sentences containing the above four components used in literature, media, and textbook. By measuring these fine-tuned models’ accuracy, we show how accurately and efficiently the neural network model predicts Chinese directional complements. Furthermore, we interpret and analyze the model’s decision using the Sampling and Occlusion algorithm and visually present which components of the sentence influence the choice for complements.

3. DSTC9@AAAI

   TOM: End-to-End Task-Oriented Multimodal Dialog System with GPT-2

   Younghun Jeong, Sejin Lee, and Jungyun Seo

   In Proceedings of the AAAI Workshop: 9th Dialog System Technology Challenge, Jan 2021

   PDF