Distribution Corrected Offline Data Distillation for Large Language Models

Yumeng Zhang, Zhengbang Yang, Yevin Nikhel Goonatilake, Zhuangdi Zhu
August 2026
PDF

Abstract

DISCORD is an offline reasoning distillation framework for large language models that corrects teacher-student distribution drift while preserving the efficiency of teacher-generated supervision.

Type
Preprint
Publication
Efficient Reasoning Workshop at COLM 2026
LLMs Reasoning Distillation Efficient AI
Zhuangdi Zhu
Zhuangdi Zhu
Assistant Professor (Tenure-Track)

My research focuses on making AI models safe and aligned.

Related