Improving Influence-based Instruction Tuning Data Selection for Balanced Learning of Diverse Capabilities

Qirun Dai, Dylan Zhang, Jiaqi W. Ma, Hao Peng

Under Review

When instruction tuning LLMs for learning multiple diverse tasks, we identify the poor performance of data selection methods built upon gradient-based influence estimation techniques, and attribute this problem to an inherent bias in cross-task influence.

We then propose BIDS, a Balanced and Influential Data Selection algorithm that addresses this problem and through instance-level normalization and iterative selection.

When training on UltraInteract, a SOTA high-quality dataset designed to enhance diverse reasoning capabilities, we show that a 15% subset selected by BIDS can outperform full-dataset training in terms of the overall performance on 7 benchmarks spanning coding, math, STEM, logical reasoning and instruction following. We provide in-depth analyses on what might be the good properties of a balanced set of influential data.

Demonstration Distillation for Efficient In-Context Learning

Tong Chen, Qirun Dai, Zhijie Deng, Dequan Wang

Submitted to ICLR 2024

To optimize context efficiency for LLM in-context learning (ICL), we developed DGS, an agentic framework that iteratively compresses demonstrations with three LLM agents: Distillist, Generalist and Specialist.

DGS achieves up to a 4.3x compression ratio and a 5% accuracy improvement on various QA tasks, showing that overly concise demonstrations under human perception can still be informative for LLMs to learn at inference time, and elicit even stronger reasoning capabilities.