Grouped story
ByteDance study finds that asking LMMs questions beats making it transcribe text for long document training
A new study from ByteDance and HKUST reveals that training multimodal AI models, like the MMProLong, is more effective when using question-answer pairs rather than traditional text transcription. This approach allows the model to better navigate long documents, outperforming larger competitors while maintaining stability at high input lengths.
Key points
The MMProLong model outperforms larger competitors with only 128,000 tokens.
Question-answer training improves long-document performance by requiring models to locate relevant passages.
Diverse training examples, rather than just long documents, yield better results in multimodal AI.