LeCun’s JEPA study challenges scale-first training

Yann LeCun and Brown University researcher Randall Balestriero have presented research suggesting that training methods can matter more than parameter count for aligning text and visual representations. Their text JEPA, trained with 109 million parameters, scored 0.184 on a cross-modal representation alignment metric, compared with 0.154 for Qwen2.5-72B and 0.151 for Llama-3-70B. Those results do not establish superiority in reasoning, coding or knowledge-based questions.

The approach uses training pairs called Good Views: different observations that describe the same underlying scene. Under specific mathematical conditions, independently trained JEPA models can learn representation spaces that align through a rotation transformation. The framework favors learning shared information across observations rather than modality-specific details. In controlled comparisons, the method reached an alignment level with about 21.6 million training samples that next-token prediction required 900 million samples to achieve.

The researchers also connected separately trained visual and text models through a linear transformation to demonstrate image captioning. On COCO, the resulting system achieved a CIDEr score of 0.838. The findings support LeCun’s longstanding focus on world models and abstract representation learning, while leaving open whether that approach can replace mainstream language models across broader applications.