Startups look beyond transformers as AI research shifts

Startups are pursuing alternatives to the transformer, the neural network architecture that powers every major large language model. As models grow, transformers are becoming a bottleneck because dense attention gets more expensive as text volume increases and struggles to track large amounts of information at once. New ideas under exploration could make LLMs faster, more efficient and potentially more capable.

University AI researchers are also adjusting to a shifting environment. The Schmidt Sciences AI2050 program, funded by Eric and Wendy Schmidt, has gathered prominent academics whose work involves AI, reflecting both the field’s scientific momentum and the unusual pressures facing researchers based in universities.

AI infrastructure remains a major focus for investors. Nvidia has secured $500 billion from Wall Street for AI infrastructure through deals with BlackRock, Goldman Sachs and others, underscoring the pull of compute and the emergence of AI infrastructure as an asset class. Other developments include Mark Zuckerberg’s open-source AI manifesto, Bernie Sanders calling for a pause in AI development, and lawsuits over social media platforms’ addictive design moving forward in US court.