A startup claims it broke through a bottleneck that’s holding back LLMs

Subquadratic, an AI startup, has announced a significant breakthrough in large language model (LLM) technology. The company claims to have solved a long-standing mathematical bottleneck that has limited LLMs for nearly a decade. This innovation could lead to more efficient and powerful AI systems.

The company's new LLM, named SubQ, reportedly outperforms existing models in several key areas. SubQ is said to be faster, more cost-effective, and consumes considerably less energy. Furthermore, it boasts the ability to process up to 12 times more text at once, enabling it to handle data-intensive tasks such as analyzing extensive documents or entire codebases.

Initially, Subquadratic's claims were met with skepticism due to a lack of detailed evidence. However, recent independent evaluations conducted by Appen have validated many of the company's assertions. These findings suggest that SubQ can match the performance of top models from leading AI companies like Google DeepMind, OpenAI, and Anthropic in crucial tasks, including coding.

The core of Subquadratic's innovation lies in its departure from the "dense attention" mechanism, which is central to current transformer-based LLMs. Dense attention involves a quadratic expansion of computations as text length increases, making LLMs power-hungry and inefficient. Subquadratic's SubQ utilizes "sparse attention," a method that drastically reduces the number of computations by selectively processing relationships between words.

While sparse attention has been explored before, previous attempts struggled to maintain performance comparable to dense attention models. Subquadratic asserts that SubQ is the first sparse-attention LLM to successfully rival the performance of mainstream dense-attention models. This advancement could pave the way for a new era of efficiency in LLM development, potentially rendering current transformer architectures obsolete in the coming years.