What is XLSTM?

Hochreiter talks about an alternative architecture for so-called Large Language Models. At the moment, these LLMs use so-called transformer architectures, which are very brute-force and computationally intensive. XLSTM is an autoregressive approach, can abstract and Hochreiter wants to keep it in Europe. "XLSTM is faster, we need less memory and XLSTM has a linear runtime." What's more, XLSTM can always predict the next word better. This means simpler coding and logic. In addition, the technology from Linz has a memory part and a semantic part. Thanks to this combination, XLSTM can draw better conclusions, explains Hochreiter in the interview.

What is the current status?

Pierer invested several million euros and the company NXAI was founded. The special thing about it: Pierer is taking a risk. The initial results are positive, but there is no product. Rarely are companies in Europe so courageous.

When will the paper arrive?

The AI community is calling for a paper every time XLSTM is published. There will be a paper, but Hochreiter's focus is currently on a product.

Open source or patent?

The question is still open. Hochreiter's goal is to develop XLSTM for the European industry. It was mainly the companies in Silicon Valley that got rich with his invention LSTM. The technology from Linz also has a memory part and a semantic part. Thanks to this combination, XLSTM can draw better conclusions, explains Hochreiter in the interview.

When will the results be available?

At the moment, over 1000 GPUs around the world are computing XLSTM. The focus is on small and medium-sized data sets. The results should be available in March. The large data sets are due in June. Hochreiter and his team want to beat all known models - in all categories.

For whom is this interesting?

The focus of NXAI is on industry. Some German industrial companies are already making appointments in Linz. Its predecessor, the LSTM algorithm, is now also used in many AI applications in industry. It is one of the most cited algorithms in computer science. Over 100,000 citations will be cracked in mid-February.