Nvidia researchers have introduced a new technique that dramatically reduces how much memory large language models need to track conversation history — by as much as 20x — without modifying the model ...

Google researchers have proposed TurboQuant, a method for compressing the key-value caches that large language models rely on during inference. In a preprint, the team reports up to six times lower KV ...