Tether open-sources TurboQuant, with a local AI device KV cache compression ratio of up to 5 times
The Tether AI research team announced the open-source release of the TurboQuant production version and its integration into the QVAC SDK 0.12.0.TurboQuant is based on a memory compression algorithm from Google Research, which can compress the KV cache of AI runtime by up to 5 times while maintaining output quality close to that of uncompressed models.This means that laptops, mobile phones, and edge devices can handle longer conversations, larger files, and more complex tasks without the need to upload data to the cloud.This open-source release includes a complete quantization pipeline, mainstream inference framework adapters, and developer documentation, aimed at developers and startups deploying AI on consumer-grade hardware, edge devices, and peer-to-peer networks.
