Context compression finally works in production: new research cuts LLM input 16x without the accuracy hit

LCLMs compress LLM context before decode — 8.8x faster at 16x compression, beating every KV cache method tested. Open-sourced by NYU and Columbia.
VentureBeat

LLMs are surprisingly great at compressing images and audio, DeepMind researchers find

Large Language Models (LLMs), often recognized as AI systems trained on vast amounts of data to efficiently predict the next part of a word, are now being viewed from a different perspective. A recent ...