From a Lossless (~1.5:1) Compression Algorithm for Llama2 7B

This paper attempts to address and reconcile two different issues: the existence of multiple numerical data formats (such as int8, bfloat16, fp8, etc., often non optimal for the application and not directly compatible with one another) and the necessity to reduce their bandwidth requirements, especially in the case of power hungry and slow DRAM.

Related Keywords

Tim Dettmers , Ruslan Svirschevski , Tim Dettimers , Jarek Duka , Furu Wei , Alexander Borzunov , Shuming Ma , John Gustafson , Vincenzo Liguori , Zhongyu Wang , Artix Ultrascale , Kintex Ultrascale , Eric Winsor , Beren Millidge , Foundation Models , Nvidia , Connections For Efficient Neural Networks , Ocean Logic Pty Ltd , Asics , Cnn , Ocean Logic Pty , Large Language Model , Asymmetrical Numeral Systems , Flexible Numerical Data , Bit Layer Multiply Accumulator , Inference Example , Weight Compression , Open Foundation , Their Own Game , Efficient Neural Networks , Language Model , Democratizing Foundation Models , Training Quantization , Sparse Quantized Representation , Large Language , All Large Language Models , Neural Inference Processor Supporting Compressed , Variable Precision Weights , Stanford Seminar , Variable , Precision , , Orange , Ncompressed , Numeric , Data , Types , Inn , Blm ,