A Survey of Quantization Methods for Efficient Neural Network Inference

🤖 Plain-English Summary

This chapter provides approaches to the problem of quantizing the numerical values in deep Neural Network computations, covering the advantages/disadvantages of current methods. Loosely related to NN quantization is work in neuroscience that suggests that the human brain stores information in a discrete/quantized form, rather than in a continuous form.

🔑 Key Findings

• Over the past decade, people have observed significant improvements in the accuracy of Neural Networks (NNs) for a wide range of problems, often achieved by highly over-parameterized models.
• Achieving efficient, real-time NNs with optimal accuracy requires rethinking the design, training, and deployment of NN models.
• Model distillation involves training a large model and then using it as a teacher to train a more compact model.

🔗 Related Resources