Asked about Making a model smaller

How do you decide whether a compressed model is acceptable?

An answer that works

Measure on your own eval rather than trusting published aggregates, because degradation is uneven across capabilities. Then compare size, latency and quality together, which makes it a product decision with numbers rather than an argument.

Reading an answer is not being able to give one. The section that teaches this is Making a model smaller, 6 hours of Compression & distillation.

Learn this properly

Also asked about Compression & distillation

What is knowledge distillation?Why does unstructured pruning often not speed anything up?Your feature uses a large model and costs too much. Would you distil it?Why use temperature in distillation?