Asked about Making a model smaller
Why does unstructured pruning often not speed anything up?
An answer that works
It zeroes individual weights but leaves the tensor shape unchanged, so the hardware still processes the zeros. Structured pruning removes whole units or layers, which is a genuine speedup at a larger quality cost.
Reading an answer is not being able to give one. The section that teaches this is Making a model smaller, 6 hours of Compression & distillation.
Learn this properly