quantisation.
Quantisation reduces the numerical precision of model weights and activations — from 32-bit floats to 16-bit, 8-bit integers or lower — which shrinks memory use and increases inference speed.
Where quantisation fits on a long engagement.
The gains are large and mostly free at moderate reduction. Half precision typically costs nothing measurable in accuracy while halving memory, and 8-bit integer quantisation is often acceptable with calibration. Below that, degradation becomes real and model-dependent.
Accuracy loss is not uniform across inputs. A quantised model may perform identically on typical cases and noticeably worse on rare or difficult ones, so an aggregate accuracy comparison can hide a regression exactly where it matters most.
What an assigned team does with quantisation.
Post-training quantisation is quick and sometimes insufficient; quantisation-aware training gives better results at low precision but requires retraining.
Choosing between them depends on how much reduction is actually needed, which is a deployment constraint rather than a modelling preference. Establishing it early is part of the scoping described in how an assignment runs.
What we use quantisation for.
- Memory halved at no measurable cost Half precision where the accuracy difference is not detectable.
- Accuracy checked on hard cases Per-segment comparison, because aggregate scores hide tail regressions.
- Deployment constraints established first The required reduction known before choosing post-training or aware quantisation.
How quantisation capacity is assigned.
Optimisation work is assigned inside AI capacity, with accuracy validated per segment rather than in aggregate.
Tell us what your roadmap needs quantisation for.
A service delivery manager replies with the disciplines we would assign, the monthly capacity and what the first month looks like.
Loading the contact form… You can also email hello@azendo.co.
We reply within one working day. No obligation, and no newsletter.