A digital caliper that displays 0.01 mm is showing a resolution. A manual that prints ±0.02 mm is stating a specification. The range of repeats on a known block is an observed spread. None of those three, on its own, is a measurement capability, and this site has not published an observed spread.
How do resolution, a stated figure and observed scatter differ?
Display resolution, a stated accuracy figure and the range of repeats are three different numbers. A 0.01 mm digit is a resolution. A printed ±0.02 mm is a manufacturer statement. The largest-minus-smallest reading on a named block is an observed spread. None of those, on its own, is a measurement capability, and this site has not published an observed spread.
| Column | Typical source | What it is not |
|---|---|---|
| Display resolution | The least count on the digital display, for example 0.01 mm. | Not the accuracy of the instrument, and not a shop capability. |
| Stated accuracy | The manufacturer’s specification, for example ±0.02 mm on a 0–100 mm digital caliper in Sparber’s write-up of that manual. | Not a measured result on your instrument today. |
| Observed range of repeats | Largest minus smallest reading on a named block, same day, same contact. | Not a calibration uncertainty and not a UKAS calibration and measurement capability. |
ISOBudgets writes the standard teaching term for a digital indication’s resolution contribution as Resolution / (2√3). That is a component in an uncertainty budget, for people who are writing one. It is not a certificate, and it does not become a capability because the arithmetic is tidy. The same note is explicit that a grade 0 block used as a reference does not remove the caliper’s resolution as a limiting term.
Why do shop rules of thumb stay labelled?
CNC Cookbook’s “use twice the stated accuracy” line — 0.002 inch from a 0.001 inch statement — is a teaching shortcut from that site. It is not rewritten here as a verified UK workshop standard. Sparber’s ±0.02 mm / ±0.03 mm bands are the manual he was reading, drawn as an envelope around his own later tests. Those tests remain his.
NPL GPG 40 reminds the reader that force and temperature sit outside the display. A stable last digit can still be wrong for the object in the jaws.
Why is the observed column empty on this page?
MeasureProof has not published an observed range of repeats for a micrometer or a caliper. The third column on any capability-style table from this publication stays empty until a dated record exists. Do not read a stated ±0.02 mm as “this site found 0.02 mm”.
A later original series will have to print the stated figure, the display resolution and the observed range as three numbers, with the instrument, the block and the temperature attached. Until then, use the blank record and keep the columns separate.
Which sources does this note use?
Sources checked on 23 September 2026. The numbered list below points to the original publishers, and their sample measurements are not MeasureProof measurements. Manufacturer pages, forum threads and teaching sites are identified separately from UK metrology guidance. No instrument in this note has been tested or certified by this publication.
- ISOBudgets, How to calculate resolution uncertainty ↗Richard Hogan, 28 April 2016. Teaching derivation for a digital indication: Resolution / (2√3). ISOBudgets LLC also sells laboratory consulting.
- CNC Cookbook, Digital caliper complete guide ↗States a common 6-inch digital caliper as 0.001 inch accuracy and 0.0005 inch resolution. That is a teaching statement.
- Sparber, Repeatability and Accuracy of the Harbor Freight Caliper ↗Personal workshop PDF, version 3, 5 June 2007. The specification envelope and temperature cross-test belong to the author.
- NPL Good Practice Guide 40, Callipers and micrometers ↗National Physical Laboratory. UK authority for use conditions, measuring force and temperature. It is not a workshop certificate.
Keep the conditions with the reading.
The CSV has column headings and one empty row. It contains no fabricated sample readings.
Download workshop-check record ↓
