According to the textbook, we say a measurement is precise when series of measures are close to each other.

But in real use, I feel like the term is used in a different meaning.

For example, scale A weighs to cube of lead and displays 10.5g while scale B displays 10.4977g. In this case, one say the measurement made by scale B is more precise than that of A. That is, scale B can distinguish more fine weights than A.

Isn't this usage of the word precise different from textbook definition? Please explain clearly the real usage of the term precise.

  • 1
    $\begingroup$ Just because a scale shows more digits than another, it does not need to be more precise. $\endgroup$
    – TAR86
    Jan 20, 2017 at 17:38
  • $\begingroup$ "In science, the number of digits to which you report data is important because it indicates explicitly the precision with which the data were gathered." this is a quote from minerva.union.edu/rodbelld/courses/geomorph/sigfigs.pdf. It says like the number of digits represents the precision of a measure. But the textbook definition refers to the set of measures and their closeness. So... I'm confused again. $\endgroup$
    – ben hwang
    Jan 20, 2017 at 17:56
  • $\begingroup$ I would say that this refers to a scientist reporting a figure, not a piece of lab equipment. $\endgroup$
    – TAR86
    Jan 20, 2017 at 18:24

3 Answers 3


In colloquial English, precision is synonymous with accuracy. If you look it up on Google, the first definition is:

the quality, condition, or fact of being exact and accurate.

In the field of science, however, it takes on another meaning:

the degree to which repeated measurements under unchanged conditions show the same results

There is also measurement resolution:

the smallest change [an instrument] can detect in the quantity that it is measuring

Which is similar to the definition precision takes on in numerical analysis:

the resolution of the representation, typically defined by the number of decimal or binary digit

In short, precision can take on different meanings depending on how you choose to apply it. If you talk about the precision in a set of measured data, you will be using the scientific definition; if you talk about the precision of an instrument, you will be using the numerical analysis definition.

  • $\begingroup$ Thanks. But I'm still confused with this sentence(from textbook) "We should not gain or lose precision during mathematical operations. Follow these rules when carrying significant figures through calculations..." Does this sentence using the numercial analysis definition or scientific definition? $\endgroup$
    – ben hwang
    Jan 20, 2017 at 18:28
  • $\begingroup$ The numerical analysis definition. It's saying to have the same number of significant digits in your answer as in your raw data. $\endgroup$
    – ringo
    Jan 20, 2017 at 18:32
  • $\begingroup$ So the sentence means... the usage of significant figure rules is to preserve the resolution of initial data. Is this right? $\endgroup$
    – ben hwang
    Jan 20, 2017 at 18:40
  • $\begingroup$ Yes! Exactly right. $\endgroup$
    – ringo
    Jan 20, 2017 at 18:41
  • 1
    $\begingroup$ @benhwang I think it helps to clarify "precision" used for a single measurement/value, versus "precision" of a group of measurements. When used for a group of measurements, "precision" is how close those measurements are to each other. When there's only a single value, there's no "each other" you can compare among, so it's typically measurement resolution. Or to think about it another way, what's the (group) precision of all possible measurements which round to that resolution? (e.g. "1.5" could really be 1.451, 1.476, 1.5321, etc., so what's the precision of that group?) $\endgroup$
    – R.M.
    Jan 20, 2017 at 21:28

Basic principles and the corresponding vocabulary used in metrology are standardized by the Joint Committee for Guides in Metrology (JCGM). The JCGM is made up of representatives from the International Bureau of Weights and Measures (BIPM), the International Electrotechnical Commission (IEC), the International Federation of Clinical Chemistry and Laboratory Medicine (IFCC), the International Organization for Standardization (ISO), the International Union of Pure and Applied Chemistry (IUPAC), the International Union of Pure and Applied Physics (IUPAP), the International Organization of Legal Metrology (OIML), and the International Laboratory Accreditation Cooperation (ILAC).

The definition of measurement precision (or short precision) can be found in the guide JCGM 200:2012 International Vocabulary of Metrology – Basic and General Concepts and Associated Terms (VIM 3rd edition) as well as in the equivalent international standard ISO/IEC Guide 99:2007 International vocabulary of metrology – Basic and general concepts and associated terms (VIM):

measurement precision
closeness of agreement between indications or measured quantity values obtained by replicate measurements on the same or similar objects under specified conditions

Therefore, the definition given in your textbook is correct.

Furthermore, the JCGM notes that sometimes measurement precision is erroneously used to mean measurement accuracy, which is defined as follows:

2.13 (3.5)
measurement accuracy
accuracy of measurement
closeness of agreement between a measured quantity value and a true quantity value of a measurand


When shooting in a competition, if someone has a tight cluster of hits on a target that is in the corner of the target, you would say they had high precision. They missed, but they were precise. If they were in a loose cluster all around the target, but their center was the bullseye, you would say they were accurate. But if there was a tight cluster centered in the bullseye, you would say they were precise AND accurate.

Does that help? I'm better with analogies than with definitions.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.