Perspectives on Floating Point – EigenTales
Some approximation is necessary since there are an infinite number of real numbers – even in a bounded range like between 0 and 1 – while computers have finite memory.
Let’s quantify the error introduced by this rounding. If $x$ is the real number and $\text{round}(x)$ is the rounded number, then the error is given by:
An interesting property of this method is that the maximum round-off error is independent of the magnitude of the number being stored. In the above example, we know that the error is at most $0.05$ for any number $x$.
In fact, this error depends only on which decimal place we round off at. For example, if we round off at the 2nd decimal place instead of the first, then the error is always at most $0.005$. In general,
The error we just discussed is called the absolute error. It is the absolute difference between the real number and the approximation.
Another way to quantify the error is by the relative error. It is just the absolute error normalized by the magnitude of the number:
