Resolution of the sensors

This article specifies the measurement resolution of our sensors within the binary sbgECom message protocol and its consistency with the accuracy of the data contained in these messages. 

Overview

Our sensors use:

• 32-bit messages for Euler angles (roll, pitch, heading)
• 32-bit raw IMU data outputs (accelerations and rotations)
• 64-bit messages for position (latitude, longitude, altitude)

This means they have a 32-bit resolution for Euler and IMU data, and a 64-bit resolution for position.

The rest of this article details how this resolution translates to the actual measurement and why the resolution is sufficient with regards to the accuracy of the measurements.

Resolution of specific force measurements 

Our IMU (Inertial Measurement Units), AHRS (Attitude and Heading Reference System) and INS (Inertial Navigation System) systems output specific force measurements: accelerations and rotations. These values are represented in m/s² or rad/s using 32-bit (4 bytes) integer numbers with a scaling factor (LSB - Least Significant Bit).

This method allows for a large range of values to be represented without the variable resolution and precision errors associated with floating-point numbers.

Resolution of accelerometers data

The accelerations are using a fixed scaling of 1 048 576 LSB for 1 m/s²: 

The resolution is therefore of 1/1 048 576 m/s² which can be rounded to 1x10^-7  m/s².

Gyrometers data

For rotation measurements, the gyrometers use variable scaling to accommodate different levels of dynamic motion. The two scaling factors are:

The IMU automatically switches to high-range scaling when the measured values exceed the threshold of 1 833°/s. However, since most applications operate below this threshold, we will focus on the standard scaling.

The resolution is therefore of 1 / 67 108 864 rad/s which can be rounded to 1x10^-8  rad/s or 1x10^-6 degree/s.

Resolution and accuracy of the specific force measurements

The resolution of the specific force measurements (rotations and accelerations) are of 1x10^-7  m/s² and 1x10^-6 degree/s.

The smallest relevant measurement possible with our IMUs is the bias instability which are of the order of magnitude of 5x10^-5  m/s² and 1x10^-5 degree/s. Therefore, the resolution of the IMUs does not introduce any additional loss of accuracy to the measurements, ensuring that the precision of the data remains unaffected by the measurement process.

Resolution of Euler angles

Our AHRS (Attitude and Heading Reference System) and INS (Inertial Navigation System) output Euler angles in radians, represented as 32-bit floating-point numbers (4 bytes). Below is an explanation of the floating-point format and its implications for accuracy and resolution.

What is the floating point representation?

We use the IEEE 754 standard to represent floating-point numbers in binary. In this standard, a 32-bit single-precision floating-point number (commonly referred to as float) is divided into three main components:

The floating-points numbers are expressed as follows:

• Sign bit (1 bit)
• Exponent (8 bits)
• Mantissa (or Fraction) (23 bits)

With the float value being:

Here:

• The sign determines whether the number is positive or negative.
• The exponent represents the power of two, adjusted by a bias of 127.
• The mantissa (fractional part) gives the precision of the number, representing values as sums of negative powers of 2.

Example: Storing Pi in single precision floating point

Let’s consider an example with the number π (approximately 3.1415927). In IEEE 754 single-precision format, it is stored as:

Breaking this down:

• Sign: 0 (positive number)
• Exponent: 128 (binary 10000000), which gives 128−127=1

• Mantissa: 0.57079637050628664788187 (sum of powers of 2: 2^-1 + 2^-4 + 2^-7 + 2^-10 + … + 2^-23)

The value can be calculated as:

Special Cases for Values Between 0 and 1

For values where the exponent is zero (binary 00000000), the representation changes slightly. Such numbers, called subnormal or denormal numbers, are calculated as:

This allows for representing very small numbers between zero and one.

Floating points accuracy and resolution

Floating-point numbers provide a flexible way to store a wide range of decimal values, but this comes with some limitations in terms of accuracy. Not all decimal values can be precisely represented as floats. For example, the value 0.1 cannot be exactly stored as a float; it is stored as 0.100000001490116119384765625, introducing a small error.

This error is of the order of 1.4011×10⁻⁹  in this case.

When using floats, the error should be assessed to ensure it is smaller than the required accuracy for the application (the error introduced by the representation should be an order of magnitude smaller than the least significant digit).

The smallest difference between each value that can be represented on a floating point numbers increases with larger numbers: while the spacing between two values within [1;2] is 2^-23, it is 2⁰  (=1) for values >8 388 608 (meaning above 8.4 Millions it is no longer possible to represent decimal numbers, only integers).

The smallest difference between two representable floating-point numbers is formally called machine epsilon (ε). To keep this article accessible we avoid using the more technical term and instead will use the notions of step or resolution.

This spacing is inherent to the representation and can be clearly seen in the formula (−1)^Sign×2^{Exponent−127}×(1+Mantissa): since the mantissa is multiplied by 2^Exponent  the higher the exponent, the higher the gap between each floating point value for each possible value of the mantissa.

The granularity of values can be represented as follows (source: Wikipedia):

Accuracy and resolution of Euler angle

SBG Systems' INS and AHRS output Euler angles in radians, using single-precision floating-point numbers (32-bit floats). The Euler angles are defined as:

The values are expressed in the following ranges:

• Roll (φ) : Rotation around X axis defined between [-π ; π] ;
• Pitch (θ) : Rotation around Y axis defined between [-π/2 ; π/2] ;
• Yaw (ψ) : Rotation around Z axis defined between [-π ; π].

Since, as we saw above, the resolution decreases at higher values, let’s analyze the worst case scenario and determine resolution at the maximum value: π radians.

Since floating-point numbers have a dedicated bit to indicate the sign, the behavior described here for positive numbers (like +π) applies in exactly the same way for negative numbers (such as -π).

To determine the resolution we will measure the gap between π and its closest values:

We can therefore deduce that, in the worst case scenario, we can output variations up to 3×10⁻⁷ radians, or approximately 2×10⁻⁵ degrees.

Given that the typical accuracy of Euler angles in AHRS or INS systems is 0.005° (5×10⁻³), the resolution provided by a 32-bit float (≈2×10⁻⁵ degrees) is more than adequate. Thus, the single-precision floating-point format introduces no significant error and provides sufficient resolution for these applications.

Resolution of other measurements

Our AHRS and INS can output a variety of messages in addition to the Euler angles described above.

For each message, we ensure the selected format provides sufficient resolution to maintain the accuracy of the measurement. While most of our messages are expressed using IEEE 754 single-precision floating-point numbers (32-bit floats), some are expressed in double-precision floating-point numbers (64-bit doubles) when higher precision is required.

This is particularly important for values like latitude and longitude, where large numbers are involved. Using single-precision floats would result in a resolution of around 1 meter at high latitudes or longitudes, while our systems can measure positions to sub-centimeter accuracy. By using double precision, we achieve a resolution of approximately 3 × 10⁻⁷ cm, which far exceeds the accuracy provided by any global positioning system.