How robots sense the world

A robot can only act sensibly on what it can measure. Sensors are a robot's eyes, ears, sense of touch and sense of balance, and choosing, mounting, calibrating and interpreting them is a large part of robotics engineering. No sensor is perfect: every one has a limited range, a finite resolution, noise, and conditions in which it fails, such as a camera in darkness or a lidar in heavy fog. Robots therefore combine several sensors whose strengths complement each other. This lesson explains how the most important robot sensors work, starting with sensors that measure the robot's own motion and moving on to sensors that measure distance and see the world. It introduces the key specifications used to compare sensors, works through calculations of encoder resolution, odometry and range, and explains why sensor fusion is needed.

Sensor specifications

Sensor datasheets use a common vocabulary, and understanding it is the first step in choosing a sensor:

• Range: the smallest and largest values a sensor can measure, such as distances from 0.1 m to 30 m.
• Resolution: the smallest change the sensor can detect or report, such as 1 mm or 0.1°.
• Accuracy: how close measurements are to the true value on average; a sensor can have fine resolution but poor accuracy if it is biased.
• Precision or repeatability: how closely repeated measurements of the same thing agree; noise reduces precision.
• Field of view: the angle a sensor covers, such as a camera's 90° horizontal view or a lidar's full 360°.
• Update rate: how many measurements per second the sensor produces.
• Latency: the delay between something happening and the measurement reaching the computer.

A useful picture is a dartboard. Darts clustered tightly but away from the bullseye are precise but not accurate; darts scattered around the bullseye are accurate on average but not precise. Calibration can often remove a consistent bias and improve accuracy, but it cannot remove random noise, which must be reduced by filtering, averaging or better hardware.

Encoders and odometry

An encoder measures how far a shaft has turned. Optical encoders shine light through a disc with slots or a pattern and count the pulses as the disc turns; magnetic encoders sense a rotating magnet. Incremental encoders count pulses from a starting point, so they lose track if power is cut; absolute encoders report the actual angle at any moment. Encoders are specified by their counts per revolution: a wheel encoder with 1,024 counts per revolution divides one turn into 1,024 steps. Many encoders produce two signals slightly out of step, called quadrature, which lets the electronics detect direction and count four edges per slot, multiplying the resolution by four. If the encoder is on the motor before a gearbox, its resolution at the wheel is multiplied by the gear ratio too.

Odometry uses wheel encoders to estimate how far a mobile robot has traveled and turned. Each wheel's distance is the number of revolutions times the wheel's circumference. For a robot with two driven wheels, the average of the two wheel distances gives the forward distance, and the difference between them, divided by the distance between the wheels, gives the change in heading. Odometry is cheap and updates quickly, but its errors accumulate: wheels slip, tires compress, floors are uneven and the wheel diameter is never known exactly. A 1 % error in wheel diameter becomes a 1 m error after 100 m, and small heading errors grow into large position errors as the robot travels. This drift is why odometry is combined with other sensors that measure position relative to the world.

Here \( N \) is the number of encoder counts, \( N_{\text{rev}} \) the counts per wheel revolution, \( D \) the wheel diameter, \( d_R \) and \( d_L \) the distances traveled by the right and left wheels, and \( b \) the distance between the wheels. The change in heading \( \Delta\theta \) comes out in radians.

Inertial sensors

An inertial measurement unit (IMU) combines accelerometers, which measure acceleration along three axes, with gyroscopes, which measure rotation rate about three axes, and often a magnetometer, a compass that senses Earth's magnetic field. Tiny, cheap micro-electromechanical (MEMS) IMUs, the same kind found in smartphones, are used in almost every mobile robot and drone. Gyroscopes are excellent for measuring quick turns, but integrating their readings over time to get an angle accumulates drift. Accelerometers can sense which way is down, from gravity, which helps correct drift in roll and pitch, but they cannot measure heading. Magnetometers can measure heading but are easily disturbed by steel structures and electric motors. A drone uses its IMU hundreds of times a second to stay level, and a legged robot uses one to keep its balance.

Range sensors: ultrasonic, infrared, lidar and radar

Many robot sensors measure distance by timing a signal's round trip. An ultrasonic sensor sends a pulse of high-frequency sound and times the echo. Sound travels at about 343 m/s in air at 20 °C, so the distance is half the round-trip time multiplied by the speed of sound. Ultrasonic sensors are cheap and work in the dark and on glass, but their beams are wide, their readings are affected by temperature and soft surfaces absorb sound. Infrared distance sensors use light, often by triangulation or by timing; they are small and cheap but have short ranges and can be confused by sunlight.

Lidar, short for light detection and ranging, sends out laser pulses and times their return, or measures the phase of a modulated beam. Because light travels at about 300,000 km/s, a lidar must measure time very precisely: a 1 ns timing error corresponds to 15 cm of distance. A two-dimensional lidar spins a beam in a plane and produces a ring of distance measurements, widely used for navigation in warehouses and factories; three-dimensional lidars use many beams or scanning patterns to build point clouds of the surroundings and are used on many autonomous vehicles. Lidar gives accurate distances at long range regardless of lighting, but it is more expensive than cameras, can be degraded by rain, fog, dust and snow, and struggles with glass and very dark surfaces. Radar uses radio waves; it measures distance and, through the Doppler effect, speed directly, and works well in rain, fog and darkness, which is why cars use it for adaptive cruise control, but its resolution is coarser than lidar's.

The first equation gives distance \( d \) from the round-trip time \( t \) of a signal traveling at speed \( v \); the factor of 2 accounts for the trip out and back. The second gives the spacing \( \Delta s \) between neighboring measurement points at distance \( d \) for a scanner with angular resolution \( \Delta\phi \) in radians.

Cameras and depth cameras

Cameras are the richest and cheapest sensors a robot can carry. A single camera captures color, texture and shape, which lets software recognize objects, read signs and labels, follow lines and estimate motion. Machine learning has made cameras far more useful: neural networks can detect people, cars, boxes and thousands of other object types in real time. But a single image has no direct measure of distance; a small toy car nearby and a real car far away can look the same. Robots recover depth in several ways. Stereo cameras use two cameras a known distance apart and find matching points in the two images: the closer a point, the more its position shifts between the images. Structured-light depth cameras project a known pattern and observe how it is distorted. Time-of-flight cameras measure the travel time of light for every pixel. Depth cameras produce images in which each pixel holds a distance, and are popular for indoor robots and manipulation. Cameras need light and can be dazzled by the sun or confused by reflections, and their data must be processed heavily, which needs computing power.

Touch, force and position in the world

Contact sensors range from simple bump switches, which tell a robot vacuum that it has hit something, to force-torque sensors at a robot's wrist, which measure forces and twisting moments in all directions, letting the robot insert a peg into a hole, polish a surface with steady pressure or detect a collision. Tactile skins and fingertip sensors give robot hands a sense of touch, still an active research area. Motor current can also be used to estimate force cheaply. For position in the world outdoors, global navigation satellite system (GNSS) receivers, such as GPS, give positions accurate to a few meters, or to a few centimeters with correction services such as real-time kinematic (RTK) positioning. GNSS does not work indoors, is degraded near tall buildings and trees, and can be jammed or spoofed, so robots that rely on it carry backups.

Sensor fusion and uncertainty

Because every sensor has weaknesses, robots combine them, a process called sensor fusion. Wheel odometry and an IMU give smooth, fast estimates of motion that drift over time; lidar matched against a map, cameras recognizing landmarks or GNSS give absolute position that is slower or noisier but does not drift. Fusing them gives an estimate better than any one alone. The standard tools for this, such as the Kalman filter and particle filter, treat every measurement as uncertain and weigh each by how much it can be trusted. They are covered in the probabilistic robotics course. Even at this introductory level, the key idea is clear: a robot should keep track not just of what it thinks is true but of how sure it is, and should combine sensors with complementary strengths and failure modes.