Machine vision in automation

Machine vision gives automated systems the ability to find, measure, read and inspect parts. In industrial robotics it serves two broad purposes: guiding robots to parts whose position varies, and inspecting products so that defects are caught before they reach customers. ROB-330 covered the computer vision theory behind both, from the pinhole camera model and calibration to feature matching, depth sensing and learning-based detection, and its capstone sized a camera and lens for a pick-and-place cell. This lesson takes the factory engineer's view: which kind of vision system fits which task, how lighting makes or breaks an application, how to size resolution for inspection and line-scan imaging, how robots pick parts from moving conveyors, and how to judge whether an inspection system is good enough by measuring its error rates and its measurement capability.

What vision does on the factory floor

Industrial vision tasks fall into a few categories. Location finds a part's position and orientation for a robot, or checks that it is correctly placed. Inspection checks for defects: scratches, missing components, wrong colors, contamination, incomplete welds or bad seals. Measurement, often called gauging, determines dimensions to a tolerance. Identification reads barcodes, two-dimensional codes such as Data Matrix and QR, or printed characters through optical character recognition (OCR), increasingly to meet traceability requirements in automotive, medical device, pharmaceutical and food production. Many applications combine several: a system might locate a part, read its serial code, check that all components are present and send coordinates to a robot, all in one image.

Types of vision systems

Smart cameras integrate the image sensor, processor, software and communication in one housing and are configured through a graphical tool rather than programmed. They suit single-camera inspections and simple robot guidance. PC-based systems connect one or more industrial cameras, over interfaces such as GigE Vision or USB3 Vision, to an industrial computer running a vision library or application, and suit multi-camera systems, demanding algorithms and deep learning. Dedicated code readers are optimized for barcodes and two-dimensional codes. Three-dimensional sensors produce depth maps or point clouds: laser triangulation profilers measure a profile line by line as parts move under them, structured-light sensors project patterns to capture a full 3D snapshot of a stationary scene, stereo cameras match features between two views, and time-of-flight cameras measure the round-trip time of modulated light. ROB-330 explained the principles; in practice, profilers dominate inline 3D measurement on conveyors, structured light dominates bin picking, and time-of-flight is common for coarse tasks such as detecting box positions on pallets.

Lighting is the first design decision

Experienced vision engineers say that most failed applications fail because of lighting, not software. The goal is an image in which the feature of interest has strong, consistent contrast against everything else, regardless of variations in part color, ambient light and surface finish. Lighting geometry, color and filtering are chosen to create that contrast before any algorithm runs.

Color helps too. A red LED light makes red features appear bright and green ones dark on a monochrome camera; a filter matching the light's wavelength on the lens blocks ambient light from windows and overhead lamps. Strobing the light in short, bright pulses freezes motion on moving lines and overwhelms ambient light. Infrared and ultraviolet lighting reveal features invisible to the eye, such as UV-fluorescent adhesive beads that confirm glue is present.

Resolution for inspection and gauging

Resolution requirements depend on the task. A common rule of thumb for defect detection is that the smallest defect should cover at least about 3 to 4 pixels across, so that it is distinguishable from noise. For gauging, a rule is that the pixel size should be about one-tenth of the tolerance band or smaller, although sub-pixel edge fitting can do better under good conditions. For reading codes, manufacturers specify a minimum number of pixels per module, the smallest cell of the code.

Line-scan cameras build an image one row at a time as the product moves past, which suits continuous materials such as sheet steel, film, paper and textiles, and cylindrical parts rotated in front of the camera. The camera's line rate must match the product speed so that pixels are square on the product, and an encoder on the conveyor usually triggers each line.

Picking from moving conveyors

Robots often pick parts from conveyors that do not stop. A camera upstream locates each part and records the conveyor encoder count at the moment of the image. The robot controller, also reading the encoder, knows how far each part has traveled since it was seen and computes where it is now. When the part enters the robot's pick window, the robot matches the conveyor's speed, picks the part while moving with it, and leaves. Robot vendors offer conveyor tracking options that handle the coordinate transforms; the integrator calibrates the encoder scale, the camera-to-robot transform and the conveyor direction.

# Simulating missed picks on a conveyor with random part spacing
import numpy as np
rng = np.random.default_rng(440)
def missed_share(rate_per_min, window_s, pick_s, n=200_000):
    arrivals = np.cumsum(rng.exponential(60 / rate_per_min, n))   # time each part enters window
    robot_free, missed = 0.0, 0
    for t in arrivals:
        start = max(t, robot_free)
        if start > t + window_s:          # part has left the window before robot is free
            missed += 1
        else:
            robot_free = start + pick_s
    return missed / n
for rate in (60, 90, 110):
    print(f"{rate} parts/min: missed {missed_share(rate, 1.5, 0.5):.1%}")

60 parts/min: missed 0.8%
90 parts/min: missed 4.5%
110 parts/min: missed 9.8%

Calibration to the robot

For guidance, image coordinates must be converted to robot coordinates. For parts on a flat surface at known height, a simple calibration uses a grid or a set of points whose positions are known in both image and robot frames, giving a planar transformation. For a camera mounted on the robot arm, a hand-eye calibration determines the transform between the camera and the flange, as ROB-330 explained. Calibration must be repeated if the camera is bumped, the lens is adjusted or the conveyor is moved, so robust installations include a quick verification check, such as picking up a reference part and confirming its position.

Deep learning inspection

Traditional rule-based vision uses explicit algorithms: threshold this region, count these blobs, measure this edge. It works well when defects are well defined and appearance is controlled. Deep learning inspection, discussed in ROB-330 for detection and segmentation, is now widely used for defects that are hard to describe with rules, such as cosmetic flaws on textured surfaces, weld appearance and natural products like food. Supervised classifiers need labeled examples of good and defective parts; anomaly detection models learn only from good parts and flag anything unusual, which helps when defects are rare. Data quality matters more than model choice: images must cover the real range of variation in parts, lighting and position, and labels must be consistent between inspectors.

Vision vendors now package deep learning tools that run on smart cameras or industrial PCs with modest training sets, sometimes a few dozen images per class to start. Validation must still follow the same discipline as any inspection system: a held-out test set, measured error rates and periodic re-checks as products and conditions drift.

Is the inspection good enough?

Every inspection system makes two kinds of errors. A false reject, a type I error, rejects a good part, wasting material and sometimes requiring manual re-inspection. A false accept, a type II error, passes a bad part to the customer, which may be far more costly. Moving the decision threshold trades one against the other, just as with the alarms in ROB-430. The right balance depends on the costs.

# Choosing an inspection threshold from score distributions and error costs
import numpy as np
rng = np.random.default_rng(440)
good = rng.normal(0.25, 0.10, 99_000)      # anomaly scores for good parts
bad = rng.normal(0.70, 0.15, 1_000)        # 1 per cent defect rate
cost_false_reject, cost_false_accept = 2.0, 150.0   # dollars per event
for thr in (0.40, 0.45, 0.50, 0.55):
    fr = np.mean(good > thr); fa = np.mean(bad <= thr)
    cost = fr * len(good) * cost_false_reject + fa * len(bad) * cost_false_accept
    print(f"threshold {thr:.2f}: false reject {fr:.2%}, false accept {fa:.2%}, cost per 100k parts ${cost:,.0f}")

threshold 0.40: false reject 6.86%, false accept 2.10%, cost per 100k parts $16,726
threshold 0.45: false reject 2.35%, false accept 5.80%, cost per 100k parts $13,360
threshold 0.50: false reject 0.65%, false accept 9.00%, cost per 100k parts $14,792
threshold 0.55: false reject 0.13%, false accept 16.30%, cost per 100k parts $24,716

For gauging, the question is whether the measurement system is capable of distinguishing parts within tolerance from parts outside it. Measurement system analysis, required by automotive quality standards such as IATF 16949 through the AIAG MSA manual, uses gauge repeatability and reproducibility (gauge R&R) studies: several parts are measured several times, and the variation due to the measurement system is compared with the tolerance or the process variation. A common guideline treats gauge R&R below 10 percent of tolerance as acceptable, 10 to 30 percent as possibly acceptable depending on the application, and above 30 percent as unacceptable.