Machine vision quality control: from feasibility to the production line

Short answer: The outcome of a machine vision quality control project is usually decided before the model: by the written definition of the defect, by whether the camera makes the defect visible at all, and by which kind of error is more expensive. Moving to model selection before these three are settled is the most common loss of time we see.

This post lists the questions we ask, in order, when taking a visual inspection project from feasibility to running on the line. It is written so that a quality and production team can answer them internally before the project starts.

Rule-based or deep learning?

Not every visual inspection task needs deep learning. Dimensional checks, presence/absence checks and checking a label at a fixed position can be solved reliably with good lighting and classical image processing rules. These solutions need less data and their decisions are easy to explain.

Deep learning comes in when the shape of the defect cannot be described in advance: a scratch on a surface, a paint run, flash on an injection-moulded part, irregularity in a texture, defects that look different every time. In practice most lines use both; rules check the dimensions, the model checks the surface.

Situation	Suitable approach
The defect can be described by size or position	Rule-based image processing
The defect shape varies, examples are plentiful	Classification or object detection
The defect shape varies, examples are very scarce	Anomaly detection trained on good parts
Many product variants that change often	Model plus variant management and a retraining plan

Four questions to answer at feasibility

Is there a defect catalogue? Which defects are being looked for, the name of each, a sample photo and the accept/reject limit should be written down. In a project without a catalogue the model is trained without knowing what it is learning, and where two quality specialists decide differently on the same part, the model is undecided too.

How small is the smallest defect? The size of the smallest defect in millimetres directly determines the camera resolution and the field of view. As the field of view grows, the area per pixel grows; if a small defect shrinks to a few pixels, no model can catch it reliably.

What is the cycle time? How many parts pass per second determines by when the decision must be made. That budget selects the model size and the hardware it runs on; we covered the details in our post on vision models at the edge.

How many product variants are there? Every variant that differs in colour, size or material is a separate distribution the model will see. The number of variants and how often new ones arrive determine the maintenance load of the project.

Image first, model second

In a visual inspection project the most decisive work is done in image acquisition. If the camera, lens and lighting do not make the defect visible, model training does not solve the problem; it only postpones it.

That is why the first task at feasibility is an imaging trial with sample parts. The same part is captured under different lighting set-ups: low-angle light for surface scratches, diffuse (dome) lighting for shiny surfaces, backlight for contour checks. What we look at is whether defective and good parts separate in the image. If they do, the model's job becomes easier; if they do not, the problem is solved in the optics.

The position of the part relative to the camera matters as much as the lighting. A part that rotates, slides or vibrates on the conveyor appears at a different angle in every frame. Fixturing, the trigger point and the exposure time are settled at this stage.

The problem of few defective samples

On most production lines defective parts are rare; that is good news, but a challenge for training. There are two routes.

Anomaly detection: The model is trained on good parts only and flags deviation from normal. You can start without waiting for defect samples, and it also reacts to new kinds of defects. Its weakness is that it can flag what is "different but fine" as well: a new raw material batch, a small change in the lighting.

Classification or detection: The model learns each defect type separately. It can report the type of defect, which is valuable for root cause analysis. But it needs enough examples of each type.

A route we often take is to start with anomaly detection and have the quality team label the flagged parts. As examples accumulate, the frequent defect types move to a classification model.

Whichever route is chosen, the labelling guide is written together with the quality team. Who decides on borderline parts, and how that decision is recorded, should be clear from the start.

Metrics: which error is more expensive?

Visual inspection has two kinds of error and their costs are not equal:

• Missed defect: A defective part goes on to the next station or to the customer.
• False reject: A good part is scrapped or the line stops unnecessarily.

A single "accuracy" figure mixes the two. We show performance along a threshold curve and choose the operating point together with the customer. On a safety-relevant part a missed defect is unacceptable, while on a low-cost part the false reject rate can be the deciding factor.

For borderline decisions there is a third option: setting the part aside as "uncertain" and sending it for operator review. This creates a flow that is automatic where the model is confident and involves a person where it is not.

Integration on the line

Inspection is not complete until the model's decision triggers something. What needs to be settled in integration:

• Which protocol carries the decision to the PLC (digital output, OPC UA, fieldbus), and the timing of the reject mechanism (pusher, diverter, line stop).
• Recording every decision together with its image. This is the only way to review a disputed decision after the fact.
• What the line does when the model is not running: stop, or continue while marking parts. This decision should be written down before the project.

After commissioning

A model that works well in the lab meets dust, vibration, changing ambient light and new variants on the line. To keep the model in production, the distribution of its decisions is monitored, decisions corrected by operators are fed back into the training data, and no new model version goes live before being measured on the same test set and compared with the previous one. We cover the infrastructure for this loop under MLOps.

The general frame of our work in this area is on the Computer Vision page; sector-specific problems are on the Manufacturing, Automotive and Plastics pages.

Frequently asked questions

Can we work with our existing cameras? Usually a trial with the existing cameras comes first. If the defect separates in the image, the existing hardware may be enough; if not, only the missing component (lens, lighting, resolution) is changed.

How many defective samples are needed? There is no fixed number; it depends on how variable the defect is. If samples are very scarce, you can start with anomaly detection, in which case images of good parts are enough.

Does the model run on the line or in the cloud? When a decision is needed at line speed, the model runs on an edge device next to the line. The cloud is used for training, reporting and model updates.

What is the quality team's role in the project? The defect catalogue, the labelling guide and the decision rule for borderline parts come from the quality team. The model learns these decisions; the owner of the decision does not change.