Understand how robot vision works, when to use 2D or 3D sensing, and how lighting, calibration and error testing turn images into reliable production decisions.
Machine vision turns images into information a robot or production system can use. A camera might locate a component, check whether a connector is present, read a code or measure a feature. The useful result is the decision or position delivered to the next process, not the image on a monitor.
A successful installation joins optics, lighting, software, calibration and control. This guide explains those elements, how to choose between 2D and 3D vision, and how to test a system against the parts and conditions it will encounter. It builds on our robotics introduction and practical cobot guide.
Key Takeaways
Machine vision must deliver a defined inspection, identification or positioning result through the complete control path, with a clear response to missing or uncertain information.
Lighting, optics, part presentation and calibration determine what a camera can reliably observe, so additional software cannot compensate for every physical imaging problem.
Evaluate missed defects and false rejects separately on representative independent data, then verify item tracking and physical action under actual production conditions.
Machine Vision, Computer Vision and Robot Guidance
Computer vision is the broader field of extracting information from images. Machine vision commonly refers to a complete inspection or guidance application in an industrial process, including image capture, processing and communication with machinery.
Robot guidance is one use of vision. Cognex’s guidance documentation describes using a vision system to align a robot with objects and accommodate position changes. A camera may be fixed above a station or mounted on the robot.
Inspection asks whether something meets a requirement. Guidance asks where the robot should act. Identification asks which item or job is present. Combining these functions can be valuable, but each needs its own acceptance criteria.
For example, recognizing a part type does not establish its exact pickup pose. Locating its outline does not prove that a hidden underside is undamaged. State what the camera must determine and what remains the responsibility of another sensor or process.
The Complete Vision Pipeline
A typical application begins with a trigger: a component arrives, a robot reaches an inspection pose, or an encoder reports a position. The system captures an image with specified lighting and exposure, processes the relevant region and produces a result.
That result must remain associated with the correct physical item. If a conveyor carries several components between the camera and a reject station, the controller must track which component received which decision. An accurate classification applied to the wrong item is still a production failure.
Cognex’s system overview describes the combination of image acquisition, analysis and communications with devices such as a robot or programmable controller. Treat those interfaces as part of the design, not work to finish after the camera has been selected.
Define what happens when no image is acquired, no object is found or the result arrives late. The system needs an explicit outcome for uncertainty rather than silently reuse a previous position or decision.
Start With the Feature You Need to See
Write the requirement in physical terms. “Check assembly quality” might mean confirming that four screws are present, detecting a damaged seal or verifying that a label belongs to the selected product. Those are different imaging problems.
Identify the smallest relevant feature, its contrast against the background, the range of possible positions and the conditions under which it can be obscured. If a defect is hidden inside an opaque housing, an external camera cannot directly inspect it.
Collect examples of acceptable variation as well as defects. A scratch may be unacceptable on one surface and irrelevant on another. A cosmetic shade change may be normal between material batches. The engineering and quality teams need a shared definition before the software is tuned.
A clear specification might say: confirm a particular connector is seated within a defined visible position range, across approved color variants and normal fixture tolerances. It should also state the required response when seating cannot be determined.
2D or 3D Vision?
Two-dimensional images can be useful when the relevant information appears in a controlled view: markings, edges, patterns or a component lying on a known plane. Three-dimensional sensing adds information about depth or surface geometry, which can help when height and orientation vary.
Cognex describes inspection, measurement and robot guidance among the uses of its 3D systems. The choice still depends on the object, required detail, motion and sensing conditions. A more expensive sensor does not automatically solve poor presentation or an unclear defect definition.
Task
Possible approach
Important limitation
Read a label
Controlled 2D view
Glare, print quality and motion can obscure characters
Locate flat parts
2D pattern or edge matching
The assumed working plane must remain valid
Check surface height
Depth or profile measurement
Sensor uncertainty and surface properties affect the result
Pick mixed orientations
3D sensing with pose estimation
A detected pose still needs a feasible, verified grasp
Ask suppliers to explain what information the proposed sensor measures and what the software infers. This helps distinguish a directly observed feature from an assumption built into the algorithm.
Lighting and Optics Come Before More Software
The camera records light. Reflections, shadows, transparency and changing ambient illumination can therefore change the input even when the part is identical. Stable presentation and appropriate lighting can make a difficult software problem much simpler.
Consider a shiny metal surface. A bright spot that shifts with the part may resemble a defect in one view and hide a real defect in another. Testing a different light position or imaging geometry may be more effective than adding examples to a classifier.
Field of view and detail also trade off. In an illustrative setup, 2,000 pixels covering 400 mm gives a nominal scale of 0.2 mm per pixel across that direction. This arithmetic describes image sampling; it is not a claim of 0.2 mm measurement accuracy.
Lens behavior, focus, calibration, contrast and image processing influence the final measurement. Ask for performance on the feature that matters, at the expected working distance and across the relevant area of the image.
For moving parts, include capture timing and motion blur in the trial. A crisp stationary image does not demonstrate inspection performance at production speed.
Calibration Connects the Image to the Robot
A robot moves in physical coordinates, while a camera initially observes image coordinates. Calibration establishes the relationships needed to translate a detected feature into a useful robot position. When the camera moves with the arm, its relationship to the robot also matters.
NIST’s calibration research discusses the integration challenges of robot workcells, including robot-sensor registration. The practical implication is simple: a vision algorithm can find the correct image feature while the physical motion remains wrong because the coordinate relationship is inaccurate.
Check performance across the working area rather than at one convenient point. Include the fixture and tool that will be used in production. If a camera mount is moved, a tool is changed or a fixture is replaced, decide which checks are required before the system returns to service.
Keep a known reference or verification procedure so the operating team can distinguish a calibration shift from a change in part appearance. Repeatedly adjusting a detection threshold will not fix a displaced camera.
Rules-Based Vision and AI
A rules-based approach applies explicit operations such as finding an edge, measuring a distance or matching a trained pattern. A learned model derives its behavior from training examples. Either can be appropriate; a single application can also combine them.
Choose based on the requirement and the variation. A clearly visible dimension may suit a conventional measurement tool. A difficult appearance classification may benefit from a learned approach, provided there is enough representative data and a reliable labeling process.
For AI systems, separate training examples from evaluation examples. Images that differ only slightly because they came from the same capture session may not reveal how the model behaves on a new material batch or a different shift.
Define an uncertain outcome if the application needs one. A model’s score is not automatically a calibrated probability that an assembly is acceptable. The threshold should be justified by observed error behavior and the consequence of each type of mistake.
Why One Accuracy Number Can Mislead
Inspection usually has at least two costly errors: accepting a bad part and rejecting a good one. They have different operational consequences, so report them separately.
Consider an illustrative evaluation of 1,000 parts: 20 are genuinely defective and 980 are acceptable. The system detects 18 defects, misses two and incorrectly rejects 30 good parts. It makes 968 correct decisions overall, giving 96.8% accuracy.
That same result means 90% of the defects were detected, two defective parts escaped and 30 good parts required unnecessary rejection or review. Only 18 of the 48 rejected parts were actually defective. The headline accuracy does not communicate those tradeoffs.
Results also depend on the evaluation mix. A test containing almost entirely acceptable parts can produce a flattering accuracy figure while providing little evidence about rare defects. Report the sample composition and the types of defects tested.
Agree the acceptance limits with the people who own quality and production. An appropriate threshold for a cosmetic sorting task may be inappropriate for an assembly feature whose failure has serious consequences.
Build a Representative Test Set
Use approved product variants, different batches and normal changes in position, illumination and surface condition. Include difficult acceptable parts near the quality boundary, not only obvious good and bad examples.
Ground truth: Establish the reference decision through a documented quality process.
Coverage: Record which product, defect and operating conditions each example represents.
Separation: Keep evaluation data independent from training and routine tuning.
Traceability: Save the software version, configuration and decision associated with each test.
Do not infer an extremely low failure rate from a small fault-free trial. A result of zero observed misses applies to the tested sample; it does not prove that the system will never miss a defect.
Run the final evaluation through the real control path, including triggers, item tracking and reject actuation. A folder of correctly classified images does not verify that the right products leave the line.
Robot Picking Needs More Than Detection
A detected object must have an accessible grasp, adequate clearance and a placement that the robot can execute. Other objects, the container wall or the tool geometry may prevent a seemingly obvious pickup.
After the attempt, verify the physical result. A command to close the gripper is not proof that a part was acquired. Decide whether grip confirmation, a second view or another check is needed before the robot continues.
Our report on multi-step manipulation illustrates the broader importance of task state and recovery. Vision supplies evidence to that process; it does not remove the need to manage an interrupted or unsuccessful action.
For an initial project, consider whether a better tray or fixture can eliminate unnecessary pose variation. A simpler physical input can reduce both perception errors and grasp failures.
Maintain the Result After Commissioning
Plan cleaning, reference checks, replacement parts and controlled software changes. Dust on a lens, a loose mount or a substituted light can change performance even if the algorithm is untouched.
Monitor false rejects, escapes discovered downstream and instances where an operator overrides the decision. Investigate changes by product and shift before retraining or widening thresholds. The cause may be the process rather than the model.
Store enough information to reproduce a problem without collecting unnecessary sensitive data. Define image retention, access rights and the owner of labeling and configuration changes.
A useful machine-vision system produces a trustworthy action under known conditions. Start with the decision, make the feature visible, verify the coordinate and control interfaces, then measure the errors that matter. Our automation guide connects those checks to the wider deployment process.
For the next physical step after locating a part, our gripper guide explains how to verify acquisition, retention and release.
Frequently Asked Questions
Is machine vision the same as AI? No. Many vision applications use conventional image-processing and measurement tools. AI is one possible component, especially for learned classification or difficult visual variation.
Does a robot always need a camera? No. Fixed presentation and suitable sensors may provide all the information a task needs. Add a camera when it solves a defined inspection, identification or positioning problem.
Is 3D vision always better than 2D? No. Use the information the task requires. A controlled 2D view may be sufficient for a label or flat feature, while variable height or orientation may justify depth sensing.
What is hand-eye calibration? It establishes the spatial relationship between a camera and a robot so visual information can be used for physical motion. Its accuracy must be checked in the intended working area.
How should inspection accuracy be reported? Report missed defects and false rejects separately, together with sample composition and operating conditions. An overall accuracy percentage alone can hide important weaknesses.
Can an ordinary vision camera protect workers? Object detection alone does not make a camera a validated protective device. Safety-related sensing and the complete application require their own appropriate assessment and validation.
