For example, on sorting potatoes, a conventional vision system might reject anything with a dark spot larger than 5 mm, a green area covering more than 2% of the surface or an irregular shape beyond a threshold. An AI-based system, trained with thousands (or millions) of images labeled by experienced quality inspectors can begin to recognize defects like pressure bruises, hollow-heart indications, scab, black dot disease, greening, mechanical damage, soil versus true discoloration, sunburn and insect damage.

Evolution of Vision Systems and AI/ML

Historically, inspection criteria were built around what a human operator or quality team could feasibly spot-check at a frequency set by plant food-safety and quality plans across multiple lines and high production volumes, says Courtney Crichlow, KPM Analytics project manager. Those checks were inherently limited not by what was actually detectable, but by human bandwidth. There’s been growing opportunity to audit product quality more consistently and catch foreign material risk more comprehensively. Much of the equipment on the market today is still built to meet historical inspection standards rather than to push forward on reducing food waste, improving product quality and preventing consumer-facing foreign material incidents.

"Traditional machine vision works extremely well when we can clearly define the requirement," says Ryan Park, engineering and application manager, SPI Automation, part of the Grote Company Family of Brands. "If a package needs to be a certain dimension, a label has to be present, a feature needs to be in a particular location or a color has to fall within a defined range, conventional vision can be very effective."

EXPLORE MORE

"Vision-based AI expands what we can realistically inspect" Park adds. "When we train an AI/ML-based learning system, we use examples. That way, the system can learn a product’s normal variation and recognize patterns or anomalies that would be difficult to describe with conventional rules. The setup is different as well. With traditional vision, much of the work is developing and tuning rules, thresholds, measurements and lighting. With an AI system much of the work shifts to obtaining representative images and classifying them correctly."

"We used to spend up to a month building a single visual inspection," says Peter Greif, senior business unit director for Jabil. An engineer would input 50 ways the product should look and 50 ways it shouldn’t, but that approach was never successful. Now, an automated visual inspection system learns from the line, and every call goes back into the AI model in real time — and offline image collection disappears, Greif says.

"The quality of that training data matters," Park adds. Showing a system 50 nearly identical examples of a good product isn’t necessarily as valuable as showing it the actual range of acceptable production — different shifts, raw materials, colors, sizes and normal process variation.

Traditional vision systems rely on fixed thresholds to detect, for example, the number of pixels in a blob or the value of a grayscale or color gradient in a region of inspection (ROI), says Andy Buteyn, senior vision engineer, Gray AES – Robotics and Vision. That hard-coded math works well for manufactured items but can lead to high false-reject rates for foods with natural variation. Today, AI evaluates overlapping variables like texture, color gradients and shape context, allowing systems to distinguish subtle environmental variations or acceptable characteristics from true defects such as rot, blemishes or imperfections caused by processing or cleaning. A traditional system can be quick to deploy and tune (depending on what is being inspected), but every shift in appearance due to growing conditions or ingredient variation can require an engineer to manually rewrite those rules. Setting up an AI system can shift that workload to a dedicated camera or PC for data collection, where quality teams can label real-world images so the model continuously learns what acceptable product looks like.

Siemens provides a comprehensive suite of inspection equipment, software and integration services for the food and beverage sector. This includes advanced AI-driven visual inspection systems like the Visual Inspection Cockpit (VIC) on Industrial Edge for defect and overlap detection, packaging integrity and quality control.

According to Miao He, Siemens Industry, Inc. system engineer, with Siemens VIC semantic segmentation on overlapped objects capability, the loose objects can be segmented into target categories in pixel accuracy level, more than bounding box area. From the segmentation result, the metrology analysis can be implemented to enable the sorting criteria such as object size, object geometry measurements (like roundness, squareness and color contract degree) and object surface quality measurements (like pits, dents, contamination, etc.). The criteria can also adapt regional standards if needed. For example, potato sorting for Russet potatoes in Idaho differs from fingerling potatoes in Washinton, and VIC can adapt the specific requirements without changing the underlying hardware or software backbone.

Non-AI/ML inspection systems require the operator user to constantly monitor the machine settings to ensure that the operation is set correctly, says Emmett Keim, Eriez Inspection Division global product manager. This is one of many reasons producers implement hourly quality checks for the inspection system. ML-based machines can help mitigate system setup errors that occur during production.

For example, Eriez PrecisionGuard X8 metal detectors only require one product pass compared to five-plus passes in today’s metal detector market. The ML algorithm is conservative so as not to pass a "no-good" product.

Notwithstanding, there’s a common misconception that AI vision models are usable and immediately valuable with minimal setup, Crichlow says. This is likely carried over from general knowledge and widespread news around consumer-facing large language models (LLMs). Traditional rule-based vision systems have a comparatively quick turnaround after installation, often requiring only a handful of setting adjustments to build inspection parameters for different materials. AI-based vision requires meaningfully larger initial data sets to inform the model on product variability, defect prevalence, background irregularity and the specific texture, shape, color and shadowing characteristics of the foreign materials or defects being targeted.

While it may seem that rules-based systems provide quick turnaround, these traditional systems typically require a vision specialist to manually fine tune parameters, says Robb Robles, director of machine vision product management at Zebra Technologies. Today, tools like Zebra Aurora Deep Learning learn from example. "Instead of programming rules, we train the system using images labeled as good or bad," Robles says. "The AI learns to identify complex and subtle defects, like bruises, scabs or slight discolorations by recognizing intricate patterns in texture and shape. With Zebra’s deep learning tools, the training is often unsupervised or sample-based, requiring a minimal amount of real-world examples to help the model learn acceptable variations. This dramatically reduces setup time and makes it easier to adapt to new product variations compared to reprogramming traditional rules from scratch."

Metal and X-Ray Detection Slower to Adopt AI/ML

This upfront data and training requirement is sometimes seen as a hurdle by manufacturers accustomed to the fast turnaround of longstanding equipment like metal detectors and X-ray systems, Crichlow adds. These technologies are highly effective but fundamentally different in approach.

"AI hasn’t come to X-ray in a way that actually works, but I can tell you how it probably will come about," says John Hilgendorf, marketing at ABM Equipment. "Right now, machines use auto-learn to establish algorithms on certain parameters like density difference, shape, size, etc., and then operators fine-tune the parameters to avoid rejects they know to be good, and to reject bad product that was missed under the auto-learn algorithms. Current applications of AI in X-ray are way too slow, likely because people are trying to train AI to inspect images, which are large — and there are usually multiple images of each product. So the best application of AI in X-ray will likely not be inspecting images but setting the algorithm. An operator would tell an AI ‘Good product, bad product, good product, bad product…,’ then the AI would use the input to write the algorithm. An AI would be able to write better algorithms than technicians because it could edit fundamental code, setting different conditions for environment conditions, region in the product, upline equipment, etc. This could make algorithms near-perfect without the AI having to process 5 MB images."

Combine Vision, X-Ray, Metal Detection and NIR for the Big Picture

Vision systems probably won’t directly detect internal issues — say, a potato that has grown around a nail, metal fragment or stone — or has hollow or black heart or internal browning. Why not add the other modes of inspection to get a complete picture of product?

Visible-light cameras can only make decisions based on information available at the surface, says SPI Automation’s Park. "If the concern is inside the product, another sensing technology may be required. The technology depends on what we are trying to detect."

The question shouldn’t necessarily be, "Which camera do we need?" It should be, "What physical characteristic are we trying to detect?" Once we understand that, we can determine whether RGB vision, 3D, NIR, hyperspectral, thermal, X-ray — or some combination — makes sense, Park adds.

Because the onus is on processors to produce contaminant-free products, Crichlow says it’s not uncommon to see KPM vision systems in production environments with X-ray systems and mental detectors located nearby. NIR and hyperspectral systems have their own benefits and limitations in food processing environments.

NIR/hyperspectral and X-ray are valuable additions and can be integrated directly with vision rather than run as standalone end-of-line checks, says Siemens’ He. "NIR/HSI adds chemical composition and internal-quality detection (such as moisture, browning, ripeness), and X-ray adds density-based foreign-object and structural checks. Data from all sensors can be fused in a platform like the Visual Inspection Cockpit for a single pass/fail decision."

X-ray can be valuable for detecting foreign material such as metal, stone, glass and bone. Commercial food X-ray systems are already being used not only for contamination detection but also for measurements and grading. Near-infrared and multispectral imaging provide different information, Park says.

"X-ray is the best method for rejecting hollow hearts and other embedded contaminants, and we have a number of systems in the field doing those things," says ABM Equipment’s Hilgendorf. "X-ray can be integrated with vision, but it has to be two different systems, basically just sharing a reject. But they have to be on the same conveyor, because if they transfer between conveyors, product will jostle and the reject will become less reliable. As far as location, X-ray is most commonly in final product because producers want to catch contaminants (like a pen) if a worker accidently drops one in. But other locations are at intake to verify quality of supplier’s inputs, before certain processes (like before cutters you’ll want to remove golf balls because they break the knives), or before packaging when either the packaging reduces efficacy or you have a high enough percentage of contaminants that leaving them in will waste too much packaging."

Standard RGB cameras are limited to surface inspection. Detecting internal defects requires penetrating technologies like X-ray for dense foreign materials (metal, stone) or NIR/hyperspectral imaging for moisture and chemical issues like hollow heart. These technologies are highly complementary to Zebra’s machine vision ecosystem, Robles says. "A common setup places an X-ray or NIR system inline with our vision controllers. Our vision software supports standard GigE and CXP camera interfaces, allowing developers to fuse data streams from third-party cameras and sensors with our surface-level and 3D data."

Another thought on placing these systems from Andy Buteyn, Gray AES: "These systems can be more effective when placed upstream, right after raw receiving or washing to eject out-of-spec product before it reaches downstream equipment that could be damaged or contaminate otherwise good batches of product. From an integration standpoint, we can combine optical AI vision, X-ray and hyperspectral data into a single control scheme. This gives processors a unified internal and external quality profile through a single synchronized rejection mechanism."

Integration Challenges with AI/ML-Based Vision

AI/ML integration challenges are varied. "Proficient ML powered sorting systems typically require a wide calibration range by capturing samples below and above a certain threshold," says Eriez’s Keim. "The strongest benefit of this new technology is that it can extrapolate beyond the calibration points established during the initial product setup. Non-AI/ML based vision inspection systems simply cannot extrapolate beyond the calibration set."

"The most common misstep I see with AI or machine learning systems is in model training," Buteyn adds. "In simple applications, less is more. What I mean is that it is possible to ‘overtrain’ the system. Using a dataset that is too large or too narrow can cause it to fail when exposed to normal line variations." Buteyn notes AI requires much more powerful GPU edge processing, which will require managing continuous large data pipelines feeding into PLC and MES systems.

A traditional challenge with AI/ML systems has been the complexity of software integration and the need for large datasets, Robles says. "AI inference engines typically require high processing power and complicated data science expertise to link with machine control systems (PLCs). At Zebra, we’ve specifically engineered our platforms to eliminate these hurdles. Zebra Aurora Design Assistant provides a drag-and-drop, no-code environment that allows users to construct applications by configuring flowcharts rather than writing traditional program code."

Like its regular vision predecessors, AI-based vision still needs good lighting and repeatable product presentation, says SPI Automation’s Park. "You still have to manage motion, line speed, vibration, washdown conditions, sanitation and changing ambient conditions."

The biggest challenge with integrating vision inspection is understanding that cameras can only inspect what is visible to the lens, says KPM’s Crichlow. Metal detectors and X-ray systems can reliably detect dense contaminants down to ≤1 mm because those technologies inspect an entire product or inspection area using penetrating methods.

The ideal integration for vision inspection is to utilize reported data and detections to make informed decisions around product quality and food safety, Crichlow adds. However, not all applications are equal. A facility inspecting frozen fruit in a monolayer can rely on a vision system as a final inspection point in place of human operators, whereas a beef processor may need a frequency and severity matrix to handle foreign material detections considering only the top portion of trim is visible to cameras. Defect and foreign material inspection is optimized in processes with repeatable and consistent product appearance.

AI/ML in the Future

The exciting part of AI exists in the opportunities this technology holds, Crichlow says. Almost anything is possible when it comes to AI, with the tradeoff being that every solution has an associated time, resource and financial investment. Reducing false-positive rejects is one of the clearest and most measurable benefits of moving to AI-based vision inspection, but this comes with a caveat worth understanding. AI models make assumptions based on the data presented in their training set, and accuracy and confidence vary depending on what the specific model was built to identify.

For example, an AI model trained to identify foreign material in a packaged potato casserole meal could flag a large black spot within the potatoes as rubber, when in reality the spot is a bruise or rot from the IQF potatoes used in the meal. The model identified an issue based on the most similar data in its training set, but no actionable change occurs until that detection is clarified as a separate defect or foreign material class in the underlying data.

One of the more interesting parts of working with AI is learning what models identify that processors didn’t realize was a potential issue or recurring problem in their manufacturing process. Following AI model deployment on many KPM systems, various defects and foreign material types have surfaced that were previously unknown yet prevalent in facilities where vision inspection was newly implemented. These types of detections often lead to meaningful conversations about how manufacturers want to use their vision systems and how to better inform their AI models moving forward, Crichlow says.