Maximum speed and detection rates of preferably 100% – these are the requirements for quality control in the packaging industry. The company INNDEO shows how these high requirements can be achieved with a sophisticated automation solution based on machine vision and deep learning technologies.
INNDEO, headquartered in Zaragoza, Spain, offers high-quality machine vision solutions for the automation of quality inspections with its INSPECTRA brand. Founded in 2016, the company has so far mainly been active in the food industry with its solutions and would like to gain a foothold in logistics in the future.
For this purpose, the company has developed the Thermoseal & Label Inspector solution, which can be used to reliably inspect packaging and read labels. The device combines a wide range of equipment sophisticated technologies such as high speed and processing capture with INSPECTRA HSP® technology, hyperspectral vision, deep learning, and high- performance RGB.
Automating packaging inspection from end to end
INSPECTRA’s goal was to develop an end-to-end automated solution for the packaging industry. The advantages offered by such a solution based on machine vision are: Higher detection rates of packaging defects, cost savings, as well as the comprehensive digitalization of production processes in order to be able to monitor and improve them. In practice, many companies’ employees still carry out the inspection process manually.
In this process, defects are often not seen by the human eye. As a result, defective products enter the supply chain and end up with the end customer. The goal of an automated solution must therefore be to reliably detect all conceivable defects in packaging. These include, for example, anomalies in the sealed or heat-sealed area, in the tray, in the inner product, in the film, or the labelling.
Furthermore, 100% automation of quality control reduces costs on the one hand and introduces objective criteria for sorting the products to be inspected on the other. In addition, quality and production data can be continuously digitized and the corresponding indicators displayed in real time.
Although the market offers several other machine vision devices and solutions, these often prove to be not robust enough, low performance, and difficult to adapt to changes in production lines. Due to a lack of precision and reliability in defect detection, many users refrain from using such solutions and prefer to perform defect inspection manually.
INSPECTRA wants to eliminate exactly these deficiencies with its machine vision solutions: “In order to ensure faster inspection processes and more robust detection results than our competitors, we had defined a clear goal. For example, quality defects in food packaging were to be identified at a high production rate of up to two packs per second. This should enable inline rejection, which requires processing times of only a few milliseconds per image,” explains Emilio de la Red Bellvis, Chief Innovation Officer at INNDEO. To achieve these goals, it was essential to automate the application end-to-end using machine vision.
Defect detection via machine vision
So, what does the setup of the Thermoseal & Label Inspector look like in concrete terms? Cameras positioned at various inspection points take images of the objects. These are processed by the integrated machine vision software MVTec HALCON. HALCON is the comprehensive standard software for machine vision, developed by MVTec Software GmbH, based in Munich.
There are different procedures for the various applications. For example, for sealed area inspection, HALCON determines the relevant area (Region of Interest/ROI) of the image for inspection based on various parameters. For this purpose, INNDEO uses high-resolution RGB vision technology for the simplest sealed area defects, such as a piece of ham, as its color is easily distinguishable in a transparent tray. In addition, the company uses hyperspectral vision technology for more complex defects. For example, melted ham fat in the same shade as the plastic of the tray or defects in opaque or printed trays can be detected.
Deep learning, which is a method in artificial intelligence (AI), is also used to detect certain defects. Through deep learning, the software attempts to simulate the behavior of the human brain and is able to interpret the images with a higher detection speed and efficiency than the human eye. The system is able to learn through a training phase, without the need for any additional programming by the user. This technology allows the detection of wrinkles in sealing films, faults in the arrangement of the product in the tray, and the detection of quality defects that cannot be distinguished by standard machine vision algorithms.
Another application scenario is the inspection of the labeling. To detect the label, a corresponding configurable tool looks for a specific pattern. Once this is located, the inspection processes take place. To do this, the application uses the optical character recognition technologies integrated in HALCON, such as OCR (Optical Character Recognition) or Deep OCR. Deep OCR uses a holistic deep-learning-based approach and is able to find and read text in the most difficult conditions, such as characters that are too close together, blurred, or incomplete. This allows various types of label inspection to be conveniently configured. Deep learning technologies and pattern matching of color tones are also used to detect anomalies in the applied labeling, such as wrinkles, creases, or tears, and contaminants such as feathers, hair, or bone traces.
Flexibility in interface integration
For the end customer, it is important that the Thermoseal & Label Inspector can also be seamlessly integrated technically into the existing process environment. This allows the user, for example, to control the inspection system in the environment that is familiar to him. “The integration of an interface was one of the biggest challenges during the implementation. This is because the various parameters of the inspection can be configured from another system control and all the images from the various cameras have to be analyzed in a very short time. It is also important not to forget that the image acquisition devices operate at enormously high speeds. So, the machine vision software has to decide in a very short time whether a package is faulty and has to be rejected,” explains Emilio de la Red Bellvis. Consequently, a lot of programming and debugging work was needed to distribute the entire processing load among the different threads of the microprocessor. In addition, computational operations with high loads – especially those using deep learning – had to be transferred to the computer’s graphics processing unit (GPU). The integration of the solution was simplified by the possibilities offered by MVTec HALCON: “The software offers various interfaces for many types of industrial cameras and the possibility to directly run scripts within a real application using the HdevEngine and debug it. We see the wide variety of image processing algorithms, convenient programming, and seamless integration with our software as further strengths of MVTec products,” confirms Emilio de la Red Bellvis.
On the hardware side, the application setup includes various components such as several latest-generation industrial PCs. These receive the images from the individual cameras and communicate with programmable logic controllers (PLCs). In the process, Emilio de la Red Bellvis faced another challenge: “Given the current shortage of electronic components, we had to develop flexible programming. This allowed us to accommodate different camera types, processing architectures, and GPUs so that the hardware could vary depending on availability.”
Award for most innovative technology
As a result, INSPECTRA has been able to solve all the problems that producers have to face with its Thermoseal & Label Inspector inspection solution by integrating the machine vision software MVTec HALCON. In this way, the quality defects on the packaging as well as on the product itself are avoided. Therefore, costs for the destruction, replacement, and transport of defective products are eliminated. In addition, producers can guarantee their brand promise to consumers at all times due to the reliable and consistent quality. The biggest advantage, however, is that the quality control can be automated end-to-end along the entire production process. This reduces the labor costs incurred for quality assurance, lowers the error rate, and eliminates the subjectivity of the inspection criteria. Last but not least, the technology can detect defects that would have been hidden from the human eye.
“We can be very satisfied with our work because we have overcome the challenges. We are currently in contact with many customers from the food industry who are interested in our products. At the Meat Attraction 2022 trade show, the Thermoseal & Label Inspector even received an award as the most innovative technology for the ancillary industries of the meat industry. The inspection solution was ultimately a result of the interaction of MVTec HALCON with our hardware and optimized INSPECTRA software architecture. Thus, our years of research and development work have fully paid off,” Emilio de la Red Bellvis sums up.
