POKAIOK: A Standalone AI-Powered Assistant For Enhanced Visual Inspection In Production Lines

Authors: Alexandre Leclerc, Christian Gout, Carole Le Guyader, Pierre Besset, Carlos Miranda, Olivier Gibaru

Abstract:

In this work, we address the challenges faced by human operators in visually inspecting production lines in factories, especially in automotive manufacturing involving numerous checkpoints to be handled every minute for extended hours. This tedious and error-prone process calls for a solution, and we propose an AI-powered assistant to achieve this goal. The objective is to design a system that can efficiently perform visual inspections on moving items and assemblies, in a structured manner, such as on conveyors and production lines. The system task is to visually verify specific part conformity for each assembly or class of assembly. For instance, in an automotive assembly line, the assistant may need to check the presence of headlights and ensure proper rim mounting. To meet these requirements, the assistant must detect the passage of relevant assemblies and perform anomaly detection on relevent parts. Notably, the system copes with the a prior knowledge deficit challenge of which assemblies and parts the users will choose for inspection. Therefore, it needs to be versatile enough to work across a wide range of industrial use cases. By incorporating these considerations, the proposed AI-powered assistant aims to reduce the difficulty, tedium and unreliability of human visual checks on production lines. Constraints specific to the industrial sector underpin the proposal: • Easy and fast setup (< 1 hour). • Strong hardware constraints on GPU power and memory. • Short inference time (< 100 ms). The proposed approach involves leveraging recent advances in mathematical models and computer science tools complying with the above requirements. It can be summarized as follows: 1) Implementing state-of-the-art deep neural networks trained on vast and diverse datasets (known as foundation models) to assist with annotation, facilitating rapid preparation for detector training. 2) Utilizing the latest models capable of handling object detection and instance segmentation tasks to track the objects needing analysis. Result refinement is then achieved through trajectory analysis of the examined areas. 3) Incorporating a pre-trained classifier and fine-tuning it using detector-acquired data. 4) Inferring the classification model on the detections obtained from the trackers to warn the operator quickly when necessary. Furthermore, we subjected these methods to continuous testing in real industrial environments to ensure their practical applicability and effectiveness.

Keywords: anomaly detection, computer vision, deep learning, image and video processing, image segmentation, industrial inspection, production line