Short Survey of Artificial Intelligent Technologies for Defect Detection in Manufacturing

—Zero Defect Manufacturing (ZDM) can be described as the set of methodologies and strategies for the elimination of defective components during production, and is one of the main goals of Industry 4.0. ZDM is very appealing to industries grace to the reduction of operational costs associated with defective components. Efficient defect detection in modern production lines may benefit from Artificial Intelligence (AI) technologies in numerous stages of the manufacturing process. This paper presents a short review of AI technologies employed during product inspection and quality assessment. Indicative applications are reported to demonstrate that AI in its many flavors may be efficiently integrated into production environments and pave the way towards ZDM.

INTRODUCTION Industry 4.0 addresses a wide range of industrial applications, from product design to applied logistics.The opportunities and benefits of the 4th industrial revolution show potential for more flexible mass production, real-time coordination, supply chain optimization, cost reduction, and introduction of new business models.Artificial Intelligence (AI) is expected to play a significant role in Industry 4.0 targeting to enhance several aspects of the manufacturing process, such as novel machine tools for performance improvement, maintenance procedures, energy efficiency 1 Corresponding Author.etc. Two of the key concepts encountered within Industry 4.0 is quality control of products and Zero-Defect Manufacturing (ZDM).The first step towards ZDM is the early detection of defects in products and faults in the production chain, as well as the triggering of proper actions upon successful detection.
Traditional techniques such as Bayesian Networks, Logistic Regression, k-Nearest Neighbor, Support Vector Machines, Decision Trees etc. are already employed in industrial operations.However, in the past decade, more innovative approaches have also been evaluated and assessed.This paper presents the latest advances in AI-based ZDM methodologies and is organized in the following sections.Section II gives a short description of popular AI technologies typically applied to manufacturing.Section III presents applications of some key AI technologies for defect detection, namely AI-enhanced Metrology, Computer Vision, IoT and Quality Assessment.It should be noted that due to paper size limitations, only indicative examples are reported.Finally, Section IV performs an objective comparison of the employed technologies.

A. Machine Learning Methods
Machine Learning (ML) is the ability of smart systems to learn and improve through experience gained from historical data, without need of programming, or any other human intervention [1].ML is considered as a subfield of AI and it mainly pursues the enhancement of the AI predictions.Various types of ML are available, such as Supervised, Unsupervised, Semi-supervised and Reinforcement Learning.Commonly used ML techniques are Artificial Neural Networks (ANNs) and Support Vector Machines (SVMs).Some quite accurate and generally acceptable descriptions for these technologies are provided in [2], [3].
ANNs are commonly implemented in applications, where dynamic modeling, prediction, data classification or clustering is needed.To this end, their employment into industrial applications has shown that ANNs can successfully address problems such as, defect detection, health monitoring and classification, as well as fault diagnosis of industrial machine tools (e.g., bearings, gears, motors, engines, etc.) exhibiting a high accuracy rate and performance.SVM-based methods are widely used in occasions where high accuracy and precision in data classification is required, as well in regression tasks (a.k.a.Support Vector Regression, SVR).Practically, SVMs can be applied as a solution in prognostic problems, i.e., to identify defects in industrial machine tools, as well in health monitoring of the industrial equipment.

B. Deep Learning
Deep Learning (DL) is an extension of ML and describes the ability of smart systems to imitate human brain functionality in tasks such as decision making and data processing.The most popular DL methods are shortly described next: • Deep Neural Networks (DNNs) are similar to ANNs with more than three layers, trained to model nonlinear problems.
• Convolutional Neural Networks (CNNs) are mainly employed in image processing applications (semantic segmentation, image classification, instance segmentation, object detection, etc.).Their neurons architecture is based on the features of images they process (width, height, depth, etc.).Typical CNNs have a similar structure with ANN and consist of one or more filters (i.e., convolutional layers), followed by aggregation/pooling layers in order to extract features for classification tasks.

III. AI TECHNOLOGIES FOR DEFECT DETECTION
ZDM is a key concept in smart manufacturing [4], [5].It aims to create production lines without defective products and operational faults during the manufacturing procedures, which eventually could lead in the minimization of scrap parts, rework, and special machining operations.ZDM for zero defective products -considered herein -consists of several modules for product monitoring as shown in Fig. 1.Products are inspected multiple times along the production line.Measurements are collected via distributed sensor networks and transmitted -or even processed in-situ, by edge devices using IoT and Industrial IoT (IIoT) technologies.Edge computing is a distributed computing paradigm that brings computation and data storage closer to the location where it is needed to improve response times and save bandwidth [6].Data processing leads to the assessment of product quality and triggers corrective actions if necessary.

A. AI in Metrology and Computer Vision
Products can be evaluated either by traditional metrology techniques or by more innovative "no-touch" tactics like computer vision.Both approaches can be leveraged with the aid of AI technologies.

AI-enhanced metrology
Traditional metrology systems are usually hindered by data ambiguity, limited sampling rates and data processing capabilities.AI in Metrology is the field of artificial intelligence that trains computational systems to conduct precise measurements of key features on a product, despite the potential limitations of physical equipment.It is clear that uncertainty and measurement errors are rather impossible to eliminate completely, albeit, it is possible to minimize them and, therefore, improve the overall measurement quality [7].
AI has been employed in metrology systems for more than 15 years.Chang et al. [8] presented a virtual metrology technique based on ANNs that is used in the semiconductor manufacturing industry.Due to the small sizes involved in wafer quality assessment, classical metrology methods proved inefficient.Therefore, a combination of ANN methodologies, such as Piecewise Linear Neural Network (PLNN) and Fuzzy Neural Network (FNN) were employed.The system was trained on historical data of the production line and was able to detect defective products.The proposed approach offered a low-cost method to check the quality of every single part.
Ferreira et al. [9] developed a virtual metrology model for predicting the thickness of Plasma-enhanced chemical vapor deposition (PECVD) oxide films.Two mathematical models were applied synergistically: Partial Least Squares Regression (PLSR) and Decision Tree Ensemble (DTE).The interlayer thickness was classified using metrology data towards defect detection.Contrary to the previous work, AI was employed to improve metrology indirectly by enhancing data processing.The proposed method was capable to detect inhomogeneities during chemical mechanical polishing in real-time.
More recently, Kang and Kang [10] demonstrated an intelligent metrology system based on ANNs.The proposed model targeted to replace the actual metrology system without affecting the overall metrology performance.The key underlying architecture was composed of two discrete modules.First, a reliability module assessed the predicted measurement.If predictions exceeded preset limits of ambiguity, the second module suggested proper updates to the model.The proposed model yielded quite accurate measurements, at a very low cost compared to an actual metrology system.Zhou et al. [11] employed convolutional neural networks in Fourier-transform profilometry.The image recognition and feature extraction capabilities of CNNs were exploited to enhance the method.Specifically, CNNs were applied to analyze the spectrum image in order to identify carrier frequency components associated with the details of the inspected object.
In Pimenov et al. [12], several AI models were applied, such as random forest, standard Multilayer Perceptrons (MLP), Regression Trees, and radial-based functions to provide information during the manufacturing process regarding the expected surface roughness of the fabricated product.In addition, the parameters tuning of the employed models was accomplished via the grid search and the proposed methodology was tested on face-milling of a structural steel 45.
Papananias et al. [13] developed an intelligent metrology informatics system based on ANNs for multistage manufacturing processes.The goal was to assess the quality of a product after a manufacturing process, which included heat treatment and machining.They employed a MLP network with eight inputs, one hidden layer with 10 neurons and one output.The inputs were associated with measurable properties, such as RMS values from inspection sensors, surface hardness of the material, etc.The predicted results were validated against experimental measurements and were found in good agreement.The presented model lacked the capability to quantify prediction uncertainties, thus, further development and improvement is possible.
Lee and Kim [14] used CNNs for virtual metrology during semiconductor manufacturing operations.The proposed model combined a Recurrent Neural Network (RNN) and a CNN to extract time-dependent and timeindependent features.The model performance was compared to its best-known competitor (elastic-nets) and an 8.48% improvement was achieved.
Rendon-Barazza et al. [15] proposed the incorporation of artificial intelligence into optical metrology systems.They employed deep learning analysis in optical microscopy and achieved measurements of accuracy around 0.77nm, which is comparable to the accuracy of electron beam and ion beam methods.
Kotsiopoulos et al. [16] developed a quality assurance system that is applicable in machining operations.The suggested technique automates the 3D inspection and the monitoring process of defective metal components via the employment of Deep Neural Networks.The necessary data were extracted from real production processes utilizing shopfloor sensors, an ultrasound scanner as well as a laser microprofilometer.The production monitoring module analyzes data from the above-mentioned sensors for quality control tasks and suggests a fusion scheme in order to enhance even more the accuracy of the manufacturing procedure.
Charalampous et al. [17] presented a method that employed various regression-based machine learning algorithms to estimate the dimensional deviations between an additively manufactured product and its corresponding nominal digital 3D model.The introduced methodology was validated in real-life manufacturing parts with complex geometrical characteristics.Furthermore, a compensation technique was applied adjusting the dimensions of the digital 3D model in order to compensate the overall dimensional deviations of the printed object increasing that way the performance of the AM process.

AI-enhanced computer vision
AI computer vision is the field of artificial intelligence that trains computational systems to visually interpret the real world.This means that computers should be able to identify objects and patterns by processing digital images, videos etc. using AI techniques.Computer vision can be further extended to object classification, i.e., identify an object in an image, and instance segmentation, i.e., identify multiple objects of the same class within the same image.
Feng et al. [18] employed Deep CNNs to achieve highspeed 3D imaging.They combined deep learning with profilometry using the structured light illumination.Upon proper training, the system could convert fringe images to 3D shapes at a rate of 20,000fps.
Liu et al. [19] employed deep learning for real-time 3D surface measurement in additive manufacturing.Their method consisted of a supervised deep neural network for image analysis.They key idea was to find a correlation between 2D images and 3D point cloud data; this was achieved using a convolutional neural network.
Ozdemir & Koc [20] developed a visual quality control application using deep learning methods for use in smart factories.A camera was positioned at the end of the production line and captured images of each fabricated item.An ANN was first employed to detect objects.Then, more advanced models like RNNs, and You Only Look Once (YOLO) were employed to recognize each object and identify the defective ones.The object inspection system triggered proper actions to discard defective products away from the assembly line.A major benefit of the approach is that it runs in parallel with the production line, and gives results in realtime, providing this way a fast and efficient quality control.
Chouchene et al. [21] employed AI computer vision for visual inspection and product quality control in vehicle industry.An automated artificial vision system was utilized to supervise the production line and eliminate the defective products along manufacturing and assembly lines.The captured images were enhanced in terms of contrast correction, denoising, etc., and were further analyzed via supervised and unsupervised ML classification techniques, namely ANNs and SVM.The outcome of the integrated system was to identify components that do not meet the product quality specifications.Assessment and classification were validated by human operators.

B. AI-enhanced IoT/IIoT
IoT technologies are employed in various applications facilitating real-time monitoring, analytics and decision support systems.IIoT is the industrial counterpart of IoT, which can play an important role in Industry 4.0.The terms IoT/IIoT are used to describe a network connected to the internet, composed of embedded smart devices like sensors and actuators, enabling the interconnection between these devices and the whole world.Apart from the sensors, actuators and the network infrastructure, an IoT system also includes data analytics tools which collect and analyze data.The main purpose of these systems is to alleviate latency, increase scalability, and enhance the access to information so more reliable and faster decisions can be made based on the collected data.
The main task of smart IoT/IIoT devices is to actually implement AI algorithms on edge devices.TinyML [24] is one of those recent developments in AI that enables the employment of machine learning and deep learning technologies on embedded devices.The popular TensorFlow (TF) library has been ported (TF Lite) to mobile and IoT applications [25] and platforms; Arduino Nano BLE SENSE and IoT [26], [27], the Sparkfun edge [28] etc.To this end, such low-cost devices can accommodate state-of-art ML algorithms without the need for intensive processing power.Of course, efficient AI algorithms are necessary to be adapted so as to employ a robust machine learning model of a small footprint, and therefore be able to fit into the limited hardware of a microcontroller; algorithm development and training still remains a complex and computationally demanding task, and heavily relies on high-end systems such as servers and cloud/fog computing.However, the arrival of MicroPython [29] opens new research directions for miniaturized smart devices.MicroPython is a Python 3 flavor optimized to run on microcontrollers.AI code can be developed on a PC and then transferred to a microcontroller.This way, data acquisition, processing and decision making can take place on-site and in real-time.
IoT/IIoT sensor networks can benefit from the emerging Blockchain technologies [30], [31] as a decentralized architecture for secure and reliable data sharing among nodes.Rathore et al. [32] developed a blockchain-based decentralized security IoT architecture named Block.Sec.IoT.Net.This approach is based on Software Defined Networking (SDN), blockchain and fog computing.It is a network-based model, whose basic role is to locate cyber-attacks in traffic data in IoT networks.Singh et al. [33] employed a blockchain-enabled AI-enhanced IoT architecture named as Block.IoT.Intelligence.In this architecture, blockchain technology was used as a secure way of communication and interaction between the data centers, fog nodes and database stations located in each layer in this IoT.The expected purpose of this model was to provide in secure IoT for industrial applications.Kamath et al. [34] proposed a Digital Twin (DT) model based on IoT opensource platforms Eclipse Hono, Eclipse Ditto, Apache Kafka, Influx DB and Grafana.The collective functionally of these platforms was developed in order to enable digital twin capabilities, including IoT real-time data acquisition, virtual representation and management, real-time analytics, and visualization.The main challenge of this model is the real time operation to achieve predictive maintenance in the industry during the manufacturing process.

C. AI for quality assessment
It must be noted that the production of high-quality endproducts coupled with minimum manufacturing cost is a high priority in industry.Industry 4.0 has already exhibited its potential via the utilization of core technologies such as AI and Machine Learning in order to reach the abovementioned goals as described in the previous paragraphs.Therefore, through the utilization of AI algorithms, manufacturers could enhance the product quality and achieve quality assessment.After having all sensor data collected and processed, AI can be employed to classify products based on predefined quality standards.
Jegadeeshwaran and Sugumaran [35] proposed a ML defect detection system in automobile companies for testing the quality of hydraulic brakes in vehicles.This approach was based on vibration signals applied on the products, which were subsequently classified into faulty and healthy based on signal processing.SVM, Decision Trees and other statistical approaches were employed in the study.Finally, the authors proposed that Radial Basis Function (RBF) applied on SVM model provides the highest accuracy rate.
Wu et al. [36] introduced a random forest based prognostic technique for predicting the tool's wear in machining operations.The results showed that the utilized algorithms demonstrated better performance compared to more classical machine learning methods like feed-forward back propagation artificial networks.The inputs for the development of the algorithms were collected from cutting forces, vibrations, and acoustic emission during the material removal process.The authors declared that in future work, they will focus in applying these techniques in large-scale and real-time prognosis.
Scime and Beuth [37] presented a multiscale convolutional neural network for the autonomous anomaly detection and classification in laser powder bed diffusion additive manufacturing processes.The proposed neural network could detect the anomalies and other key information at multiple size scales.The authors claimed that the proposed network was more efficient than previously applied methodologies.
In Vafeiadis et al. [38], an early stage-decision support system was utilized to inspect printed circuit boards and investigate the inference faults due to the deposition of excess glue on the board.More specifically, a pixel-wise vector of the inspected areas was applied coupled with various stateof-the art machine learning algorithms to evaluate the efficiency of the proposed defect detection system.The results exhibited that Support Vector Machine (SVM) polynomial classifier achieved the best performance.Cunha et al. [39] employed an approach that combines Non-Destructive Testing (NDT) and ML for quality control tasks.Inhomogeneities and defects are detected in ceramic tiles via Acoustic Emission Testing (AET).The information about product quality is processed by classification ML techniques such as SVM and Nearest Neighbors.As concluded in the study, combining AET with a ML technique, the classification precision reaches a 95% hit rate.
Lin et al. [40] proposed a cascading convolutional neural network for the detection of defects on steel surfaces.Quality control was performed in two stages.First, modified "single shot multibox detector model" was used to learn possible defects, and then, deep residual network were employed to classify three types of defects, namely rust, scar and sponge.The model was experimentally assessed using industry datasets and exhibited high precision and recall scores.
Wiciak-Pikuła et al. [41] employed neural networks to predict the wear of cutting tools during milling of aluminum matrix composites.MLPs were employed to associate the wear level of the tool with acceleration and cutting forces.This enables timely replacement of the tool in order to preserve the quality of the machining and increase the quality of the machined component.Spruck et al. [42] employed DNNs to achieve quality assurance of seam welding operations.Their system was actually a DNN-based classifier, for labeling images obtained by laser triangulation.Training of the system required image assessment by experts.Upon training, classification accuracy was over 96%.Kwon et al. [22] employed DNNs to analyze laser images and assess the quality of the microstructure in metal sheets.They attempted to correlate the microstructure of the material to the laser power and pixel intensity.The proposed model exhibited a hit rate over 98.9% in classification.Laser imaging was also applied by Massaro et al. [23], where a camera system with laser was utilized to track the exact position of the examined item during the manufacturing process.This vision technique was based on k-means classification of the extracted data in order to determine possible defects.
Meiners et al. [43] proposed a two-stage batch control system of piezo actuators based on a ML architecture.It was used for batch process optimization and focused on the final product quality.Batch-control oriented ML approaches involve ANNs, SVM and Partial Least Squares models (PLS).All models were used to predict the sintering temperature in a batch of raw materials.They used material and processing characteristics, and along with the quality standards for the product as inputs.
Brito et al. [44] employed machine learning in a collaborative robotic environment for quality inspection.A robot was responsible for inspection and corrective actions in the quality control system, supported by an intelligent system that could learn and adapt to the inspected parts.The underlying method was reinforcement learning.
San-Payo et al. [45] developed a ML approach for clothing industry.In particular, a quality classification system was developed in this work, which examines the quality of produced materials and identifies defective ones during manufacturing.A CNN monitored the production line with a camera, and acted as a classifier by analyzing captured images.The proposed system was able to perform qualitybased classification of produced textile objects by comparing them with a reference prototype model.

IV. COMPARISON OF AI-USAGE
An overall comparison of the presented studies is included herein.For the sake of shortness and clarity, comparisons are summarized and visualized in graphs.
It is rather obvious that ML is becoming a trend for computer vision and quality assessment in production lines.As shown in Fig. 2, more than 80% of reported works seem to be relying on various AI-methods and machine learning to identify objects and classify them based on their quality.To shed some light, the usage of individual machine learning methods is further analyzed for the three investigated tasks discussed herein, namely metrology (Fig. 3), computer vision (Fig. 4), and quality assessment (Fig. 5).Regarding AIenhanced metrology, regression-based methods seem to be in favor (27%), followed by ANNs (18%), CNNs (18%) and Recurrent NNs in one of their variants.
ANNs and Deep NNs are also very popular among computer vision applications, as well.Their learning capacity has brought them into a dominant position, being employed in roughly half of the case studies (Fig. 4).ANN usage (50%) is followed by CNNs and Recurrent NNs at smaller utilization ratios (17% respectively).
Regarding quality assessment procedures, ANNs are again the choice of preference, along with SVMs (22%), followed by CNNs (17%).In quality assessment operations, the preference of the individual AI-technology seems to be more scattered compared to the previously-mentioned tasks.This could be attributed to the fact that quality assessment is a complex task that requires more sophisticated algorithms, therefore, multiple AI-technologies are continuously assessed.

V. SUMMARY AND DISCUSSION
The goal of ZDM is to yield a production line with no defective products.This is a rather complex and challenging task, with multiple aspects and approaches.This survey focuses on specific tasks that can be used in a production line, namely AI-enhanced metrology, computer vision and quality assessment.These tools are technically supported by IoT/IIoT technology that mainly implement AI-algorithms for data processing and sharing.
In particular, in smart manufacturing, deep learning models are the most commonly applied models for image analysis, classification and quality control.Implementation of deep learning in imaging is mainly conducted via CNNs, a relatively new and powerful technique to learn useful representations of images and other structured data.Due to the popularity of CNNs in vision systems, several applications of CNNs were investigated in the field of defect detection in various manufacturing procedures.Review studies and published articles in the last five years gather all the important, innovative, and most interesting applications of deep learning technologies.
With the introduction of CNNs, features from vision systems could be learned directly from the provided data, since they contain certain preferences in their structure that make them powerful deep learning models for image analysis and quality control procedures in production lines.Other deep learning architectures like GoogleNet, AlexNet, ResNet etc. have also been applied in manufacturing for classification and visual analysis tasks.In some cases, AI-technologies are employed coupled with other mathematical tools, like Decision Trees, k-Nearest Neighbor etc., and form hybrid methods with elevated efficiency.This work is planned to be extended towards a twostep process.As a first step, additional technologies for supporting manufacturing activities will be considered, such as AIenhanced digital twins and AI-enhanced augmented reality.Then, each of these technologies will be deeply investigated to extract their current state of exploitation within AI technologies, and develop new innovative algorithms to further improve them.The goal is to derive a unified AIframework and efficiently deploy it in a smart factory floor.
Authorized licensed use limited to: Foundation for Research and Technology (FORTH).Downloaded on July 22,2022 at 07:49:13 UTC from IEEE Xplore.Restrictions apply.

Fig. 2 .
Fig. 2. Use of Machine Learning vs. other methods in in computer vision and quality assessment.