What are the common pitfalls and mistakes to avoid when developing and deploying object detection systems?

A critical technique to improve data quality for object detection is data augmentation. By artificially expanding the dataset through transformations such as rotation, scaling, flipping, and color adjustments, data augmentation not only enhances model robustness but also mitigates overfitting. Additionally, ensuring balanced class distributions prevents bias towards overrepresented categories. Employing a combination of synthetic data generation and active learning—where the model iteratively identifies and learns from edge cases—can further address data scarcity and improve detection performance in real-world scenarios.

In my experience adding occluded data during training which is part of the real world helps in learning substance of features that makes detection robust.

In industrial series production is it in my experience important that we get data over different days, product batches maybe under different parameter conditions. Some of the mistakes are, that a lot of similar data is used, which could not generalize the whole production spectrum. If similar vision system running in different machines, also images from each system should be used.

Quality > Quantity And with quality I don't mean resolution, I mean a data set that accurately represents the real world variations. Also be creative with augmentation. You can use a dtandard library for it but it is often better to find creative approaches for your specific problem. Like if an defect occurs on one part, you could just take that region and place it over other places if you have a small data set.

In my experience often the data quality is more important than to find the right model. If the data annotation is not done by domain expert or the models in the images are not marked well you could exchange ai models, without any effect. Data curation should come before model optimization.

An essential technique for effective model selection is leveraging transfer learning. Pre-trained models like YOLO, Faster R-CNN, or DETR can significantly reduce training time while achieving high accuracy on custom datasets. Once a base model is selected, hyperparameter tuning—adjusting parameters such as learning rate, batch size, and anchor box dimensions—plays a pivotal role in optimization. Techniques like grid search or Bayesian optimization can systematically identify the best settings. Additionally, using pruning and quantization can optimize the model for deployment on resource-constrained devices without sacrificing significant accuracy.

During the evaluation and deployment stage, consider evaluating the performance of the object detection method by testing how it performs with occluded parts, overlaps, objects near the edges to check robot collisions and similar (but wrong) parts. The pre-series with data for training is important, but don't forget to test the "common" failure scenarios.

It's good practice to have a large collection of data specifically for the evaluation process and not for training (different from the validation data). So far, however, I have never been able to follow these 'good practices' myself, since 99% of the projects I deal with don't even have a proper collection of defective samples for the initial training.

For evaluation, using precision, recall, and F1-score alongside the mean Average Precision (mAP) metric provides a comprehensive understanding of model performance. It's crucial to evaluate on a diverse validation set that mimics real-world conditions, including edge cases and varying environments. During deployment, adopting techniques like model compression (e.g., pruning and quantization) ensures the system meets hardware constraints. Additionally, employing edge computing can reduce latency for real-time applications. Monitoring the deployed system with live feedback loops and periodically updating the model with fresh data ensures long-term reliability and adaptability in production environments.

For industrial environment: 1. make the best images you can. First try to highlight want you want to see, by optimization lighting and camera system 2. Data curation – make a small perfect data set is better to have a big, bad annotated data set. 3. Use pretrained models from industrial vision software companies will save a lot of time.

What should be considered when designing the architecture of a #VisionSystem for object detection? 1. Is the scene/objects steady or dynamic? Some methods are only available for steady applications. 2. Are the parts textured or are they shiny or translucent? Shiny and translucent objects are more complex to work on. 3. Are the objects rigid or deformable? Rigid objects may benefit from feeding their CAD data into the detection methods. 4. Do we need the full pose of the object or only the XY position? The more degrees of freedom are required, the more complex will be the detection method

Real industrial applications of deep learning algorithms (which cannot be considered trivial) are very often full of unexpected complexities. Projects often start with foggy and unrealistic expectations from customers and evolve into a mess of new requirements, new types of defects never seen before during the feasibility tests, and unspoken working conditions that compromise the existing datasets. This slows down the proper workflow and adds a lot of additional working hours, which I suggest you take into account in your quotation to avoid misunderstandings and upset customers. No one likes to be charged extra for every single software addition, even when there is miscommunication of the requirements.