Contents x
      Error Analysis Troubleshooting
      • 21 Dec 2022
      • 3 Minutes to read
      • Print
      • Dark
        Light
      • PDF
      Contents

      Error Analysis Troubleshooting

      • Updated on 21 Dec 2022
      • 3 Minutes to read
      • Print
      • Dark
        Light
      • PDF
      Article summary

      Refer to this troubleshooting article when debugging your data.

      SignalUse CasePotential Root CauseFix
      • Summary: There are a lot of False Negatives or False Positives.
      • Visualization: The Model draws the bounding box in the right area but detects the wrong Class.
      		The Model confuses detections from Class A and Class B.The Model confuses detections from Class A and Class B.
      • Update the Label Book to provide more details to help users label objects consistently.
      • Review existing Classes and merge if needed.

      Class A or Class B was mislabeled in the Train Set.Check for mislabels and fix them as needed.
      • Summary: Class A has a lot of False Negatives
      • Visualization: The Model says the image is OK due to a low Intersection over Union (IoU). However, the image is a defect, or the Model predicts nothing in the defective area.
      		The Model failed to locate Class A.The object to detect is too small or not visible after resizing.Increase the image size for Model Training, or improve the image resolution.
      The data augmentation is too strong, and the object to detect is no longer visible.Reduce the strength of the data augmentation.
      Users are inconsistently labeling with Class A, which falsely penalizes the Model's Predictions and makes the Model more conservative when predicting Class A.

      Update the Label Book to provide more details to help users label objects consistently.


      • Summary: Class A has a lot of False Positives
      • Visualization: The Model predicts that the image has Class A but the image doesn't.
      		There are many False Positives of Class A.There are too many NG examples, so the dataset induces bias in the Model.Upload more counter-examples or apply Class weights to the loss.
      The segmentation output has small patches/pixels of "noise".Apply the post-processing filter noise in the segmentation output.
      The Confidence Threshold is too low.Increase the Confidence Threshold.
      The environmental noise causes OK regions to look like defects.Try removing the noise from the data collection process, or use data augmentation to simulate.
      The labeling convention does not differentiate between OK and NG regions, or the labelers accidentally drew extra labels.
      • Update the labeling convention and relabel the data.
      • If there are only a couple of cases, remove the example.
      • Summary: Images of low quality have False Negatives.
      • Visualization: The Model fails to detect Classes in low-quality images.
      		The images are blurry.The image quality is too low to detect Classes.Remove the blurry examples and improve the image acquisition system. If not possible, add a random blur augmentation to make the Model more robust to blurriness.
      • Summary: Class A has False Positives.
      • Visualization: The Model predicts Class A in areas with noise.
      		There are False Positives on noise or artifacts on images.The noise/artifacts look similar to actual defects, so the  Model gets confused. The Train Set does not have sufficient samples with noise.Improve the image acquisition or use custom data augmentation to make the training data look more like noisy validation data.
      Filter the predictions in the area that is not the region of interest.


      Was this article helpful?
      What's Next