Use Case: Model Fails to Detect Classes
  • 06 Dec 2022
  • 2 Minutes to read
  • Dark
    Light
  • PDF

Use Case: Model Fails to Detect Classes

  • Dark
    Light
  • PDF

Article summary

Overview

There may be times when a Model fails to locate a specific Class. This article will help you:

  • Identify the signal and error
  • Diagnose the root cause
  • Implement the fix

Identify the Signal and Error

Models that fail to detect Classes can have lots of False Negatives. Therefore, reviewing the images the Model is detecting incorrectly is good practice to understand the error better.

Example

Take a look at the example image below. There are two labeled instances of creases on the Ground Truth. However, the Model did not detect those creases; therefore, there is a False Negative.

False Negative on the Crease

Error Signs

If the Model fails to detect Classes, you may notice these signs:

  • Lots of False Negatives on a specific Class
  • The Model fails to locate a particular Class 
  • The Model does not predict anything in an area that should be identified

Diagnose the Root Cause

For this use case of failing to detect Classes, there are five potential causes:

  1. The object to identify is too small or not visible after resizing.
  2. The data augmentation is too strong, and the object to identify is no longer visible.
  3. There are not enough examples of this instance in the Train Set.
  4. The Confidence Threshold is too high.
  5. The labeling is inconsistent. 

Take a look at the example image below. Say this image is part of the Train Set. Do you notice any issues? You are correct if you spotted that the creases next to the arrows were not labeled. Having inconsistent and incorrect labels can affect the performance of your Model.

Creases Are Not Consistently Labeled 

Fix

Now that you've learned how to identify the signal and diagnose the cause, you can iterate your data to achieve higher accuracy by following these tips:

Root CauseSolution
Object is too small or not visible after resizingIncrease the image size or improve the image resolution
Data augmentation is too strong, and the object to identify is no longer visible
Reduce the strength of data augmentation
Not enough examples of the instance in the Train SetAdd more examples 
Confidence Threshold is too highDecrease the Confidence Threshold
Labeling is inconsistent

Add details to the Label Book to help eliminate confusion and gain consistent labeling


Was this article helpful?