Use Case: Model Fails to Detect Classes

Overview

There may be times when a Model fails to locate a specific Class. This article will help you:

• Identify the signal and error
• Diagnose the root cause
• Implement the fix

Identify the Signal and Error

Models that fail to detect Classes can have lots of False Negatives. Therefore, reviewing the images the Model is detecting incorrectly is good practice to understand the error better.

Example

Take a look at the example image below. There are two labeled instances of creases on the Ground Truth. However, the Model did not detect those creases; therefore, there is a False Negative.

False Negative on the Crease

Error Signs

If the Model fails to detect Classes, you may notice these signs:

• Lots of False Negatives on a specific Class
• The Model fails to locate a particular Class 
• The Model does not predict anything in an area that should be identified

Diagnose the Root Cause

For this use case of failing to detect Classes, there are five potential causes:

1. The object to identify is too small or not visible after resizing.
2. The data augmentation is too strong, and the object to identify is no longer visible.
3. There are not enough examples of this instance in the Train Set.
4. The Confidence Threshold is too high.
5. The labeling is inconsistent. 

Take a look at the example image below. Say this image is part of the Train Set. Do you notice any issues? You are correct if you spotted that the creases next to the arrows were not labeled. Having inconsistent and incorrect labels can affect the performance of your Model.

Creases Are Not Consistently Labeled 

Fix

Now that you've learned how to identify the signal and diagnose the cause, you can iterate your data to achieve higher accuracy by following these tips: