Use Case: Model Confuses Detections from Class A and Class B

Overview

There may be times when a Model confuses detections from one Class with another. This article will help you:

• Identify the signal and error
• Diagnose the root cause
• Implement the fix

Identify the Signal and Error

To identify the root cause of poor Model performance, try to identify a signal in your error analysis report. Are there a lot of False Negatives? False Positives? These metrics can indicate to you that there is a problem in your dataset that is causing poor Model performance. 

Models that mistakingly detect Classes can have many False Positives and False Negatives. Therefore, reviewing the images the Model is detecting incorrectly is good practice to understand the error better. 

Example

Refer to the two example images below.

The Model Locates the Objects But Misclassifies Them

The Model Locates the Object But Misclassifies It

Compare the Ground Truth and Predictions of the example images. You can see that the Model has correctly identified the objects but makes the wrong classification. This indicates that your Model is confusing water spots with oil spots.

Error Signs

If the Model is misclassifying objects, you may notice these signs:

•  Lots of False Negatives and False Positives
• The Model is detecting the right areas but choosing the wrong Class

Diagnose the Root Cause

You can diagnose the root cause by parsing through your training data. For this use case of confusing one Class with another, there are two potential causes:

1. The samples of Class A and B are not distinguishable, like oil and water in the previous example.
2. There are mislabels between Class A and B in the Train Set.

If the root cause is that the examples of Class A and B are indistinguishable, it may help to consult with a subject matter expert to help define distinguishable visual features. Then update the Label Book accordingly.

Fix

Now that you've learned how to identify the signal and diagnose the cause, you can iterate your data to achieve higher accuracy by following these tips: