- 21 Dec 2022
- 2 Minutes to read
Use Case: Model Confuses Detections from Class A and Class B
- Updated on 21 Dec 2022
- 2 Minutes to read
There may be times when a Model confuses detections from one Class with another. This article will help you:
- Identify the signal and error
- Diagnose the root cause
- Implement the fix
Identify the Signal and Error
To identify the root cause of poor Model performance, try to identify a signal in your error analysis report. Are there a lot of False Negatives? False Positives? These metrics can indicate to you that there is a problem in your dataset that is causing poor Model performance.
Models that mistakingly detect Classes can have many False Positives and False Negatives. Therefore, reviewing the images the Model is detecting incorrectly is good practice to understand the error better.
Refer to the two example images below.
Compare the Ground Truth and Predictions of the example images. You can see that the Model has correctly identified the objects but makes the wrong classification. This indicates that your Model is confusing water spots with oil spots.
If the Model is misclassifying objects, you may notice these signs:
- Lots of False Negatives and False Positives
- The Model is detecting the right areas but choosing the wrong Class
Diagnose the Root Cause
You can diagnose the root cause by parsing through your training data. For this use case of confusing one Class with another, there are two potential causes:
- The samples of Class A and B are not distinguishable, like oil and water in the previous example.
- There are mislabels between Class A and B in the Train Set.
If the root cause is that the examples of Class A and B are indistinguishable, it may help to consult with a subject matter expert to help define distinguishable visual features. Then update the Label Book accordingly.
Now that you've learned how to identify the signal and diagnose the cause, you can iterate your data to achieve higher accuracy by following these tips:
|Multiple Classes are visually indistinguishable.|
|There are mislabels in the training set.||Check your labels in the training set and correct any mislabels.|