Data Quality is Important | Car Classifier

Classy Cars

Most of the time, data matters more than the model.

This article was updated on Thursday, 10 November 2022.

I recently created a car classifier that classified cars into their respective brands.

Despite having almost 5000 images in my training set, I ended up trying out over a hundred layers in my model, and twenty epochs. Even then, I had an error rate of 17.4%.

The culprit? My dataset.

I scraped 5000 images of cars (500 for each company) from DuckDuckGo. Naturally, as expected, the data quality is not so good.

Why? Below are some potential reasons:

• Noncar images present in dataset
• Cars of incorrect company present in dataset
• F1 cars present in dataset
• A large variety of cars from different time periods present in dataset
• Different companys’ cars look similar
• Modded cars present in dataset
• Concept cars present in dataset
• Multiple cars present in a single image
• Certain angles of cars appear more than others
• Cars appear in certain backgrounds more than others
• The search term {car_brand} car could be skewing results

I could have absolutely achieved better results with fewer layers and fewer epochs if I trained the model on better quality data — or manually combed through the 5000 images 💀. However, I did use fastai’s GUI for data cleaning. This GUI sorts images by their loss which helps to determine if certain images should be relabeled or deleted.

Below is the confusion matrix for this model.

It can be seen that this model “confuses” between quite a few different brands: Ford and Chevrolet, Chevrolet and Ford, Jaguar and Aston Martin, Renault and Ford.

But why is data quality important? Because without good data, the model will not be able to “see” things the way they actually are, and in turn end up making worse predictions and not generalize to other data.

Let’s say you did not know how, say, a toaster looked like. So I taught you by showing you pictures of a kettle. Then to test you, I showed you a set of pictures depicting various kitchen appliances and told you to find the toaster. You would not be able to.

Extending upon this example, say I showed you toasters only from the last two years and from two brands only. You would not be able to identify toasters older than two years, and toasters from other brands to much success.

Obviously, humans are smarter and can infer. AI methods can only infer to a certain degree, mainly based on what is in their dataset. This talk does start to become more philosophical.

The point of this post is to emphasize the importance of data quality and different aspects to consider as to why data quality may not be good. You can have the best architecture in the world, but it is useless if you do not have good data.

If you have any comments, questions, suggestions, feedback, criticisms, or corrections, please do post them down in the comment section below!