Video Annotation Best Practices with Annika Deurlington
In this post, we sit down with Annika Deurlington, Commercial Program Manager at CrowdAI. She shares best practices for video annotation and reveals hidden industry tricks.
You’ve finally stepped foot into AI and are interested in computer vision. You might not have any imagery yet but you plan to collect some going forward. How should you go about it?
The very first step in image collection is defining your problem. You need to ask yourself why you are collecting imagery and what problem you’re trying to improve or solve. After all, computer vision is a tool to solve a problem.
These best practices will not fit every situation. However, below are a few general guidelines that apply to almost all use cases.
How is your model going to detect a broken lightbulb if you never show it what a broken lightbulb looks like? Think about the real-world variables the model may see and collect samples to match. Does the lighting change throughout the day? Do your target objects always appear at the same angle, or does the angle vary? Does the background change? Maybe your object of interest has variation in size or color. A dog detector will need to see lots of different dogs, or else you’ll only have a shiba detector.
The more images, the more training data to build your model. We recommend at least 100 example images for every object/feature you want to detect, but more is almost always better. The more training data, the more successful your model will be—and don’t forget that more variety means you need more examples.
It is better to collect fewer high quality images than thousands of poor examples. Remember: if a human can’t tell what’s in the photo, a machine won’t be able to, either. Inconsistent or poor examples will prevent your model from learning the right visual patterns to find your object of interest in new imagery. Try to collect imagery at a resolution high enough to see what you’re looking for, but not so high that you end up using far more storage space than you need to.
Again: make sure the thing that will help you solve your problem is visible in your images. Check 20–30 random examples to make sure the object or feature is visible and can be labeled consistently.
AI is just like a science experiment: you want to collect consistent data over a long period of time under a controlled environment. Knowing the parameters under which your data is collected can help you identify errors or mishaps that may happen.
For example, make sure your cameras are stationary or mounted when possible. If you use multiple cameras, try to have the lighting and specs consistent for each camera. If you’re collecting video, collect it at the same frame rate and shutter speed.
As much as possible, the only thing that should be changing is the images themselves, not the environment.
Your department or team might not collect imagery, but others in your business probably do. Don’t begin by creating an entirely new camera system—begin by scouring your organization because you might find that someone already has tons of imagery. Don’t start from scratch! Imagery can be expensive sometimes, so consider the cost early on.
Remember, you must define the problem you want to solve before you outline which image collection practices work best for you. Every case will be different. It’s not one size fits all!
Need help? CrowdAI experts can help you explore more imagery collection tools. Contact us today!