Object Descriptions

Introduction

Object descriptions are the English text that you use to tell EyesOnIt what to detect. To provide accurate results, EyesOnIt requires at least two descriptions. Typically, these two descriptions describe that object you want to detect, and the background scene without the object. Multiple descriptions can also be used to distinguish objects of the same category.

In our warehouse driveway example, we’d like to know when a vehicle enters the driveway. In this case, the object description might be something like “vehicle”. The background scene could be “driveway”. The image below shows the results of descriptions with a vehicle in the driveway.

In this image, we see that EyesOnIt is 97% confident that there is a vehicle in the image. Note, we’ll dive into confidence levels in the next section.

Multiple Object Descriptions

What if we wanted to determine what type of vehicle is in the driveway? As an example, assume that the GMC Yukon pictured above is my vehicle. I don’t want an alert when I drive into the driveway, but I do want an alert if anyone else drives into the driveway. To achieve this, I could add another object description for my GMC Yukon like this:

Based on these object descriptions, EyesOnIt determines that “GMC Yukon” is a better object description for the detection region than “vehicle”. If another type of vehicle drives into the driveway, EyesOnIt determines that “GMC Yukon” is not a good match for that vehicle. EyesOnIt assigns a higher confidence to the more general “vehicle” description. As we’ll see in the section on alerts, we can use these object descriptions with their associated confidence levels to create alerts that get triggered for some vehicles but not others.

So far, we’ve described object descriptions. In the images above, you see that you can add multiple object descriptions. You add a new object description by clicking the green + icon. You can also remove an object description by clicking the garbage can next that object description.

Background Description

As we mentioned above, the “Background Description” describes the background in the image. EyesOnIt needs to know what’s in the background, because that background is an important part of the image. Let’s go back to the first image at the top of this page where we used “vehicle” as the object description and “driveway” as the background description.

It’s true that the image includes a vehicle, but it also includes some of a driveway. The most accurate description for the image includes both the words “vehicle” and “driveway”. When you include a background description, EyesOnIt adds that background description to each of your object descriptions. In the first example above, EyesOnIt is looking at the image and deciding whether “vehicle driveway” or “driveway” is the best description for the image. It decides with 97% confidence that “vehicle driveway” is the better description.

Note that the background description is optional. In some cases, you will get better results by using multiple object descriptions without a background description. This is especially the case if the background of your image is very busy or inconsistent throughout the image. If you are processing video where the background changes often, you may also get better results without using a background description. The background description can be helpful in circumstances where the background is consistent, but try removing it to see if you can get better results without it.

Conclusion

We have used vehicles in this section as an example because many people are familiar with vehicles. It’s important to understand that EyesOnIt has been trained using billions of images from across the Internet. As a result, EyesOnIt is good at detecting thousands of different types of objects, and you can tell EyesOnIt what you want to detect simply by describing those objects using your English descriptions. Use some of your own images and experiment with things like people, animals, structures, construction materials, weather conditions, poses, industrial equipment, or anything else you can think of. You’ll be amazed at what EyesOnIt can detect using object descriptions.