Back to vidrovr.com/careers

Getting started with custom detectors

Over the last few years you have probably heard about artificial intelligence (AI) and machine learning (ML). AI and ML can be hard to implement, because you need lots of labeled data for examples images with captions to allow these solutions to learn from the data. Without the proper annotated data, it's almost impossible to get meaningful value from these solution. Thankfully, Vidrovr has developed a framework for folks that want to find concepts in their video libraries, but don't have the labeled data to start training their own AI, we call this solution custom detectors. Before we dive in, I want to give you a little background on what exactly these detectors try to solve and how you can use them.

Effectively, custom detectors are concept classifiers. They learn what things like dogs, cats, rockets, galaxies look like and then apply them to our video processing engine to enable you to find anything and everything you are looking for.

The way you use it, is you type in what you want to find and Vidrovr just figures it out. Behind the scenes Vidrovr uses it's proprietary data and data collection algorithms to understand what your query means, and then uses our labeled data to train the custom detectors for you. Examples of the types of content that we'll collect can be seen below, whether you want to find a 'hot dog' or a 'dog', Vidrovr will use the information it knows about already and new information to find examples for you.

vs.

If you already have examples of your own that are labeled we can use those to help train your custom detectors, but it's not necessary.

All this being said learning any ML model is a statistics problem, so if our algorithms cannot find what you are looking for or if the things you want to find are too generic the detectors we'll build for you won't be great. Everytime a detector is trained you will receive a performance score for the category (class) you want. And you will be able to select the cateogies you would like to apply to your set of videos sitting in Vidrovr.

Currently, We limit our custom detectors to 100 categories per detector.

If you are having difficulties feel free to shoot us a message at support@vidrovr.com

Without further ado let’s dive in and learn a bit about Vidrovr’s Custom Detectors.

Overview

Let us first begin by creating our first detector. For this example let us create a hot dog vs anything else - let’s say a puppy.

vs.

Setup a Custom Detector

The method to begin this process in our API for this is called create_detector.

Once again to start this process, we don’t need any labeled images to tell our system what they are. Simply need to pass a POST request to our system with the following parameters:

Parameter
Description
Required
name
name of the detector that you wish to be assigned
yes
categories
a comma seperated list of the categories you would like to train a detector for
yes
api_key
The API-KEY requrired for your account
yes

Here is the request:

POST /public/api/v01/custom_detector/create_detector?api_key=<
API-KEY>&name=test1&
categories=hot%20dog,cat
HTTP/1.1
   Host: platform.vidrovr.com
   Connection: close
   User-Agent: Paw/3.1.8 (Macintosh; OS X/10.14.2) GCDHTTPRequest
   Content-Length: 0

And if your request is a success then you should see this in the response.

{
 "status": 200,
 "added": true,
 "id_asset": "08bbd7aea3ec4505b0b2cd2af5bbf1c5"
}

Nice Job! This means you have created your first Custom Detector.

While the custom detector is being created, the detector has not yet been trained and cannot be applied to videos.

If you really want your own images/videos of hot dogs (who can blame you) this is your chance to upload them. We have a method for that: upload_image_custom_detector

Note: If you dont want our system to try to make your model from our internal data stores, you need to provide at least 100 images/videos otherwise our system will augment your data.

Here is what you need to upload an image/video:

Parameter
Description
Required
name
name of the detector that you wish to be assigned
yes
id_asset
a comma seperated list of the categories you would like to train a detector for
yes
api_key
The API-KEY requrired for your account
yes
image
Test image (should be Multipart form data)
yes
keyword
Category associated with this image
yes

And here is an example request:

POST /api/v01/custom_detector/upload_image_custom_detector?api_key=<
API-KEY>&
id_asset=08bbd7aea3ec4505b0b2cd2af5bbf1c5&
keyword=hot%dog
HTTP/1.1
   Content-Type: multipart/form-data; charset=utf-8; boundary=__X_PAW_BOUNDARY__
   Host: platform.vidrovr.com
   Connection: close
   User-Agent: Paw/3.1.8 (Macintosh; OS X/10.14.1) GCDHTTPRequest
   Content-Length: 2432854

   --__X_PAW_BOUNDARY__
   Content-Disposition: form-data; name="image "; filename="magic_hot_dog.jpg"
   Content-Type: image/jpeg

and if it’s successful you should get:

{
 "status": 200,
 "added": true
}

That’s it! Now you got a detector that’s ready to go with your own data!

Training a Detector

So at this point you have a Custom Detector and maybe a bunch of uploaded images. Now all you need to do is start training the detector and sit back.

To do this you need to toggle the training state of the detector - either on or off. This is just a switch so if you are note sure about the state of your detector just send a request to our get_detector enpoint. The request is a POST

Parameter
Description
Required
api_key
The API-KEY requrired for your account
yes

POST /api/v01/custom_detector/toggle_training?api_key=<
API-KEY>&
id_asset=08bbd7aea3ec4505b0b2cd2af5bbf1c5
HTTP/1.1
   Host:  platform.vidrovr.com
   Connection: close
   User-Agent: Paw/3.1.8 (Macintosh; OS X/10.14.2) GCDHTTPRequest
   Content-Length: 0

with a response:

POST /api/v01/custom_detector/toggle_training?api_key=<
API-KEY>&
id_asset=08bbd7aea3ec4505b0b2cd2af5bbf1c5
HTTP/1.1
   Host:  platform.vidrovr.com
   Connection: close
   User-Agent: Paw/3.1.8 (Macintosh; OS X/10.14.2) GCDHTTPRequest
   Content-Length: 0

Note: When the training is complete you will be notified via an email to the account list email, so you don't need to keep pinging this endpoint all the time.

Applying a Trained Detector

Once you have a your detector trained. All you have to do is to apply it and every video you then upload to Vidrovr will have this custom detector run on it. We'll find all the hot dogs in your videos!

Here is how you apply a detector to any new video being passed to this account system - 1 method:
apply_custom_detector

You will have the option of applying this detector to new videos, old videos or both. These are the options that you can set inside of the apply_targets parameter in the POST request.

And all you will need to pass in are the following:

Parameter
Description
Required
model_id_asset
Identifier for the custom detector, returned by create_detector
yes
apply_targets
This is the parameter you would set to one of the three options of videos to apply to. ["new", "old", "both"]. Note: this parameter is required and will error if no option is passed
yes
api_key
The API-KEY requrired for your account
yes
black_list
This is the parameter you would use to black list classes. This should be a comma seperated string. “cat, dog”
yes
performance_threshold
Use this parameter to black list all the classes below a certain accuracy.
yes

Here is a request:

POST /api/v01/custom_detector/apply_custom_detector?api_key=<
API-KEY>&
model_id_asset=08bbd7aea3ec4505b0b2cd2af5bbf1c5&
apply_targets=both&
;black_list=cat,%20dog&
performance_threshold=0.6
HTTP/1.1
   Host: localhost:8000
   Connection: close
   User-Agent: Paw/3.1.8 (Macintosh; OS X/10.14.2) GCDHTTPRequest
   Content-Length: 0

THE END

That’s it.
From now on you will be detecting when hot dogs or cats appear on screen in a video you upload!

If you are still having difficulties feel free to shoot us a message at support@vidrovr.com