Getting started

Creating a Vertx account

To start using Vertx API one has to create an account at Vertx portal. Navigate to https://portal.vertx.ai and click "Create account" link.

Enter your email address, choose a password and provide a cellphone number for 2-fac authentication.

You'll receive a confirmation email with the following test:

Your Vertx account email verification code is *******

Confirm email address prividing the verification code:

The last step is confirm SMS verification code

Once the SMS code confirmation is successful, you'll get to the portal home page with the API key and accessible content buckets, both private and public.

alt text

For convenience, export the key as VERTX_API_KEY environmental variable:

export VERTX_API_KEY=<KEY>

In the following sections of this tutorial we'll assume that ${VERTX_API_KEY} environmental variable contains a valid Vertx API key.

At this point, one can start making API calls to search and register multimedia content.

Private content buckets

Each organization is granted access to a private content bucket. The initial list of private buckets contains a single bucket that can be used to register content.

For convenience, we'll export private bucket name as an environmental variable:

export BUCKET="rough-bush" # replace with your private bucket name

In the following sections we are going to register a sample multimedia file into assigned private bucket and send multiple search request.

Installing vertx command-line tool

Although most of the Vertx API is available through the Vertx REST endpoint, in order to be able to process large files, one has to use Vertx command-line tool, which can be downloaded from the releases section https://github.com/vertxai/vertx_cli/releases/latest of the corresponding GitHub repository. For example, to download and extract release 1.3.1 of vertx cli

# Download vertx command line tool
curl -L https://github.com/vertxai/vertx_cli/releases/download/v1.3.1/vertx_cli-1.3.1.tar.gz --output vertx_cli-1.3.1.tar.gz
tar -xzf vertx_cli-1.3.1.tar.gz && cd vertx_cli-1.3.1

Alternatively, vertx command-line tool can be invoked from a docker container vertxai/vertx_cli

docker pull vertxai/vertx_cli

Making API calls

Vertx API was designed with the simplicity in mind. It allows for managing and searching audio-visual content.

Searching in public buckets

Download a sample video file

curl -L https://github.com/vertxai/vertx_cli/releases/download/v1.3.1/sample.mp4 --output sample.mp4

Searching in public buckets is as simple as

# Vertx CLI
vertx search ./sample.mp4

# Docker
docker run --rm \
  -v $(pwd)/sample.mp4:/sample.mp4 \
  -e VERTX_API_KEY=${VERTX_API_KEY} \
  vertxai/vertx_cli /app/vertx search /sample.mp4

# cURL
curl -X POST "https://api.vertx.ai/v1/search" \
  -H "X-Session-Token: ${VERTX_API_KEY}" \
  -F "media_file=@sample.mp4"  

Regardless the way search API was called, the response json would contain the metadata for all the segments matched with the reference movies and music:

[
  {
    "duration": 24.9375,
    "error": null,
    "matches": [
      {
        "metadata": {
          "album": null,
          "artist": null,
          "bucket": "movies",
          "cover_url": "/assets/images/movies/you_got_served_2004/large-cover.jpg",
          "duration": 5674.875,
          "extra": null,
          "imdb_id": 365957,
          "title": "You Got Served",
          "uid": "1566530810344932800",
          "year": 2004
        },
        "segments": [
          {
            "duration": 24.875,
            "que_offset": 0.0,
            "ref_offset": 3500.6875
          }
        ]
      },
      {
        "metadata": {
          "album": "Elephunk",
          "artist": "Black Eyed Peas; Papa Roach",
          "bucket": "music",
          "cover_url": null,
          "duration": 218.5,
          "extra": null,
          "imdb_id": null,
          "title": "Anxiety",
          "uid": "6475547275973858650",
          "year": null
        },
        "segments": [
          {
            "duration": 24.875,
            "que_offset": 0.0,
            "ref_offset": 18.125
          }
        ]
      }
    ],
    "media_type": "audio",
    "request_id": "1c984237-2b1a-4c0d-a54c-e299d99a616b",
    "source_path": "./sample.mp4",
    "source_uid": "1758730481226206085",
    "status": "OK"
  },
  {
    "duration": 24.9375,
    "error": null,
    "matches": [
      {
        "metadata": {
          "album": null,
          "artist": null,
          "bucket": "movies",
          "cover_url": "/assets/images/movies/Troy_2004/large-cover.jpg",
          "duration": 11148.0,
          "extra": null,
          "imdb_id": 332452,
          "title": "Troy",
          "uid": "1342729426672482861",
          "year": 2004
        },
        "segments": [
          {
            "duration": 21.625,
            "que_offset": 0.0,
            "ref_offset": 8608.5625
          }
        ]
      }
    ],
    "media_type": "video",
    "request_id": "1c984237-2b1a-4c0d-a54c-e299d99a616b",
    "source_path": "./sample.mp4",
    "source_uid": "1758730481226206085",
    "status": "OK"
  }
]

Please, find detailed description of the json response fields in the API section.

By default, the content is searched in all public buckets. The list of public buckets consists of music,movies.
Search can be restricted to a specific set of private and public buckets by specifying form data field for REST API call -F 'buckets="movies"' and --search_buckets command-line flag for the Vertx cli and docker. The following calls demonstrate how to search in movies search bucket:

# Vertx CLI
vertx search ./sample.mp4 --search_buckets="movies" 

# Docker
docker run --rm \
  -v $(pwd)/sample.mp4:/sample.mp4 \
  -e VERTX_API_KEY=${VERTX_API_KEY} \
  vertxai/vertx_cli /app/vertx search /sample.mp4 --search_buckets="movies"

# cURL
curl -X POST "https://api.vertx.ai/v1/search" \
  -H "X-Session-Token: ${VERTX_API_KEY}" \
  -F 'buckets="movies"' \
  -F "media_file=@sample.mp4"  

Managing content

There is a set of operations that allow for listing, registering and deleting multimedia assets.

List registered content

The following GET call lists all the registered content:

curl -X GET "https://api.vertx.ai/v1/media/${BUCKET}" \
  --header "X-Session-Token: ${VERTX_API_KEY}"

JSON response will contain an empty list, which is expected since no multimedia assets have been registered.

{"data":[],"links":{"next":null}}

Searching in private bucket leads to an empty set of matches, which is also expected:

vertx search ./sample.mp4 --search_buckets=${BUCKET}

Content registration and deletion

Let's add signatures to the bucket and attach metadata describing the multimedia asset:

# vertx CLI
vertx register \
    --bucket "${BUCKET}" \
    --meta_title='sample_title' \
    --meta_album 'sample_album'  \
    --meta_artist 'sample_artist' \
    --meta_cover_url '/images/cover.jpg' \
    --meta_imdb_id 4423 \
    --meta_year 2004 \
    --meta_extra 'extra_meta' \
    sample.mp4

# Docker
docker run --rm \
  -v $(pwd)/sample.mp4:/sample.mp4 \
  -e VERTX_API_KEY=${VERTX_API_KEY} \
  vertxai/vertx_cli /app/vertx register \
    --bucket "${BUCKET}" \
    --meta_title='sample_title' \
    --meta_album 'sample_album'  \
    --meta_artist 'sample_artist' \
    --meta_cover_url '/images/cover.jpg' \
    --meta_imdb_id 4423 \
    --meta_year 2004 \
    --meta_extra 'extra_meta' \
    /sample.mp4

# cURL
curl -X POST "https://api.vertx.ai/v1/media/${BUCKET}" \
    --header "X-Session-Token: ${VERTX_API_KEY}" \
    -F media_file="@sample.mp4" \
    -F metadata='{"album": "sample_album", "artist": "sample_artist", "cover_url": "/images/cover.jpg", "extra": "extra_meta", "imdb_id": 4423, "title": "sample_title", "year": 2004}'

If the insert was successful, the following matches should show up when searching in the private bucket:

vertx search ./sample.mp4 --search_buckets=${BUCKET}

[
  {
    "duration": 24.9375,
    "error": null,
    "matches": [
      {
        "metadata": {
          "album": "sample_album",
          "artist": "sample_artist",
          "bucket": "rough-bush",
          "cover_url": "/images/cover.jpg",
          "duration": 24.9375,
          "extra": "extra_meta",
          "imdb_id": 4423,
          "title": "sample_title",
          "uid": "1758730481226206085",
          "year": 2004
        },
        "segments": [
          {
            "duration": 24.875,
            "que_offset": 0.0,
            "ref_offset": 0.0
          }
        ]
      }
    ],
    "media_type": "audio",
    "request_id": "fa483dff-738c-4848-a810-0821a7e76109",
    "source_path": "./sample.mp4",
    "source_uid": "1758730481226206085",
    "status": "OK"
  },
  {
    "duration": 24.9375,
    "error": null,
    "matches": [
      {
        "metadata": {
          "album": "sample_album",
          "artist": "sample_artist",
          "bucket": "rough-bush",
          "cover_url": "/images/cover.jpg",
          "duration": 24.9375,
          "extra": "extra_meta",
          "imdb_id": 4423,
          "title": "sample_title",
          "uid": "1758730481226206085",
          "year": 2004
        },
        "segments": [
          {
            "duration": 24.875,
            "que_offset": 0.0,
            "ref_offset": 0.0
          }
        ]
      }
    ],
    "media_type": "video",
    "request_id": "fa483dff-738c-4848-a810-0821a7e76109",
    "source_path": "./sample.mp4",
    "source_uid": "1758730481226206085",
    "status": "OK"
  }
]

The inserted asset now shows up in the list of multimedia assets when listing the bucket:

curl -X GET "https://api.vertx.ai/v1/media/${BUCKET}" \
  --header "X-Session-Token: ${VERTX_API_KEY}"

JSON response will contain a list containing a single multimedia asset we've just registered:

{
  "data": [
    {
      "uid": "1758730481226206085",
      "bucket": "rough-bush",
      "title": "sample_title",
      "year": 2004,
      "imdb_id": 4423,
      "album": "sample_album",
      "artist": "sample_artist",
      "cover_url": "/images/cover.jpg",
      "label": null,
      "extra": "extra_meta"
    }
  ],
  "links": {
    "next": "/v1/media/rough-bush?page[size]=50&page[after]=EV5dvsBTFGPS_JugHblbg9Yr3KnhkwaxpjvrnVLnMAe-5iUJsl_Gz2g30oXFXBk="
  }
}

Signatures for assets that have been registered in the private buckets can be deleted in the following way:

curl -X DELETE "https://api.vertx.ai/v1/media/${BUCKET}/1758730481226206085" \
  --header "X-Session-Token: ${VERTX_API_KEY}"

Now the bucket is empty again:

curl -X GET "https://api.vertx.ai/v1/media/${BUCKET}" \
  --header "X-Session-Token: ${VERTX_API_KEY}"

{"data":[],"links":{"next":null}}

Searching deleted content in private buckets leads to empty results.

Working with live content

In certain cases, there is the need to register and/or search content in real time. Let's create a video stream providing sample.mp4 file as the source of the broadcast content:

#!/bin/bash
vertx register \
    --streaming \
    --bucket=${BUCKET} \
    --meta_title='streaming_title' \
    --meta_album 'streaming_album'  \
    --meta_artist 'streaming_artist' \
    --meta_cover_url '/images/streaming.jpg' \
    --meta_imdb_id 43 \
    --meta_year 2010 \
    --meta_extra 'streaming_meta' \
    udp://localhost:5000/1758730481226206085 &
sleep 2
ffmpeg -re -i ./sample.mp4 -acodec copy -vcodec copy  -f flv udp://localhost:5000/1758730481226206085 -hide_banner -loglevel panic

The script starts Vertx client in streaming mode processing stream from URI udp://localhost:5000/1758730481226206085. After a short pause of 2 seconds, we start sending the contents of sample.mp4 to the stream URI. Every 2 seconds client reports successful or failed attempt to insert a chunk of data to Vertx indices:

{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"video","offset":0.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"audio","offset":0.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"video","offset":2.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"audio","offset":2.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"video","offset":4.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"audio","offset":4.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"video","offset":6.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"audio","offset":6.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"video","offset":8.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"audio","offset":8.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"video","offset":10.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"audio","offset":10.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"video","offset":12.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"audio","offset":12.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"video","offset":14.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"audio","offset":14.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"video","offset":16.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"audio","offset":16.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"video","offset":18.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"audio","offset":18.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"video","offset":20.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"audio","offset":20.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"video","offset":22.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":2.0,"error":null,"media_type":"audio","offset":22.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":0.9375,"error":null,"media_type":"audio","offset":24.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}
{"bucket":"rough-bush","duration":0.9375,"error":null,"media_type":"video","offset":24.0,"request_id":"5ea7a07d-4095-4098-9d32-110c03e961b1","source_uid":"5554329625350814294","status":"OK"}

To search live content in real time mode, one has to specify --streaming command-line flag.

vertx search \
    --streaming \
    --search_buckets=${BUCKET} \
    udp://localhost:5000/1758730481226206085 &
sleep 2
ffmpeg -re -i ./sample.mp4 -acodec copy -vcodec copy -f flv udp://localhost:5000/1758730481226206085 -hide_banner -loglevel panic

When processing data in streaming mode, vertx client prints intermediate results to the standard output pipe every five seconds:

{"duration":5.0625,"error":null,"matches":[{"metadata":{"album":"streaming_album","artist":"streaming_artist","bucket":"rough-bush","cover_url":"/images/streaming.jpg","duration":24.9375,"extra":"streaming_meta","imdb_id":43,"title":"streaming_title","uid":"5554329625350814294","year":2010},"segments":[{"duration":5.0,"que_offset":0.0,"ref_offset":0.0}]},{"metadata":{"album":"sample_album","artist":"sample_artist","bucket":"rough-bush","cover_url":"/images/cover.jpg","duration":24.9375,"extra":"extra_meta","imdb_id":4423,"title":"sample_title","uid":"1758730481226206085","year":2004},"segments":[{"duration":5.0,"que_offset":0.0,"ref_offset":0.0}]}],"media_type":"video","request_id":"d21c72f2-22eb-4604-b3cd-26356a5761ca","source_path":"udp://localhost:5000/1758730481226206085","source_uid":"5554329625350814294","status":"OK"}
{"duration":5.875,"error":null,"matches":[{"metadata":{"album":"sample_album","artist":"sample_artist","bucket":"rough-bush","cover_url":"/images/cover.jpg","duration":24.9375,"extra":"extra_meta","imdb_id":4423,"title":"sample_title","uid":"1758730481226206085","year":2004},"segments":[{"duration":5.8125,"que_offset":0.0,"ref_offset":0.0}]},{"metadata":{"album":"streaming_album","artist":"streaming_artist","bucket":"rough-bush","cover_url":"/images/streaming.jpg","duration":24.9375,"extra":"streaming_meta","imdb_id":43,"title":"streaming_title","uid":"5554329625350814294","year":2010},"segments":[{"duration":5.8125,"que_offset":0.0,"ref_offset":0.0}]}],"media_type":"audio","request_id":"d21c72f2-22eb-4604-b3cd-26356a5761ca","source_path":"udp://localhost:5000/1758730481226206085","source_uid":"5554329625350814294","status":"OK"}
...

Once the stream is finished, aggregate results will be printed to the standard output for both, audio and video matches.

Creating a Vertx account​

Private content buckets​

Installing vertx command-line tool​

Making API calls​

Searching in public buckets​

Managing content​

List registered content​

Content registration and deletion​

Working with live content​