You’ve built a great model. In fact, your model is so good that you want to let other people access its predictions. Often, you will want to make your predictions available to another program (e.g. a dashboard or a web application). The de-facto standard for programs to communicate with each other is through REST APIs.

For instance, let’s pretend you’ve built a model that determines whether someone is a cat or a dog person depending on their address. Your organisation wants to automatically classify customers who sign up to a mailing list so that they can be sent animal-appropriate advertising material. By wrapping your model in an API, you expose a fixed URI that can be queried by a software engineer to determine whether the user is a cat or a dog person. Typically, as soon as a user signs up, application supporting your website will query your model by sending an HTTP request with a JSON payload like:

  "firstName": "Enrico",
  "lastName": "Fermi",
  "address": "22 Clark Street",
  "city": "Chicago",
  "country": "United States"

Your API would read this payload and query your model to find out whether Enrico Fermi is more likely to be a cat or a dog person. It might then return:

  "probabilities": { "cat": 0.3, "dog": 0.7 },
  "mostLikely": "dog"

The application supporting your website can use this to send Fermi an email with pictures of chihuahuas.

Wrapping your model behind an APIs allows other programs in your organisation to use it.

APIs on SherlockML

Creating an API from the ground up involves a lot of boilerplate. You need to:

  • set up an authentication and authorisation system to let the right people access your API.
  • set up DNS records so that the Internet knows how to find your API.
  • set up TLS so that your API can be accessed over https.

SherlockML aims to remove this boilerplate so you can concentrate on the work that adds value. It generates API keys for you and hides your API behind a program that filters out unauthenticated requests. You can then distribute those keys to the people who need to access your API. It also gives you a stable URI that clients of your API can target to get their predictions. The target URI is only accessible over https, so communications to your API are always encrypted.

Deploy an API

You have written this complex classifier that you now want to deploy behind an API:

class CatVsDogPersonClassifier(object):
    def predict(self, first_name, last_name):
        """Determine whether someone is a cat or a dog person."""
        if last_name in {'Schrodinger', 'Schrödinger', 'Schroedinger'}:
            return 'cat'
            return 'dog'

Let’s wrap this in a Flask application. Flask is a popular lightweight Python library for writing APIs.

Save the following in a file called app.py in the root of your project:

# coding: utf-8

import logging

from flask import Flask, jsonify, request, abort, make_response


app = Flask(__name__)

PORT = 8000

class CatVsDogPersonClassifier(object):
    def predict(self, first_name, last_name):
        """Determine whether someone is a cat or a dog person."""
        if last_name in {'Schrodinger', 'Schrödinger', 'Schroedinger'}:
            return 'cat'
            return 'dog'

classifier = CatVsDogPersonClassifier()

def _parse_request_body():
    """Extract first and last name from request."""
    request_body = request.get_json()
        first_name = request_body['firstName']
        last_name = request_body['lastName']
    except KeyError:
        # Missing first or last name fields.
        logging.info('Badly formatted request body: {}'.format(request_body))
        response_body = {
            'error': "Missing 'firstName' or 'lastName' key in request body"
        response = make_response(jsonify(response_body), 400)
    return first_name, last_name

@app.route('/predict', methods=['POST'])
def predict():
    """Respond to requests for a prediction."""
    first_name, last_name = _parse_request_body()
    person_type = classifier.predict(first_name, last_name)
    logging.info('Predicted that {} {} is a {} person'.format(
        first_name, last_name, person_type))
    return jsonify({"mostLikely": person_type})

if __name__ == '__main__':
    logging.info('Listening on port {}'.format(PORT))
    app.run(debug=True, host='', port=PORT)

We can now create a shell script that will run our API. Save the following to a file called run.sh in the same directory as app.py):


exec python -u app.py

We can now create our API. Go to the APIs page for your project in SherlockML and create a new API. You will be asked to choose both a name and a domain name for your API. Your domain name needs to be unique across all of SherlockML. I suggest a domain name like cats-vs-dogs-2118, where you replace 2118 with a random string of your choice. In the API settings page, change the working directory to /project and the script to run.sh. You can leave the environments dropdown empty.


We can now create a server running our API. Go to the development tab and create a new server. This server will automatically start serving your API.


Test your new API

We now want to test that our API works. This is often done with a REST client. Postman is a popular graphical client. You can also use cURL or httpie from the command line.

There are two important pieces of information that you need for accessing the API running on your newly created API server:

  • the URI is the unique address of your server. It will look like https://cube-64e0d7d8-1a3d-4fec-82da-fa7afea9138e.api.sherlockml.io.
  • the API key for your server. You will need to provide the key in every request to the API.

You can get the URI and the API key for your development server from the Development tab:


Paste these into Postman (or another REST client). The API key needs to be passed as value to the SherlockML-UserAPI-Key header.


Send a POST request to your API with body:

  "firstName": "Enrico",
  "lastName": "Fermi",
  "address": "22 Clark Street",
  "city": "Chicago",
  "country": "United States"

Your API will reply with the correct payload:

  "mostLikely": "dog"

Congratulations! You now have a working API that can be accessed from anywhere in the world. If you want other people to access your API, just share the URL and the API key with them and they will be able to reach it too.

You can also query your API from Python. The requests library has become standard for interacting with APIs:

import requests

API_URL = 'https://cube-64e0d7d8-1a3d-4fec-82da-fa7afea9138e.api.sherlockml.io'
API_KEY = 'i0cgsdYL3hpeOGkoGmA2TxzJ8LbbU1HpbkZo8B3kFG2bRKjx3V'

headers = {'SherlockML-UserAPI-Key': API_KEY}

body = {
    'firstName': 'Enrico',
    'lastName': 'Fermi',
    'address': '22 Clark Street',
    'city': 'Chicago',
    'country': 'United States'
response = requests.post('{}/predict'.format(API_URL), headers=headers,

response.raise_for_status()  # Raise an exception if return code is not 200
print(response.json())  # Extract response body as JSON

Developing your API

To modify your API, you just need to change the code. Since we started Flask in debug mode, the API will automatically watch the directory in which it is running and reload the code when it detects changes.

Sometimes, you will want to actually restart the API. To do this, you need to SSH into the API server. You can do this either from a terminal running locally or from the Jupyter terminal of a server in the same project:

$ sml shell <project-name> 'Development Server'

You will need to replace <project-name> with the name of the right project. If the project name contains whitespace, you will need to quote it. Once you are in the API server, run:

$ sudo sv restart sherlockml-api

This will restart the API running on that server.


When things go wrong

To debug your API, you will want to see any print statements or log statements printed by your API. These are stored in the development server in the file /var/log/sherlockml-api.log. To access these, SSH into the server (see the previous section) and enter the command:

$ tail -f /var/log/sherlockml-api.log

Lifting the hood

SherlockML aims to remove a lot of the boilerplate around API creation. If you use APIs a lot, you will want to know exactly what that boilerplate is.

When you create an API, SherlockML creates a reverse proxy server. Any request to your API will first go through the reverse proxy. The proxy looks for the Sherlockml-UserAPI-Key header and validates its value (against an internal API). Assuming the key is valid, it redirects the request to port 8000 on the server running the API. The proxy also does TLS termination.


You can find additional examples demonstrating additional capabilities of APIs here: