NAV
cURL JavaScript

Authentication

Before you can start using the REST API, you must visit your profile page in the Waylay application and fetch your API keys. Make sure you have the correct permissions, for this API the byoml permission will be checked. If the scope of your permission is me, you will only be able to access the models you uploaded yourself. Everyone with the admin role will automatically have the permissions assigned to access everything.

Overview of the API

Once you have retrieved your keys, you can verify whether your keys and the REST server work by issuing the following command:

curl --user apiKey:apiSecret "https://byoml.waylay.io/models"

Waylay will assign you an endpoint URL to upload and run your models: e.g. byoml.waylay.io. The following operations are supported

Method Endpoint Description Required Permission
GET /models Get a list of the available models, visible to you read
POST /models Upload a zipped model with a unique name and a description create
POST /models/<name> Overwrite a model with a new zipped model update
GET /models/<name> Get information about the model read
DELETE /models/<name> Delete the model delete
GET /models/<name>/content Get the uploaded zipfile from the specified model read
GET /models/<name>/examples Get an example input for the predict endpoint of the model read
POST /models/<name>/predict Get predictions from the model read
POST /models/<name>/classify Get classification from the model (only for TensorFlow models) read
POST /models/<name>/regress Get regressions from the model (only for TensorFlow models) read

How to upload your model

When you upload a model, we expect the following form:

curl -u apiKey:apiSecret -X POST 'https://byoml.waylay.io/models' \
	-F file=@your/save/path/zipfile.zip \
	-F framework=ml-framework \
	-F name=my-model-name \
	-F description=optional-model-description

For example if the available endpoint is byoml.waylay.io, you can upload the model using curl. You should get a response containing: Model succesfully uploaded. Depending on the size of the model it might take a while till your model is available. The contents of the zipfile differ for each machine learning framework as explained below.

Some example notebooks on creating and uploading a model can be found here. Important to note is the version of the framework you are using. Make sure the version you are using is compatible with the version listed below:

Framework Version
scikit-learn 0.22.0
XGBoost 1.0.1
PyTorch 1.4.0
TensorFlow 2.1.0

scikit-learn

We expect the uploaded zip to contain a single file model.joblib that is a dump of an sklearn model, saved using the joblib library:

  
from joblib import dump
dump(model, './model.joblib')
  

The saved model object should have a predict method. This means you can also upload the result of a gridsearch or a Pipeline.

XGBoost

We expect the uploaded zip to contain a single file model.bst that is a dump of a Booster object, saved using the save_model method, which is supported by both the Booster class and the scikitlearn wrappers (XGBClassifier and XGBRegressor):

  
model.save_model('./model.bst')
  

PyTorch

We expect the uploaded zip to contain a single file model.pt which is a dump of a TorchScript object, saved using torch.jit.save. For example if your model is an torch.nn.Module object:

  
traced_model = torch.jit.trace(model, example_input)
torch.jit.save(traced_model, "model.pt")
  

Consult the PyTorch documentation for more information about saving a model using TorchScript or adding pre- and postprocessing to your model.

Tensorflow

We expect you to save your model in the SavedModel format:

tf.saved_model.save(your_model, "your/save/path/")

estimator.export_saved_model("your/save/path/", serving_input_fn)

After saving your model, you will have to make a zipfile of the saved model. Go to the directory containing the .pb file (this file specifies the graph and weights). Inside the directory you might also find directories named variables and/or assets. Zip the .pb file together with variables and assets directory if they exist. You can now upload the zip to Waylay using the REST API available to you.

Model Signature

When you save a model a default signature is generated. It is also possible to create your own signature(s) and specify the inputs and outputs. Some best practices are listed below.

Keras

If your Model is built using the Keras API and you do not specify the signature, it will use the names of the specified inputs. Therefore it is recommended to always name your inputs, as the user of the model will (in most cases) have to name them in his requests, an example input for you model might be:

  
{
  "instances":
    [
      {
        "previous_month_input": [...],
        "average_temperature": 5
      },
      {
        "previous_month_input": [...],
        "average_temperature": 17
      }
    ]
}

But if you do not name your inputs, the user will have to specify it as:

{"lstm_input_4: [...], "x": 5}

Avoiding this is easily achieved by naming all Input layers. For example:

  
historical_input = keras.Input(shape=(None,), name='previous_month')
temperature_input = keras.Input(shape=(None,), name='average_temperature')
  

Estimator

If we want to export this model in the SavedModel format, we will need to specify a serving input function. We can add the following code:

  
feature_spec = tf.feature_column.make_parse_example_spec(feature_columns)
serving_input_receiver_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(feature_spec)
linear_est.export_saved_model("your/save/path/", serving_input_receiver_fn)
  

This model can then be zipped and uploaded as described above, but the user of the model will not be able to know how to specify the inputs for this, as the auto-generated example will not be able to retrieve TensorInfo. You will have to document the expected inputs to the users of the model.

A working request for this model looks like this:

curl -u apiKey:apiSecret -X GET 'https://byoml.waylay.io/models/my-model-name/classify' -d '{\
    "examples": [\
	{\
	    "sex": "female",\
	    "n_siblings_spouses": 2,\
	    "parch": 0,\
	    "class": "First",\
	    "deck": "F",\
	    "embark_town": "Queenstown",\
	    "alone": "y",\
	    "age": 14.0,\
	    "fare": 20.0\
	}\
    ]\
}'

If we want the user of the model to be able to use the predict API and request example input, we can use the following method to save the model:

  
def serving_input_receiver_fn():
    receiver_tensors = {
        'sex': tf.compat.v1.placeholder(tf.string, [None]),
        'n_siblings_spouses': tf.compat.v1.placeholder(tf.int64, [None]),
        'parch': tf.compat.v1.placeholder(tf.int64, [None]),
        'class': tf.compat.v1.placeholder(tf.string, [None]),
        'deck': tf.compat.v1.placeholder(tf.string, [None]),
        'embark_town': tf.compat.v1.placeholder(tf.string, [None]),
        'alone': tf.compat.v1.placeholder(tf.string, [None]),
        'age': tf.compat.v1.placeholder(tf.float32, [None]),
        'fare': tf.compat.v1.placeholder(tf.float32, [None]),
    }
    return tf.estimator.export.ServingInputReceiver(receiver_tensors=receiver_tensors, features=receiver_tensors)
export_path = linear_est.export_saved_model(
  "without_features", serving_input_receiver_fn)
  

Or when running TensorFlow 1.x:

  
def serving_input_receiver_fn():
    receiver_tensors = {
        'sex': tf.placeholder(tf.string, [None]),
        'n_siblings_spouses': tf.placeholder(tf.int64, [None]),
        'parch': tf.placeholder(tf.int64, [None]),
        'class': tf.placeholder(tf.string, [None]),
        'deck': tf.placeholder(tf.string, [None]),
        'embark_town': tf.placeholder(tf.string, [None]),
        'alone': tf.placeholder(tf.string, [None]),
        'age': tf.placeholder(tf.float32, [None]),
        'fare': tf.placeholder(tf.float32, [None]),
    }
    return tf.estimator.export.ServingInputReceiver(receiver_tensors=receiver_tensors, features=receiver_tensors)
export_path = linear_est.export_saved_model(
  "without_features", serving_input_receiver_fn)
  

As the predict API supports generating example input, having your model support this endpoint is definitely a plus.

Checking out your model

curl -u apiKey:apiSecret -X GET 'https://byoml.waylay.io/models/my-model-name'

When you have uploaded multiple models you might want to check the description of a model. This gives the information you specified when uploading your model.

It is also possible to download the model you uploaded, using the content endpoint.

curl -u apiKey:apiSecret -X GET 'https://byoml.waylay.io/models/my-model-name/content' --output model.zip

Overwriting a model

When your model needs to be updated because you trained it on more data or it was trained on some bad inputs, you can overwrite it.

curl -u apiKey:apiSecret -X POST 'https://byoml.waylay.io/models/my-model-name' \
	-F file=@your/save/path/zipfile.zip \
	-F framework=ml-framework \
	-F description=optional-model-description

Example input

If you uploaded a TensorFlow Model supporting the predict endpoint, it is possible to generate example input for the model. This will use the signature specified by the data scientist to generate an instances list with one example input. The default for most models is the predict endpoint.

curl -i apiKey:apiSecret -X GET https://byoml.waylay.io/models/model-name/examples

A working call to the predict endpoint:

curl -u apiKey:apiSecret -X GET 'https://byoml.waylay.io/models/my-model-name/classify' -d '{\
           "instances": [\
                {\
                    "body": [\
                        0.5,\
                        0.5,\
                        0.5,\
                        0.5\
                    ],\
                    "tags": [\
                        0.5,\
                        0.5,\
                        0.5,\
                        0.5,\
                        0.5,\
                        0.5,\
                        0.5,\
                        0.5,\
                        0.5,\
                        0.5,\
                        0.5,\
                        0.5\
                    ],\
                    "title": [\
                        0.5,\
                        0.5,\
                        0.5,\
                        0.5\
                    ]\
                }\
            ],\
            "signature_name": "serving_default"\

}'

For example if we use this model, we get the following output when requesting an example input:

  
[
    {
        "endpoint": "predict",
        "payload": {
            "instances": [
                {
                    "body": [
                        0.5,
                        0.5,
                        0.5,
                        0.5
                    ],
                    "tags": [
                        0.5,
                        0.5,
                        0.5,
                        0.5,
                        0.5,
                        0.5,
                        0.5,
                        0.5,
                        0.5,
                        0.5,
                        0.5,
                        0.5
                    ],
                    "title": [
                        0.5,
                        0.5,
                        0.5,
                        0.5
                    ]
                }
            ],
            "signature_name": "serving_default"
        }
    }
]

The response is a list of example inputs for each known signature (in this case 1 signature). A data scientist can specify more than one possible input/output combination which might require other inputs (even for the same endpoint). In this case we only have the serving_default signature for the predict endpoint, meaning that the data scientist only provided a signature for one input/output combination. The expected payload to retrieve predictions with, is automatically generated based on this signature. The original model expected 3 features as we can see here. Both the length of the body and the title is not specified so this can be anything between an empty list and ten of thousands of floats. The model always expects tags to be a list of 12 floats. We see that the generated example satisfies these requirements.

Predictions

Depending on the model, one or more of the following endpoints will be available:

In most cases the predict endpoint will available as this is the default for many cases. For more information about the expected payloads, consult the TensorFlow documentation as we expect the same JSON objects in the requests.

To summarize, for the predict endpoint we expect a json payload with an instances key and as value a list of instances to predict, for example with two instances:

  
    {
       "instances": [
	   [1,2,3],
	   [4,5,6]
       ] 
    }

Or named features (only supported by TensorFlow and XGBoost models):

  
    {
       "instances": [
	   {
	   	"my_feature_1": 1,
		"my_feature_2": 2,
		"my_feature_3": 3
	   },
	   {
	   	"my_feature_1": 4,
		"my_feature_2": 5,
		"my_feature_3": 6
	   }
       ] 
    }

Or when the model supports probabilties, for example like some models in sklearn:

  
    {
       "instances": [
	   [1,2,3],
	   [4,5,6]
       ],
       "probabilities": true
    }

Deleting a model

When a model is not needed anymore, it is best to remove it. This can be achieved by the following call:

curl -u apiKey:apiSecret -X DELETE 'https://byoml.waylay.io/models/my-model-name'

You should get a response containg the message: Model succesfully deleted