NAV
cURL JavaScript

Waylay Broker

Waylay Broker is at this URL: https://data.waylay.io

For testing you can use labs.

Authentication

When connecting to the broker, you need to provide your credentials (apiKey, apiSecret) by http basic authentication or where not possible in the url (websockets)

For more sensitive environments we also have a device gateway where you can get per-device credentials. These credentials only allow sending or receiving data on the device’s channel.

Resources

All data handled by the broker is linked to a resource. A resource is a generic identifier and can represent devices, persons, cars, customers, anything…

Events and commands

For each resource we provide two channels:

Posting messages towards Broker

The waylay data endpoint lets you store and distribute messages. This can be performed over different protocols: http or websockets.

As soon as data is send to the Broker, data is stored in two different databases, time series database and document database. In the document storage, data is stored without any pre-processing, with original JSON object as it was received. When JSON object (or array of JSON objects) comes to the Broker, Broker also tries to save data in the time series database. In order to achieve that, broker will inspect incoming JSON object and store every metric that is found in the JSON object.

To keep your data private you use your waylay api key and secret. This will also enable the forwarding of your data to your tasks or buckets.

A submitted message is defined by 2 things

For instance, the resource can be the phone and the parameters something like temperature, humidity, acceleration etc.

Important Note : Data will be stored with the timestamp when the object arrived at the Broker. Should you wish to insert data with other timestamp, you must in the JSON object provide a timestamp with a value that is in epoch in milliseconds. For instance: {"temp":21, "humidity": 0.35, "timestamp" : 1475139600000}

HTTP REST API

The REST API is mainly intended for devices that have an HTTP stack available and don’t need to send huge amounts of data. It also allows you to fetch the current and last 100 items from the store.

First you will need to fetch your API key from the profile page.

Available urls

Available urls

GET           /resources/:resourceId/events              // last n / sse / ws publish
GET           /resources/:resourceId/events/subscribe    // ws
GET           /resources/:resourceId/events/publish      // ws
POST          /resources/:resourceId/events

GET           /resources/:resourceId/commands            // last n / sse / ws subscribe
GET           /resources/:resourceId/commands/subscribe  // ws
GET           /resources/:resourceId/commands/publish    // ws
POST          /resources/:resourceId/commands

Posting data (object) to the storage and rule engine

When you post a message, by default, the message is both forwarded to the rule enging and stored in three different databases (document store, time series database and metamodel).

Example of posting data using resource events endoint:

curl -i --user apiKey:apiSecret  -H "Content-Type: application/json" -X POST
    -d '{
          "foo":123,
          "bar":"hello"
      }'
    https://data.waylay.io/resources/testresource/events

Example of posting data with resource defined in the payload:

curl -i  --user apiKey:apiSecret -H "Content-Type: application/json" -X POST
    -d '{
        "foo":123,
        "bar":"hello",
        "resource":"testresource"
        }'
    https://data.waylay.io/messages

There are multiple ways to post the same message towards the Broker.

Posting array of objects

You can post objects as an array.

BULK import

curl -i  --user apiKey:apiSecret
    -H "Content-Type: application/json"
    -X POST \
    -d '[
      {   
          "foo": 12,
          "bar":"hello",
          "resource":"testresource1",
      },  {   
          "foo": 33,
          "bar":"world",
          "resource":"testresource2",
      }]'
    https://data.waylay.io/messages

Forwarding data to the rule engine, without the storage

Example of posting data to the engine only, using resource events endoint:

curl -i --user apiKey:apiSecret  -H "Content-Type: application/json" -X POST
    -d '{
          "foo":123,
          "bar":"hello"
      }'
    https://data.waylay.io/resources/testresource/events?store=false

Forwarding data to the storage, without forward to the engine

Example of posting data to the engine only, using resource events endoint:

curl -i --user apiKey:apiSecret  -H "Content-Type: application/json" -X POST
    -d '{
          "foo":123,
          "bar":"hello"
      }'
    https://data.waylay.io/resources/testresource/events?forward=false

Posting data with a specified time-to-live

Example of posting data which will only be available for 3600s:

curl -i --user apiKey:apiSecret  -H "Content-Type: application/json" -X POST
    -d '{
          "foo":123,
          "bar":"hello"
      }'
    https://data.waylay.io/resources/testresource/events?ttl=3600

When posting data, you can specify how long the data needs to be available in the system (both in the timeseries database and the document database). After this time the data is automatically removed.

You specify this by the query parameter ttl, either specified as a number of seconds, or with a period in the format #[w,d,h,m,s] ( as in # weeks/days/hours/minutes/seconds )

Example of posting data which will be available for 52 weeks:

curl -i --user apiKey:apiSecret  -H "Content-Type: application/json" -X POST
    -d '{
          "foo":123,
          "bar":"hello"
      }'
    https://data.waylay.io/resources/testresource/events?ttl=52w

Websockets

You can setup a websocket for posting data in a streaming fashion (multiple resources)

Endpoint to post data in a streaming fashion

wss://data.waylay.io/socket

Messages posted to this endpoint must contain a resource key to identify the thing the data is coming from and optionally can have a timestamp message with the time(stamp) (in epoch milliseconds) of the data (if timestamp is not present the data will be stored with the timestamp it arrived at the broker)

Per resource

You can also set up web sockets for a specific resource

Endpoints to connect to using secure web sockets:

wss://data.waylay.io/resources/:resourceId/commands/subscribe (listening to commands)
wss://data.waylay.io/resources/:resourceId/commands/publish (publishing commands to an other device)
wss://data.waylay.io/resources/:resourceId/events/subscribe (listening to events from a device)
wss://data.waylay.io/resources/:resourceId/events/publish (publishing events)

The labs lets you play with websocket support. Remark that the subscribe connections are kept alive by empty {} json messages if no traffic has passed in 1 minute.

When submitting invalid data you will get a response back with an error message.

{
  "error": "Json parse error",
  "details": [
    "/resource <- String value expected"
  ]
}

Retrieving data

Time series data

Getting raw time series data

curl -i --user apiKey:apiSecret
    'https://data.waylay.io/resources/testresource/series/temperature?from=1472947200000&until=1474588800000'

Getting the latest value for a series

curl -i --user apiKey:apiSecret
    'https://data.waylay.io/resources/testresource/series/temperature/latest'

Getting raw time series data

You can specify from and until timestamps. Both are supplied as epoch time in milliseconds (i.e. the number of milliseconds since 1970-01-01 00:00:00.000 GMT).

If until is not specified, the current time will be used. If from is not specified, it will be set to seven days before the until value.

Both from and until are inclusive, meaning that data points which fall on these exact timestamps will be included in the response.

Aggregates

Group by example:

curl -i \
     --user apiKey:apiSecret \
     'https://data.waylay.io/resources/testresource/series/temperature?from=1472947200000&until=1474588800000&grouping=hour&aggregate=mean'

You can get data on which aggregation is computed using the following aggregation operations:

Multiple aggregates can be specified as a comma-separated list, and the returned response will include a value at each sample point for the requested aggregation.

Multiple aggregates example

curl -i \
     --user apiKey:apiSecret \
     'https://data.waylay.io/resources/testresource/series/temperature?from=1472947200000&until=1474588800000&grouping=hour&aggregates=mean,min,max'

Grouped by

Grouping can also be specified with a grouping expression consisting of a value and a time unit. For example, 42m to represent 42 minutes.

The supported time units are

All groups are aligned with the from timestamp. For example, if a from timestamp of 0 is specified along with a grouping of 1s (one second), then the first sample point returned will be labelled with timestamp 0 and include aggregated data from 0 (inclusive) to 1000 (exclusive). The second sample point will be labelled with timestamp 1000 and include aggregated data from 1000 (inclusive) to 2000 (exclusive), and so on.

All sample points from the from timestamp to the until timestamp will be included in the time series aggregation response. If there is no data for a given sample point, the sample point will be populated with a null value.

Metadata

Retrieving list of existing timeseries on resource testresource

curl -i --user apiKey:apiSecret \
   https://data.waylay.io/resources/testresource/series?metadata

You can request the list of timeseries that exist for a certain resource by asking for the series metadata. The response will be an array of objects with following data:

Messages

Getting latest message from storage

curl -i --user apiKey:apiSecret \
    https://data.waylay.io/resources/testresource/current

Getting last X messages from storage

curl -i --user apiKey:apiSecret \
    https://data.waylay.io/resources/testresource/messages

Below call has been deprecated. Please use the above call to retrieve the last X messages

curl -i --user apiKey:apiSecret \
    https://data.waylay.io/resources/testresource/series

You can always retrieve up to the last 100 messages for every resource over the REST calls. Coming back to the example where resource is the phone, waylay platform would store 100 messages, where each message would hold information about underlying parameters (temperature, humidity etc.)

Streaming all messages using NDJSON

This is a firehose stream of all messages that are sent to the broker. If you can not keep up with reading the stream messages will be dropped! The connection is kept alive by sending empty {} json messages if no traffic has passed in 1 minute.

curl -i --user apiKey:apiSecret https://data.waylay.io/messages

Streaming using WebSockets

Streaming using WebSockets

wss://data.waylay.io/resources/testresource/socket?apiKey=...&apiSecret=...

You can stream data for a specific resource by setting up a WebSocket to the following url. Remark that the connection is kept alive by empty {} json messages if no traffic has passed in 1 minute.

Deleting data

Messages

Removing all messages

curl -i --user apiKey:apiSecret -X DELETE \
    https://data.waylay.io/resources/testresource/messages

You can remove all latest messages for a resource. This will not delete timeseries data for the properties of those messages.

All data for a resource

Removing all data

curl -i --user apiKey:apiSecret -X DELETE \
     https://data.waylay.io/resources/testresource

Removing all data before some date

curl -i --user apiKey:apiSecret -X DELETE \
     https://data.waylay.io/resources/testresource?until=1501538400000

You can delete all data (both messages and timeseries data) for a resource.

Specifying the query parameter until will only delete the data until (and including) the provided timestamp (milliseconds since epoch). The parameter also allows to specify a relative period in the format #[w,d,h,m,s,ms]. This period will be substracted from the current time to become the until timestamp

Remove all data older then 7 days

curl -i --user apiKey:apiSecret -X DELETE \
     https://data.waylay.io/resources/testresource?until=7d