Rules

Rules are defined by JSON, below is an example.

{
  "id": "rule1",
  "sql": "SELECT demo.temperature, demo1.temp FROM demo left join demo1 on demo.timestamp = demo1.timestamp where demo.temperature > demo1.temp GROUP BY demo.temperature, HOPPINGWINDOW(ss, 20, 10)",
  "actions": [
    {
      "log": {}
    },
    {
      "mqtt": {
        "server": "tcp://47.52.67.87:1883",
        "topic": "demoSink"
      }
    }
  ]
}

The parameters for the rules are:

Rule Logic

There are two ways to define the business logic of a rule. Either using SQL/actions combination or using the newly added graph API.

SQL rule

By specifying the sql and actions property, we can define the business logic of a rule in a declarative way. Among these, sql defines the SQL query to run against a predefined stream which will transform the data. The output data can then route to multiple locations by actions. See SQL for more info of eKuiper SQL.

Sources

eKuiper provides the following built-in sources,

• MQTT source, see MQTT source stream for more detailed info.
• EdgeX source by default is shipped in docker images, but NOT included in single download binary files, you use make pkg_with_edgex command to build a binary package that supports EdgeX source. Please see EdgeX source stream for more detailed info.
• HTTP pull source, regularly pull the contents at user’s specified interval time, see here for more detailed info.
  • Sources can be customized, see extension for more detailed info.

Sinks/Actions

Currently, below kinds of sinks/actions are supported:

• log: Send the result to log file.
• mqtt: Send the result to an MQTT broker.
• edgex: Send the result to EdgeX message bus.
• rest: Send the result to a Rest HTTP server.
• nop: Send the result to a nop operation.

Each action can define its own properties. There are several common properties:

Data Template

User can refer to Use Golang template to customize analaysis result in eKuiper for more detailed scenarios. If sendSingle is true, the data template will execute against a record; Otherwise, it will execute against the whole array of records. Typical data templates are:

For example, we have the sink input as

[]map[string]interface{}{{
    "ab" : "hello1",
},{
    "ab" : "hello2",
}}

In sendSingle=true mode:

• Print out the whole record

"dataTemplate": "{\"content\":{{json .}}}",

• Print out the ab field

"dataTemplate": "{\"content\":{{.ab}}}",

if the ab field is a string, add the quotes

"dataTemplate": "{\"content\":\"{{.ab}}\"}",

In sendSingle=false mode:

• Print out the whole record array

"dataTemplate": "{\"content\":{{json .}}}",

• Print out the first record

"dataTemplate": "{\"content\":{{json (index . 0)}}}",

• Print out the field ab of the first record

"dataTemplate": "{\"content\":{{index . 0 \"ab\"}}}",

• Print out field ab of each record in the array to html format

"dataTemplate": "<div>results</div><ul>{{range .}}<li>{{.ab}}</li>{{end}}</ul>",

Actions could be customized to support different kinds of outputs, see extension for more detailed info.

Functions supported in template

With the help of template functions, users can do a lot of transformation including formation, simple mathematics, encoding etc. The supported functions in eKuiper template includes:

1. Go built-in template functions.
2. An abundant extended function set from sprig library.
3. eKuiper extended functions.

eKuiper extends several functions that can be used in data template.

• (deprecated)json para1: The json function is used for convert the map content to a JSON string. UsetoJson from sprig instead.
• (deprecated)base64 para1: The base64 function is used for encoding parameter value to a base64 string. Convert the pramater to string type and use b64enc from sprig instead.

Dynamic properties

In the sink, it is common to fetch a property value from the result data to achieve dynamic output. For example, to write data into a dynamic topic of mqtt. The dynamic properties will be parsed as a data template. In below example, the sink topic is gotten from the selected topic using data template.

{
  "id": "rule1",
  "sql": "SELECT topic FROM demo",
  "actions": [{
    "mqtt": {
      "sendSingle": true,
      "topic": "prefix/{{.topic}}"
    }
  }]
}

In the above example, sendSingle property is used, so the sink data is a map by default. If not using sendSingle, you can get the topic by index with data template {{index . 0 “topic”}}.

Codecs

When the source of the rule reads in the event, it needs to parse and decode the various types of data from different types of sources into map type data for internal processing. After the rules are computed, the sink needs to encode the internal map type data into a proprietary format for various types of external systems. There are two main types of source/sink we support, one is that the connected external system has a fixed private format, so the codec work is already included in the source/sink implementation, e.g. EdgeX, Neuron; the other is that the connected external system only specifies the protocol of the connection, and the transmitted data allows a custom format, e.g. MQTT and ZeroMQ. The latter type of sourcing/sink must specify format and schemaId parameters to implement flexible codecs. For the management of supported codec formats and schemas, please refer to codecs.

Graph rule

Since eKuiper 1.6.0, eKuiper provides graph property in the rule model as an alternative way to create a rule. The property defines the DAG of a rule in JSON format. It is easy to map it directly to a graph in a GUI editor and suitable to serve as the backend of a drag and drop UI. An example of the graph rule definition is as below:

{
  "id": "rule1",
  "name": "Test Condition",
  "graph": {
    "nodes": {
      "demo": {
        "type": "source",
        "nodeType": "mqtt",
        "props": {
          "datasource": "devices/+/messages"
        }
      },
      "humidityFilter": {
        "type": "operator",
        "nodeType": "filter",
        "props": {
          "expr": "humidity > 30"
        }
      },
      "logfunc": {
        "type": "operator",
        "nodeType": "function",
        "props": {
          "expr": "log(temperature) as log_temperature"
        }
      },
      "tempFilter": {
        "type": "operator",
        "nodeType": "filter",
        "props": {
          "expr": "log_temperature < 1.6"
        }
      },
      "pick": {
        "type": "operator",
        "nodeType": "pick",
        "props": {
          "fields": ["log_temperature as temp", "humidity"]
        }
      },
      "mqttout": {
        "type": "sink",
        "nodeType": "mqtt",
        "props": {
          "server": "tcp://${mqtt_srv}:1883",
          "topic": "devices/result"
        }
      }
    },
    "topo": {
      "sources": ["demo"],
      "edges": {
        "demo": ["humidityFilter"],
        "humidityFilter": ["logfunc"],
        "logfunc": ["tempFilter"],
        "tempFilter": ["pick"],
        "pick": ["mqttout"]
      }
    }
  }
}

The graph property is a json structure with nodes to define the nodes presented in the graph and topo to define the edge between nodes. The node type can be built-in node types such as window node and filter node etc. It can also be a user defined node from plugins. Please refer to graph rule for more detail.

Options

The current options includes:

For detail about qos and checkpointInterval, please check state and fault tolerance.

The rule options can be defined globally in etc/kuiper.yaml under the rules section. The options defined in the rule json will override the global setting.