Mapper-Framework

Mapper-Framework

Mapper-Framework provides a new Mapper development framework, which integrates DMI management plane and data plane capabilities, allowing devices to process data at the edge or in the cloud, improving the flexibility of device data management. Mapper-Framework can automatically generate users’ Mapper projects, simplify the complexity of user design and implementation of Mapper, and improve Mapper development efficiency.

Architecture

1. Mapper-Framework provide a data push interface for pushing to user applications, the destination rule are defined through CRD.
2. Mapper-Framework provide a database interface that can save data to database, the push rule are defined through CRD.
3. Mapper-Framework provide the REST API, these APIs can access devices to obtain data. API does not support changing device property, because this will result in Inconsistent messages between cloud and edge.
4. Mapper-Framework provide device driver interfaces to initialize the device and obtain device data.
5. Mapper-Framework provide Makefile that can generate a mapper with one command.

DMI device management plane and device data plane can be implemented by mapper-framework, Developers only need to focus on device drivers. The grey part in generator (Driver) means that developers need to implement it. Defined interfaces DevPanel to manage devices, new interfaces will be added when features are added.

Details

Data Stream

Data Normalization

In order to transfer data between interface modules, standardization of data is necessary. These data should contain the necessary information generated by the data. The standardized data definition is DataModel.

Push

The data push module can push property values of devices to reachable user apps for consumption according to destination rules that defined by CRD. To meet the new requirements, the current v1beta1 CRD definitions of Device Instance add new fields PushMethod.

An example of a configuration file that defines mapper to push data to user applications is as follows:

apiVersion: devices.kubeedge.io/v1beta1
kind: Device
metadata:
  name: beta1-device
spec:
  deviceModelRef:
    name: beta1-model
  nodeName: worker-node1
  properties:
    - name: temp
      pushMethod:
        mqtt:
          address: tcp://127.0.0.1:1883
          topic: temp
          qos: 0
          retained: false
      ...

In the current CRD definition, MQTT and HTTP protocols are supported, and the cycle(default 1s) is defined by the DeviceProperty.ReportCycle. When mapper is executed, it will automatically parse the value of the pushMethod field and execute the DataPanel interface to push data.
In the future, DataPanel interface will add more interfaces to ensure data security.

DataBase

The database module can store device data to a local database according to destination rules that defined by DBMethod. An example of a configuration file that defines mapper to push data to user database is as follows:

apiVersion: devices.kubeedge.io/v1beta1
kind: Device
metadata:
  name: beta1-device
spec:
  deviceModelRef:
    name: beta1-model
  nodeName: worker-node1
  properties:
    - name: temp
      pushMethod:
        dbMethod:
          influxdb2:
            influxdb2ClientConfig:
              url: http://127.0.0.1:8086
              org: test-org
              bucket: test-bucket
            influxdb2DataConfig:
              measurement: stat
              tag:
                unit: temperature
              fieldKey: beta1test
      ...

Now we provide Influx2,Redis,TDengine database interfaces, we will add more databases later.

Pull

The HTTP server was created to provide API services. It supports directly obtaining device data from the device. The URLs listed below are given in the form of local IP. You can use these services from any network accessible to mapper.

Port 7777 is enabled by default.

deviceInstance-ID according to your own CRD definition. propertyName according to your own CRD definition.

Ping

1. Detect whether the RESTful service starts normally
   Method: GET
   Url: https://127.0.0.1:7777/api/v1/ping
   Response:

   {
    "apiVersion": "v1",
    "statusCode": 200,
    "timeStamp": "2023-08-18T09:57:29+08:00",
    "Message": "This is v1 API, the server is running normally."
    }

Device Data

1. Get device’s Data
   Method=GET
   Url: https://127.0.0.1:7777/api/v1/device/deviceInstance-ID/propertyName Response:

   {
    "apiVersion": "v1",
    "statusCode": 200,
    "timeStamp": "2023-08-18T09:57:35+08:00",
    "Data": {
        "DeviceName": "deviceInstance-ID",
        "PropertyName": "propertyName",
        "Value": "data",
        "Type": "dataType",
        "CollectTimeStamp": 1692323855044
        }
    }

Device MetaData

1. Get device’s Model
   Method=GET
   Url: https://127.0.0.1:7777/api/v1/meta/model/deviceInstance-ID
   Response:

   {
    "apiVersion": "v1",
    "statusCode": 200,
    "timeStamp": "2023-08-18T09:57:37+08:00",
    "name": "model-name",
    "properties": [
        {
            "name": "propertyName-1",
            "dataType": "property data type",
            "description": "property description",
            "accessMode": "ReadWrite",
            "defaultValue": 100
        },
        ...
        ]
    }