Mapper-Framework

Mapper-Framework

Mapper-Framework provides a new Mapper development framework, which integrates DMI management plane and data plane capabilities, allowing devices to process data at the edge or in the cloud, improving the flexibility of device data management. Mapper-Framework can automatically generate users’ Mapper projects, simplify the complexity of user design and implementation of Mapper, and improve Mapper development efficiency.

Architecture

mapper framework

  1. Mapper-Framework provide a data push interface for pushing to user applications, the destination rule are defined through CRD.
  2. Mapper-Framework provide a database interface that can save data to database, the push rule are defined through CRD.
  3. Mapper-Framework provide the REST API, these APIs can access devices to obtain data. API does not support changing device property, because this will result in Inconsistent messages between cloud and edge.
  4. Mapper-Framework provide device driver interfaces to initialize the device and obtain device data.
  5. Mapper-Framework provide Makefile that can generate a mapper with one command.

DMI device management plane and device data plane can be implemented by mapper-framework, Developers only need to focus on device drivers. The grey part in generator (Driver) means that developers need to implement it. Defined interfaces DevPanel to manage devices, new interfaces will be added when features are added.

Details

Data Stream

dmi datapanel

Data Normalization

In order to transfer data between interface modules, standardization of data is necessary. These data should contain the necessary information generated by the data. The standardized data definition is DataModel.

Push

The data push module can push property values of devices to reachable user apps for consumption according to destination rules that defined by CRD. To meet the new requirements, the current v1beta1 CRD definitions of Device Instance add new fields PushMethod.

An example of a configuration file that defines mapper to push data to user applications is as follows:

  1. apiVersion: devices.kubeedge.io/v1beta1
  2. kind: Device
  3. metadata:
  4. name: beta1-device
  5. spec:
  6. deviceModelRef:
  7. name: beta1-model
  8. nodeName: worker-node1
  9. properties:
  10. - name: temp
  11. pushMethod:
  12. mqtt:
  13. address: tcp://127.0.0.1:1883
  14. topic: temp
  15. qos: 0
  16. retained: false
  17. ...

In the current CRD definition, MQTT and HTTP protocols are supported, and the cycle(default 1s) is defined by the DeviceProperty.ReportCycle. When mapper is executed, it will automatically parse the value of the pushMethod field and execute the DataPanel interface to push data.
In the future, DataPanel interface will add more interfaces to ensure data security.

DataBase

The database module can store device data to a local database according to destination rules that defined by DBMethod. An example of a configuration file that defines mapper to push data to user database is as follows:

  1. apiVersion: devices.kubeedge.io/v1beta1
  2. kind: Device
  3. metadata:
  4. name: beta1-device
  5. spec:
  6. deviceModelRef:
  7. name: beta1-model
  8. nodeName: worker-node1
  9. properties:
  10. - name: temp
  11. pushMethod:
  12. dbMethod:
  13. influxdb2:
  14. influxdb2ClientConfig:
  15. url: http://127.0.0.1:8086
  16. org: test-org
  17. bucket: test-bucket
  18. influxdb2DataConfig:
  19. measurement: stat
  20. tag:
  21. unit: temperature
  22. fieldKey: beta1test
  23. ...

Now we provide Influx2,Redis,TDengine database interfaces, we will add more databases later.

Pull

The HTTP server was created to provide API services. It supports directly obtaining device data from the device. The URLs listed below are given in the form of local IP. You can use these services from any network accessible to mapper.

Port 7777 is enabled by default.

deviceInstance-ID according to your own CRD definition. propertyName according to your own CRD definition.

Ping

  1. Detect whether the RESTful service starts normally
    Method: GET
    Url: https://127.0.0.1:7777/api/v1/ping
    Response:

    1. {
    2. "apiVersion": "v1",
    3. "statusCode": 200,
    4. "timeStamp": "2023-08-18T09:57:29+08:00",
    5. "Message": "This is v1 API, the server is running normally."
    6. }

Device Data

  1. Get device’s Data
    Method=GET
    Url: https://127.0.0.1:7777/api/v1/device/deviceInstance-ID/propertyName Response:

    1. {
    2. "apiVersion": "v1",
    3. "statusCode": 200,
    4. "timeStamp": "2023-08-18T09:57:35+08:00",
    5. "Data": {
    6. "DeviceName": "deviceInstance-ID",
    7. "PropertyName": "propertyName",
    8. "Value": "data",
    9. "Type": "dataType",
    10. "CollectTimeStamp": 1692323855044
    11. }
    12. }

Device MetaData

  1. Get device’s Model
    Method=GET
    Url: https://127.0.0.1:7777/api/v1/meta/model/deviceInstance-ID
    Response:

    1. {
    2. "apiVersion": "v1",
    3. "statusCode": 200,
    4. "timeStamp": "2023-08-18T09:57:37+08:00",
    5. "name": "model-name",
    6. "properties": [
    7. {
    8. "name": "propertyName-1",
    9. "dataType": "property data type",
    10. "description": "property description",
    11. "accessMode": "ReadWrite",
    12. "defaultValue": 100
    13. },
    14. ...
    15. ]
    16. }