Use Guide

How to use EMQ X?

EMQ X Broker is free and it can be download at https://www.emqx.io/downloads#brokerUse guide - 图1 (opens new window).

EMQ X Enterprise can be downloaded and evaluated for free. You can download it from https://www.emqx.io/downloads#enterpriseUse guide - 图2 (opens new window), and then apply trial license at https://www.emqx.io/licenses#trialUse guide - 图3 (opens new window).

Also you can use the EMQ X enterprise version through public cloud service.

How to update EMQ X license?

Tags: Update

You need to two steps：

  1. After clicking “Download License”, browse to the “license.zip” file that you downloaded.

  2. Copy the two files(emqx.lic, emqx.key) in the zip file to the EMQX license directory.

  • If your installation package is a zip file, the licenses are under “emqx/etc/“;
  • For DEB/RPM package, the licenses are under “/etc/emqx/“;
  • For Docker image, the licenses are under “/opt/emqx/etc/“.

After the copy is completed, the license needs to be reloaded from the command line to complete the update：

  1. emqx_ctl license reload [license file path]

The update commands for different installation modes：

  1. ## zip packages
  2. ./bin/emqx_ctl license reload etc/emqx.lic
  4. ## DEB/RPM
  5. emqx_ctl license reload /etc/emqx/emqx.lic
  7. ## Docker
  8. docker exec -it emqx-ee emqx_ctl license reload /opt/emqx/etc/emqx.lic

Can EMQ X support customized protocols? How to implement?

Tags: Protocol Extends

TODO…

Can I capture device online and offline events? How to use it?

Tags: WebHook System Topic

EMQ X supports to capture device online and offline events through below 3 approaches,

  • Web Hook

  • Subscribe related $SYS topics

    • $SYS/brokers/${node}/clients/${clientid}/connected
    • $SYS/brokers/${node}/clients/${clientid}/disconnected

  • Directly save events into database

    The final approach is only supported in enterprise version, and supported database includes Redis, MySQL, PostgreSQL, MongoDB and Cassandra. User can configure database, client.connected and client.disconnected events in the configuration file. When a device is online or offline, the information will be saved into database.

I want to control topics can be used for specific clients, how to configure it in EMQ X?

Tags: ACL Pub/Sub

EMQ X can constrain clients used topics to realize device access controls. To use this feature, ACL (Access Control List) should be enabled, disable anonymous access and set acl_nomatch to ‘deny’ (For the convenience of debugging, the last 2 options are enabled by default, and please close them).

ACL can be configured in configuration file, or backend databases. Below is one of sample line for ACL control file, the meaning is user ‘dashboard’ can subscribe ‘$SYS/#’ topic. ACL configuration in backend databases is similar, refer to EMQ X document for more detailed configurations.

  1. {allow, {user, "dashboard"}, subscribe, ["$SYS/#"]}.

Can EMQ X support traffic control?

Tags: Rate Limit

Yes. Currently EMQ X supports to control connection rate and message publish rate. Refer to below for sample configuration.

  1. ## Value: Number
  2. listener.tcp.external.max_conn_rate = 1000
  4. ## Value: rate,burst
  5. listener.tcp.external.rate_limit = 1024,4096

How does the EMQ X achieve high concurrency and high availability?

Tags: Performance

High concurrency and availability are design goals of EMQ X. To achieve these goals, several technologies are applied:

  • Making maximum use of the soft-realtime, high concurrent and fault-tolerant Erlang/OTP platform;
  • Full asynchronous architecture;
  • Layered design of connection, session, route and cluster;
  • Separated messaging and control panel;

With the well design and implementation, a single EMQ X cluster can handle million level connections.

EMQ X supports clustering. The EMQ X performance can be scale-out with the increased number of nodes in cluster, and the MQTT service will not be interrupted when a single node is down.

Can EMQ X store messages to database?

Tags: Persistence

The EMQ X Enterprise edition supports data persistence. Supported databases are:

  • Redis
  • MongoDB
  • MySQL
  • PostgreSQL
  • Cassandra

Can I disconnect an MQTT connection from EMQ X server?

Tags: REAT API Dashboard

Yes. You can do it by invoking REST API provied by EMQ X, but the implementation is different in EMQ X 2.x and 3.x:

  • EMQ X customized protocol in 2.x versions.
  • Follow the process defined in MQTT 5.0 protocol after version 3.0.

Refer to below for API invocation:

  1. HTTP Method: DELETE 
  2. URL：api/[v2|v3]/clients/{clientid} 
  3. <!--Please notice the 2nd section in URL, and use the correct version number according to your EMQ X version. -->
  5. Returned response: 
  6. {
  7.     "code": 0,
  8.     "result": []
  9. }

Can EMQ X forward messages to Kafka?

Tags: Kafka Bridge Persistence

The EMQ X Enterprise edition integrates a Kafka bridge, it can bridge data to Kafka.

I use Kafka bridge in EMQ X enterprise, when will the MQTT Ack packet sent back to client? Is the time when message arriving EMQ X or after getting Ack message from Kafka?

Tags: Kafka

It’s up to Kafka bridge configuration, the configuration file is at /etc/emqx/plugins/emqx_bridge_kafka.conf

  1. ## Pick a partition producer and sync/async.
  2. bridge.kafka.produce = sync
  • Sync: MQTT Ack packet will be sent back to client after receiving Ack from Kafka.
  • Async: MQTT Ack packet will be sent back to client right after EMQ X receiving the message, and EMQ X will not wait the Ack returned from Kafka.

If the backend Kafka server is not available, then the message will be accumulated in EMQ X broker.

  • The message will be cached in memory before EMQ X 2.4.3 version, if the memeory is exhausted, then the EMQ X server will be down.
  • The message will be cached in disk after EMQ X 2.4.3 version, message will probably lost if the disk is full.

So we suggest you to closely monitor Kafka server, and recover Kafka service as soon as possible when it has any questions.

Does EMQ X support cluster auto discovery? What clustering methods are supported?

Tags: Cluster

EMQ X supports cluster auto discovery. EMQ X clustering can be done manually or automatically.

Currently supported clustering methods:

  • Manual clustering
  • Static clustering
  • Auto clustering using IP multi-cast
  • Auto clustering using DNS
  • Auto clustering using ETCD
  • Auto clustering using K8S

Can I forward MQTT messages EMQ X to other MQTT broker, like RabbitMQ?

Tags: RabbitMQ Bridge Persistence

EMQ X support forward messages to other MQTT broker. Using MQTT bridge, EMQ X can forward messages of interested topics to other broker.

Can I forward messages from EMQ X to MQTT services hosted on public cloud?

Tags: Bridge Cloud

EMQ X can forward messages to IoT Hub hosted on public cloud, this is a feature of EMQ X bridge.

Can other MQTT broker (for example Mosquitto) forward messages to EMQ X?

Tags: Mosquitto Bridge

EMQ X can receive messages from other broker, but it depends also on the implementation of other brokers, Mosquitto can forward messages to EMQ X, please refer to TODOUse guide - 图5 (opens new window)。

What should I do if I want trace the subscription and publish of some particular message?

Tags: Trace

EMQ X support the tracing of messages from particular client or under particular topic. You can use the command line tool emqx_ctl for tracing. The example below shows how to trace messages under ‘topic’ and save the result in ‘trace_topic.log’. For more details, please refer to EMQ X document.

  1. ./bin/emqx_ctl trace topic "topic" "trace_topic.log"

Does EMQ X support encrypted connection? What is the recommended deployment?

Tags: TLS

EMQ X Support SSL/TLS. In production, we recommend to terminate the TLS connection by Load Balancer. By this way, the connection between device and server(load balancer) use secured connection, and connection between load balancer and EMQ X nodes use general TCP connection.

How to troubleshoot if EMQ X can’t start after installation?

Tags: Debug

Execute $ emqx console to view the output.

  • logger command is missing

    1. $ emqx console
    2. Exec: /usr/lib/emqx/erts-10.3.5.1/bin/erlexec -boot /usr/lib/emqx/releases/v3.2.1/emqx -mode embedded -boot_var ERTS_LIB_DIR /usr/lib/emqx/erts-10.3.5.1/../lib -mnesia dir "/var/lib/emqx/mnesia/emqx@127.0.0.1" -config /var/lib/emqx/configs/app.2019.07.23.03.07.32.config -args_file /var/lib/emqx/configs/vm.2019.07.23.03.07.32.args -vm_args /var/lib/emqx/configs/vm.2019.07.23.03.07.32.args -- console
    3. Root: /usr/lib/emqx
    4. /usr/lib/emqx
    5. /usr/bin/emqx: line 510: logger: command not found

    Solution:

    • Centos/Redhat

      1. $ yum install rsyslog

    • Ubuntu/Debian

      1. $ apt-get install bsdutils

  • Missssl is missing

  1.     $ emqx console
  2.     Exec: /emqx/erts-10.3/bin/erlexec -boot /emqx/releases/v3.2.1/emqx -mode embedded -boot_var ERTS_LIB_DIR /emqx/erts-10.3/../lib -mnesia dir "/emqx/data/mnesia/emqx@127.0.0.1" -config /emqx/data/configs/app.2019.07.23.03.34.43.config -args_file /emqx/data/configs/vm.2019.07.23.03.34.43.args -vm_args /emqx/data/configs/vm.2019.07.23.03.34.43.args -- console
  3.     Root: /emqx
  4.     /emqx
  5.     Erlang/OTP 21 [erts-10.3] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:32] [hipe]
  7.     {"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,crypto}}},{kernel,start,[normal,[]]}}}"}
  8.     Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,crypto}}},{kernel,start,[normal,[]]}}})
  10.     Crash dump is being written to: log/crash.dump...done

Solution: Install openssl above version 1.1.1

  • License file is missing
  1.   $ emqx console
  2.   Exec: /usr/lib/emqx/erts-10.3.5.1/bin/erlexec -boot /usr/lib/emqx/releases/v3.2.1/emqx -mode embedded -boot_var ERTS_LIB_DIR /usr/lib/emqx/erts-10.3.5.1/../lib -mnesia dir "/var/lib/emqx/mnesia/emqx@127.0.0.1" -config /var/lib/emqx/configs/app.2019.07.23.05.52.46.config -args_file /var/lib/emqx/configs/vm.2019.07.23.05.52.46.args -vm_args /var/lib/emqx/configs/vm.2019.07.23.05.52.46.args -- console
  3.   Root: /usr/lib/emqx
  4.   /usr/lib/emqx
  5.   Erlang/OTP 21 [erts-10.3.5.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:32] [hipe]
  7.   Starting emqx on node emqx@127.0.0.1
  8.   Start http:management listener on 8080 successfully.
  9.   Start http:dashboard listener on 18083 successfully.
  10.   Start mqtt:tcp listener on 127.0.0.1:11883 successfully.
  11.   Start mqtt:tcp listener on 0.0.0.0:1883 successfully.
  12.   Start mqtt:ws listener on 0.0.0.0:8083 successfully.
  13.   Start mqtt:ssl listener on 0.0.0.0:8883 successfully.
  14.   Start mqtt:wss listener on 0.0.0.0:8084 successfully.
  15.   EMQ X Broker 3.2.1 is running now!
  16.   "The license certificate is expired!"
  17.   2019-07-23 05:52:51.355 [critical] The license certificate is expired!
  18.   2019-07-23 05:52:51.355 [critical] The license certificate is expired! System shutdown!
  19.   Stop mqtt:tcp listener on 127.0.0.1:11883 successfully.
  20.   Stop mqtt:tcp listener on 0.0.0.0:1883 successfully.
  21.   Stop mqtt:ws listener on 0.0.0.0:8083 successfully.
  22.   Stop mqtt:ssl listener on 0.0.0.0:8883 successfully.
  23.   Stop mqtt:wss listener on 0.0.0.0:8084 successfully.
  24.   [os_mon] memory supervisor port (memsup): Erlang has closed
  25.   [os_mon] cpu supervisor port (cpu_sup): Erlang has closed

Solution: Go to emqx.ioUse guide - 图6 (opens new window) to apply for a license or install the open source version of EMQ X Broker

Use of ssl resumption session in EMQ X

Tags: Performance

Modify the reuse_sessions = on in the emqx.conf configuration and take effect. If the client and the server are successfully connected through SSL, when the client connection is encountered for the second time, the SSL handshake phase is skipped, the connection is directly established to save the connection time and increase the client connection speed.

MQTT client disconnect statistics

Tags: Metrics

Execute emqx_ctl listeners to view the shutdown_count statistics under the corresponding port.

Client disconnect link error code list:

  • keepalive_timeout：MQTT keepalive time out

  • closed： TCP client disconnected (the FIN sent by the client did not receive the MQTT DISCONNECT)

  • normal： MQTT client is normally disconnected

  • einval：EMQ X wants to send a message to the client, but the Socket has been disconnected

  • function_clause：MQTT packet format error

  • etimedout：TCP Send timeout (no TCP ACK response received)

  • proto_unexpected_c：Repeatedly received an MQTT connection request when there is already an MQTT connection

  • idle_timeout： After the TCP connection is established for 15s, the connect packet has not been received yet.