Releases

e4.4.11

Release Date: 2022-11-26

This release included 23 enhancements and 21 bug fixes. Among the enhancements, there are new exciting new features worth highlighting.

  • Google PubSub integration as a Rule-Engine data bridge.
  • OCSP (Online Certificate Status Protocol) Stapling.
  • CRL (Certificate Revocation List) cache.
  • Pulsar data bridge supports data buffering.
  • OTP upgrade from 24.1.5-3 to 24.3.4.2-1.
  • Customizable client aliases to make it easier to when creating customized authentication and authorization.

It is possible to hot-upgrade from the older version e4.4 to this version. Please note though, in order to start making use of the new features such as OCSP Stapling, and CRL cache, a node restart (and configuration change) is required.

Enhancements

  • Upgraded Pulsar client to 0.7.0. Now it’s possible to buffer messages to be produced to Pulsar while EMQX has no connection to the Pulsar broker. Such messages will be retained for a configurable amount of time.

    Also, credentials such as basic auth and JWT tokens used for Pulsar authentication will be censored out when a crash happens, thus avoiding secret leakage to logs.

    Note: if an older EMQX version is upgraded to a version containing this update, and then downgraded back to the previous old version, there’s a risk that some messages that were sent in a sync manner might be sent but regarded as a timeout error, which in turn causes such clients to be disconnected.

  • Added hot-configuration support for OCSP stapling and CRL checking/caching. #1528Change log - 图1 (opens new window)

  • Added a new rule engine bridge and corresponding rule action for GCP PubSub #1523Change log - 图2 (opens new window).

  • Support to use placeholders like ${var} in the Collection field of Rule-Engine’s MongoDB actions #1503Change log - 图3 (opens new window).

  • Add a format check to the host field of the InfluxDB resource in Rule-Engine #1426Change log - 图4 (opens new window). The host field should be an ip/domain without scheme and port.

  • OTP upgrade from 24.1.5-3 to 24.3.4.2-1 #9265Change log - 图5 (opens new window). Change highlights:

  • Added support for OCSP stapling and CRL caching #9297Change log - 图7 (opens new window).

  • Added support for specifying custom modules for adding clientid and common name aliases #9297Change log - 图8 (opens new window). Now you can implement a simple callback to enrich clients with aliases, and then use the aliases in the authentication and authorization (ACL) rules’ place holders (%cida for clientid alias and %cna for username alias).

  • Added support for specifying custom modules for custom authentication #9297Change log - 图9 (opens new window). To support simple authentication rules, it is no longer necessary to implement a full-blown plugin.

  • Added a JWT management for Rule-Engine, for creating and refreshing JWT tokens in rule engine actions #9241Change log - 图10 (opens new window). This feature is so far only used in EMQX Enterprise Google PubSub integration. Can be used as webhook integration’s JWT authentication against the webhook service endpoint.

  • Make sure listener’s tls_versions config value is one or more of tlsv1, tlsv1.1, tlsv1.2, tlsv1.3 #9260Change log - 图11 (opens new window).

  • Remove useless information from the dashboard listener failure log #9260Change log - 图12 (opens new window).

  • We now trigger the 'message.acked' hook after the CoAP gateway sends a message to the device and receives the ACK from the device #9264Change log - 图13 (opens new window). With this change, the CoAP gateway can be combined with the offline message caching function (in the emqx enterprise), so that CoAP devices are able to read the missed messages from the database when it is online again.

  • Support to use placeholders like ${var} in the HTTP Headers of Rule-Engine’s Webhook actions #9239Change log - 图14 (opens new window).

  • Asynchronously refresh the resources and rules during emqx boot-up #9199Change log - 图15 (opens new window). This is to avoid slowing down the boot if some resources spend long time establishing the connection.

  • Add a warning log if the ACL check failed for subscription #9124Change log - 图16 (opens new window). This is to make the ACL deny logging for subscription behave the same as for publish.

  • JWT ACL claim supports all action to imply the rules applie to both pub and sub #9044Change log - 图17 (opens new window).

  • Added a log censor to avoid logging sensitive data #9189Change log - 图18 (opens new window). If the data to be logged is a map or key-value list which contains sensitive key words such as password, the value is obfuscated as ******.

  • Enhanced log security in ACL modules, sensitive data will be obscured #9242Change log - 图19 (opens new window).

  • Add management.bootstrap_apps_file configuration to bulk import default app/secret when EMQX initializes the database #9273Change log - 图20 (opens new window).

  • Added two new configs for deterministic order of authentication and ACL checks #9283Change log - 图21 (opens new window). The two new global config names are auth_order and acl_order. When multiple ACL or auth plugins (or modules) are enabled, without this config, the order (in which each backend is queried) is determined by the start/restart order of the plugin (or module). Meaning, if a plugin (or module) is restarted after initial boot, it may get ordered to the end of the list. With this config, you may set the order with a comma-speapated ACL or auth plugin names (or aliases). For example: acl_order = jwt,http, this will make sure jwt is always checked before http, meaning if JWT is not found (or no acl cliam) for a client, then the ACL check will fallback to use the HTTP backend.

  • Added configurations to enable more client.disconnected events (and counter bumps) #9267Change log - 图22 (opens new window). Prior to this change, the client.disconnected event (and counter bump) is triggered when a client performs a ‘normal’ disconnect, or is ‘kicked’ by system admin, but NOT triggered when a stale connection had to be ‘discarded’ (for clean session) or ‘takeovered’ (for non-clean session) by new connection. Now it is possible to set configs broker.client_disconnect_discarded and broker.client_disconnect_takeovered to on to enable the event in these scenarios.

  • For Rule-Engine resource creation failure, delay before the first retry #9313Change log - 图23 (opens new window). Prior to this change, the retry delay was added after the retry failure.

Bug fixes

  • Fix the default authentication mechanism of Kafka resource changed to NONE from PLAIN when upgrading emqx from e4.4.5 and older versions #1509Change log - 图24 (opens new window).

  • Fix an upgrade issue for JWT authentication plugin #1558Change log - 图25 (opens new window). When upgrading from e4.4.3 or earlier, an EMQX internal resource which holds the keys will have to be restarted, during the restart, clients may fail to be authenticated.

  • Fixed the option to choose the reset_by_subscriber offset reset policy in Kafka Consumer #1463Change log - 图26 (opens new window).

  • Added the missing tlsv1.3 option to tls_versions in hot-config #1532Change log - 图27 (opens new window).

  • Made Rule-Engine able to connect SQL server when its listening port is not the default (1433) #1464Change log - 图28 (opens new window).

  • Make sure Schema-Registry API supports Percent-encoding name in HTTP request URI #1497Change log - 图29 (opens new window). Note that the name in POST /api/v4/schemas request body should not be percent-encoded as it’s a JSON field value.

  • Fix an upgrade issue for JWT authentication plugin #1554Change log - 图30 (opens new window). When upgrading from e4.3.9 or earlier, an EMQX internal resource which holds the keys will have to be restarted, during the restart, clients may fail to be authenticated.

  • Fix get trace list crash when trace not initialize. #9156Change log - 图31 (opens new window)

  • Fix create trace sometime failed by end_at time has already passed. #9156Change log - 图32 (opens new window)

  • Fix that after uploading a backup file with an non-ASCII filename, HTTP API GET /data/export fails with status code 500 #9224Change log - 图33 (opens new window).

  • Improve the display of rule’s ‘Maximum Speed’ counter to only reserve 2 decimal places #9185Change log - 图34 (opens new window). This is to avoid displaying floats like 0.30000000000000004 on the dashboard.

  • Fix the issue that emqx prints too many error logs when connecting to mongodb but auth failed #9184Change log - 图35 (opens new window).

  • Fix that after receiving publish in idle mode the emqx-sn gateway may panic #9024Change log - 图36 (opens new window).

  • “Pause due to rate limit” log level demoted from warning to notice #9134Change log - 图37 (opens new window).

  • Restore old emqx_auth_jwt module API, so the hook callback functions registered in older version will not be invalidated after hot-upgrade #9144Change log - 图38 (opens new window).

  • Fixed the response status code for the /status endpoint #9210Change log - 图39 (opens new window). Before the fix, it always returned 200 even if the EMQX application was not running. Now it returns 503 in that case.

  • Fix message delivery related event encoding #9226Change log - 图40 (opens new window) For Rule-Engine’s input events like $events/message_delivered, and $events/message_dropped, if the message was delivered to a shared-subscription, the encoding (to JSON) of the event will fail. Affected versions: v4.3.21, v4.4.10, e4.3.16 and e4.4.10.

  • Make sure Rule-Engine API supports Percent-encoding rule_id and resource_id in HTTP request path #9190Change log - 图41 (opens new window). Note that the id in POST /api/v4/rules should be literals (not encoded) when creating a rule or resource. See docs Create RuleChange log - 图42 (opens new window) Create ResourceChange log - 图43 (opens new window).

  • Calling ‘DELETE /alarms/deactivated’ now deletes deactived alarms on all nodes, including remote nodes, not just the local node #9280Change log - 图44 (opens new window).

  • When republishing messages or bridge messages to other brokers, check the validity of the topic and make sure it does not have topic wildcards #9291Change log - 图45 (opens new window).

  • Disable authorization for api/v4/emqx_prometheus endpoint on management api listener (default 8081) #9294Change log - 图46 (opens new window).

e4.4.10

Release Date: 2022-10-14

Enhancements

  • Added more Kafka action parameter checks

    • TCP send buffer size and max batch size parameters are not allowed to be left blank from the configration UI.
    • The combination of “Produce Strategy” set to ‘key_dispatch’ and the “Key” set to “none” is now not allowed, the dashboard will get an error with text like: “with strategy set to ‘key_dispatch’, key is not allowed to be ‘none’”

  • TLS listener memory usage optimization #9005Change log - 图47 (opens new window). New config listener.ssl.$NAME.hibernate_after to hibernate TLS connection process after idling. Hibernation can reduce RAM usage significantly, but may cost more CPU. This configuration is by default disabled. Our preliminary test shows a 50% of RAM usage decline when configured to ‘5s’.

  • TLS listener default buffer size to 4KB #9007Change log - 图48 (opens new window). Eliminate uncertainty that the buffer size is set by OS default.

  • Disable authorization for api/v4/emqx_prometheus endpoint #8955Change log - 图49 (opens new window).

  • Added a test to prevent a last will testament message to be published when a client is denied connection #8894Change log - 图50 (opens new window).

  • More rigorous checking of flapping to improve stability of the system #9045Change log - 图51 (opens new window). Previsouly only normal disconnects are counted, now the connection rejections (e.g. authentication failure) is also included. Find more about flapping detection in EMQX documentChange log - 图52 (opens new window)

  • QoS1 and QoS2 messages in session’s buffer are re-dispatched to other members in the group when the session terminates #9094Change log - 图53 (opens new window). to prevent sessions from buffering messages, however this acknowledgement costs extra resources.

  • Fix delayed publish timing inaccuracy caused by OS time change #8908Change log - 图54 (opens new window).

Bug fixes

  • Fix load_modules reset after new node joins the cluster. Prior to this fix, if load_modules for a cluster has been changed, adding a new node to the cluster with default modules would cause the other nodes to reset to default too. In this fix, the node which is going to join the cluster will copy the loaded_modules from the oldest node in the cluster.

  • Fix getting subscriptions from backends successfully with QoS values out of range [0, 2]. Before this change, when we add subscriptions for clients from backends like Redis or MySQL, we won’t validate the QoS. For example if the QoS is an integer -1, the topic was still subscribed successfully with QoS -1, if we send a message to this topic, then an error will occur and the MQTT connection will crash. After this change QoS will be clamped into range [0, 2].

  • Fix Rule-Engine increased ‘success’ counter when get subscriptions from Redis failed (due to query Redis timeout).

  • Fix Rule-Engine increased ‘success’ counter when saving offline messages with QoS = 0. We don’t allow saving offline messages to backends with QoS = 0, so we need to increase the ‘failed’ counter instead of the ‘success’ counter in this case.

  • Fix the verify field is missing from the SSL settings of redis-cluster and redis-sentinel resources.

  • Fixed Redis resource liveness problem issue. Prior to this fix, the resource is considered alive when connection can be established. The fix is to perform a PING query to make sure the service is alive.

  • Fix the redis-cluster resource prints too many error logs when redis servers are not avaliable.

  • Fixed an internal Redis resource ID clashing. This clashing may cause resources in use getting deleted when deleting another resource.

  • Mask secret/password in the resource/module creation UI.

  • Fix HTTP client library to handle SSL socket passive signal #9145Change log - 图55 (opens new window).

  • Hide redis password in error logs #9071Change log - 图56 (opens new window). More changes in redis client included in this release:

    • Improve redis connection error logging eredis#19Change log - 图57 (opens new window). Also added support for eredis to accept an anonymous function as password instead of passing around plaintext args which may get dumpped to crash logs (hard to predict where). This change also added format_status callback for gen_server states which hold plaintext password so the process termination log and sys:get_status will print ‘******‘ instead of the password to console.
    • Avoid pool name clashing eredis_cluster#22Change log - 图58 (opens new window). Same format_status callback is added here too for gen_servers which hold password in their state.

  • Fix shared subscription message re-dispatches #9094Change log - 图59 (opens new window).

    • When discarding QoS 2 inflight messages, there were excessive logs
    • For wildcard deliveries, the re-dispatch used the wrong topic (the publishing topic, but not the subscribing topic), caused messages to be lost when dispatching.

  • Fix shared subscription group member unsubscribe issue when ‘sticky’ strategy is used. Prior to this fix, if a previously picked member unsubscribes from the group (without reconnect) the message is still dispatched to it. This issue only occurs when unsubscribe with the session kept. Fixed in #9119Change log - 图60 (opens new window)

  • Fix shared subscription ‘sticky’ strategy when there is no local subscriptions at all. Prior to this change, it may take a few rounds to randomly pick group members until a local subscriber is hit (and then start sticking to it). After this fix, it will start sticking to whichever randomly picked member even when it is a subscriber from another node in the cluster. Fixed in #9122Change log - 图61 (opens new window)

  • Fix rule engine fallback actions metrics reset #9125Change log - 图62 (opens new window).

e4.4.9

Release Date: 2022-09-17

Enhancements

  • The exp, nbf and iat claims in JWT authentication support non-integer timestamps

Bug fixes

  • Fix rule engine update behaviour which may initialize actions for disabled rules
  • Fix inaccurate delayed publish due to OS time changes
  • Fix the issue that the IP address bound to the Dashboard listener did not take effect
  • Fix the issue that shared subscriptions might get stuck in an infinite loop when shared_dispatch_ack_enabled is set to true
  • Fix the issue that the rule engine SQL crashes when subject matching null values

e4.4.8

Release Date: 2022-08-29

Enhancements

  • Add GET /trace/:name/detail API to view log trace file information
  • Improve the log when LwM2M packet parsing fails
  • Improve the rule engine error log, the log will contain the rule ID when the action execution fails
  • Improve log when loaded_modules and loaded_plugins files do not exist
  • Add a guide for changing the default password on Dashboard
  • Improved import performance for Protobuf Schema files

Bug fixes

  • Fix client.disconnected event not trigger in some cases
  • Fix the issue that the JWK authentication module could not be started later when the JWKS service was not ready in time
  • Fix the issue that setting the listener port via an environment variable would prevent either listener from being stopped
  • Fix the issue that the built-in database authentication did not distinguish the pagination statistics of the authentication data of the client ID and username
  • Fix the issue that the module status would be reset after EMQX restarts after hot upgrade
  • Fix Redis driver process leak problem
  • Fix rule engine MQTT bridge to AWS IOT connection timeout issue
  • Fix GET /listener request crashing when listener is not ready
  • Fix the issue that the comparison between any variable and null value in the rule engine SQL always returns false after e4.4.1
  • Fix the issue that when the execution priority of ExHook is higher than that of the rule engine, the topic filtered by the ExHook Message Hook will not trigger the rule engine
  • Fix the issue that the write request of TDEngine may fail because the peer end closes the network connection
  • Fix the issue that the configuration of the MQTT-SN module other than the listener would not take effect
  • Fix the issue that the ExHook management process was forcibly killed due to the supervisor shutdown timeout
  • Fix the issue that the Client ID parameter in ExProto client.connect hook is not defined
  • Fix ExProto not triggering disconnect event when client is kicked

e4.4.7

Release Date: 2022-08-11

Important Changes

  • As of version 4.4.7, we will no longer provide packages for macOS 10

Enhancements

  • Allows the connection process to be configured to be garbage collected after the TLS handshake is complete to reduce memory footprint, which can reduce memory consumption by about 35% per SSL connection, but increases CPU consumption accordingly
  • Allows configuring the log level of the TLS handshake log to view the detailed handshake process

Bug fixes

  • Fix the issue that EMQX could not be started when deployed through Helm Chart after unmounting the loaded_modules file in ConfigMap

e4.4.6

Release Date: 2022-07-29

Enhancement

  • Rules engine supports RocketMQ with ACL enabled
  • Supports searching and paging of rules in rule engine
  • Kafka in Rules Engine now supports SASL/SCRAM authentication and SASL/GSSAPI authentication. Note that the cyrus-sasl-gssapi dependency needs to be installed before using SASL/GSSAPI authentication
  • Provides CLI ./bin/emqx check_conf to actively check if the configuration is correct
  • Optimize the write performance of TDEngine in the rule engine
  • Support for clearing historical alarms on Dashboard
  • Optimizing Shared Subscription Performance
  • Add db_name field to the action of rule engine writing data to TDEngine to improve support for super table

Bug fixes

  • Fix the issue that the action count is wrong when the rule engine writes to TDEngine
  • Fix the issue that the process pool size setting does not take effect when the rule engine writes to HStreamDB
  • Fix the issue that an error was reported when querying the subscription list after the GB/T 32960 plugin was enabled
  • Fix the issue that incompatible configuration items when restoring a 4.2 backup to 4.4 with hot configuration enabled
  • Fix the issue that once the old version of EMQX is uninstalled after hot upgrade, EMQX will not be able to start again
  • Fix the issue that the keep-alive check for UDP clients in the Multilingual Protocol Extension was incorrect, causing clients not to expire
  • Fix the issue that the client information in the Multilingual Protocol Extension was not updated in time
  • Fix the issue that the license update does not take effect when running after hot upgrade to e4.4.4 and later versions
  • Fix the issue that when the client specified Clean Session as false to reconnect, the shared subscription message in the flight window would be re-dispatched to the old session process
  • Fix the issue that the new node did not use the cluster license after joining the cluster
  • Fix the issue that the emqx_lua_hook plugin cannot cancel the message publishing

e4.4.5

Release Date: 2022-06-30

Enhancement

  • Rule engine supports persisting data to HStreamDB
  • QoS and Retain flag in rule engine’s message republish actions can now use placeholders
  • Supports exclusive subscriptions, that is, only one subscriber is allowed for a topic
  • Support one-click update of cluster license through CLI
  • Dashboard and management API’s HTTPS listeners can now use password-protected private key files, providing key_password configuration item
  • Support for placeholders %u and %c in topic rewrite rules
  • Support setting MQTT 5.0 properties in the API request for message publishing, such as message expiry interval, response topic, etc.
  • Optimize the UI when creating rule engine resources, such as folding some uncommon options, etc.
  • Opened 4 TCP-related configuration items: KeepAlive, TCP_NODELAY, SO_RCVBUF and SO_SNDBUF for the underlying gRPC connection of ExHook

Bug fixes

  • Fix the issue of inaccurate memory calculation in Linux OS, and calculate the memory usage of the current OS instead of the memory usage of EMQX
  • Fix the issue that the old disconnect event of ExHook would be triggered later than the new connect event when the client reconnects
  • Improve the JWT authentication module’s judgment logic for startup status in a cluster environment
  • Extend the timeout of DynamoDB resource status query of rule engine to avoid the problem that some overseas resources may not be available
  • Fix the issue that the timestamp is not updated when the PostgreSQL data storage plugin persists messages and updates message consumption
  • Fix the issue that the rules engine’s Tablestore, Lindorm and InfluxDB resources did not check connection status when creating
  • Fix rule engine not checking connection status when creating Tablestore, Lindorm and InfluxDB resources
  • Fix rule engine not updating metrics correctly when writing to TDEngine resource fails
  • Fix the issue that the Cassandra message store plugin persisted retained messages to multiple tables, causing duplicate messages
  • Fix the issue that Kafka resources exported from versions prior to 4.3.0 (only when the Produce policy is set to first_key_dispatch) could not be imported into 4.3.0 and later
  • Fix the issue that the execution order of topic rewriting and delayed publish is not fixed, now it is fixed to execute topic rewriting first
  • Improve the null value handling when rule engine persists data to InfluxDB and Tablestore, now null values will not be written
  • Fix the issue that rule engine could not encode MQTT 5.0 user properties
  • Fix the issue that the count of connack.auth_error is inaccurate when the client uses a protocol version below MQTT v5.0 to access
  • Fix the issue that the UDP listeners of LwM2M and CoAP gateways could not bind to the specified network interface
  • Fix Dashboard not starting after removing the default Dashboard user in the configuration file
  • Fix client.subscribe hook not being able to reject subscriptions
  • If the placeholder in the ACL rule is not replaced, the client’s publish or subscribe operation will be rejected
  • Fix the issue that TLS was enabled but no TLS connection was actually established with Pulsar

e4.4.4

Release Date: 2022-06-01

Enhancement

  • Add more time transformation functions to the SQL of rule engine
  • Add the float2str/2 function to the SQL of rule engine to support specifying the output precision of floating point numbers
  • Rule engine supports message persistence to Alibaba TableStore
  • Rule engine supports connecting to Pulsar using Basic and JWT authentication
  • Add service_name option to Oracle resource of rule engine to support Oracle Database RAC
  • Support for using JWT for authorization, now MQTT clients can authorize using specific claims that include a pub-sub whitelist
  • Improved authentication related metrics to make it easier to understand, now client.authenticate = client.auth.success + client.auth.failure
  • Support binding the listener of the REST API to a specified network interface
  • Upload license will be automatically synchronized to the entire cluster, no need for each node to upload separately, provide HTTP API
  • Support multi-condition query and fuzzy query for user data in authentication and authorization using built-in database as data source
  • Supports querying clients using the length of the message queue and the number of dropped messages as conditions
  • Support to configure the log time format to be compatible with the time format in older versions
  • When use_username_as_clientid is configured to true and the client connects without specifying a username, the connection is now rejected with a reason code 0x85
  • Full randomisation of app secrets (previously partially randomised)
  • When using CLI for backup and recovery, it is no longer required that the backup file must be located in the backup folder of the EMQX data directory
  • Hot upgrades between incompatible versions will now be rejected
  • Allow white spaces in EMQX’s installation path
  • Boot script fail fast on invalid node name (improve error message readability)

Bug fixes

  • Fix the issue that the client could not get the message after going online when using the PostgreSQL offline message plugin
  • Fix the issue that the rules engine could not successfully establish a TLS connection with Pulsar in some cases
  • Fix the issue that rule engine’s SQL function hexstr_to_bin/1 could not handle half-byte
  • Fix the issue that the alarm was not cleared when the rule engine resource was deleted
  • Fix Dashboard HTTPS listener’s verify option not taking effect
  • Fix the issue that messages were lost when the peer session was terminated during the delivery of QoS 1 messages through shared subscriptions
  • Fix the issue that when the log tracer encounters large packets, the heap size grows too fast and triggers the policy of forcibly closeing the connection process
  • Fix the issue that the relevant hooks were not properly uninstalled when the module was disabled, resulting in abnormal functions
  • Fix the issue that the MQTT-SN client would be disconnected when retransmitting QoS 2 messages
  • Fix the issue that modules that were turned off in the backup file would be automatically enabled after restoring the backup
  • Fix the issue that the subscriber’s connection was disconnected due to the wrong user properties type in the message publishing API api/v4/mqtt/publish
  • Fix DynamoDB driver not adapting to OTP 24, causing it to be unavailable
  • Fix the issue that some authentication algorithms were unavailable due to the PostgreSQL driver not adapting to OTP 24
  • Fix the issue that the returned results did not match the query conditions when querying subscriptions with multiple conditions
  • Fix rule engine resource connection test not working
  • Fix multiple Dashboard display issues

e4.4.3

Release Date: 2022-04-18

Enhancement

  • Schema registry now supports decoding arbitrary binary payloads to JSON data using gRPC services
  • Support for connecting to Pulsar using TLS
  • Add mongo_date function for SQL in rule engine, which supports saving timestamps as MongoDB Date objects
  • Rule engine supports resetting metrics of the specified rule
  • Add connection confirmation and authorization completion events to the rule engine
  • Rule engine supports copying rule for fast reuse
  • SQL in rule engine supports zip, gzip and other compression and decompression functions
  • Improve the error message when rule engine fails to parse payload
  • Improve the connection test for some resources in rule engine
  • Support setting execution priority for ExHook
  • ExHook callback interface adds a Protobuf field RequestMeta meta to return the EMQX cluster name
  • Support local policy for shared subscriptions, which will preferentially send messages to shared subscribers under the node where messages flow in. In some scenarios, the efficiency of shared message scheduling will be improved, especially when the MQTT bridge is configured as a shared subscription
  • RSA-PSK-AES256-GCM-SHA384, RSA-PSK-AES256-CBC-SHA384, RSA-PSK-AES128-GCM-SHA256 and RSA-PSK-AES128-CBC- SHA256 four new TLS PSK cipher suites are supported, removing two insecure cipher suites PSK-3DES-EDE-CBC-SHA and PSK-RC4-SHA from the default configuration
  • Diagnostic logging for wait_for_table of mnesia
    • Prints check points of mnesia internal stats
    • Prints check points of per table loading stats, help to locate the problem of long table loading time.
  • Subscribing to an empty topic is prohibited in strict mode
  • Generate default files when loaded_modules and loaded_plugins files do not exist

Bug fixes

  • Fix the issue that the TLS configuration item server_name_indication is set to disable and does not take effect
  • Fix potential process leak issue in MongoDB driver
  • Fix the issue that the password of the default Dashboard user modified via the CLI command would be reset after the node leaves the cluster
  • Silence grep and sed warnings in docker-entrypoint.sh
  • Fix the backup file cannot be deleted and downloaded when the API path contains ISO8859-1 escape characters
  • Fix the issue that the Redis driver would crash when DNS resolution failed, etc
  • Fix the issue that the MQTT Bridge plugin cannot be started when only the subscription topic is configured but QoS is not configured
  • When creating a rule, if a rule with the same ID already exists, the rules engine will now report an error instead of replacing the existing rule
  • Fix the issue that the HTTP driver process pool may not be deleted
  • Fix the issue that the module parameters could not be updated again after failing to update
  • Fix the incorrect type of some fields in the GB/T 32960 access gateway module in Dashboard
  • Fix the issue that the configuration of Bridge resources such as Kafka and Pulsar could not be updated
  • Fix the issue that JT/T 808 client authentication fails when anonymous authentication is enabled

e4.4.2

Release Date: 2022-04-01

Important changes

  • For Docker images, the configuration directory /opt/emqx/etc has been removed from the VOLUME list, making it easier for users to rebuild images with changed configurations.
  • CentOS 7 Erlang runtime rebuilt on OpenSSL-1.1.1n (previously 1.0), prior to v4.3.13, EMQX will fail to handshake and trigger malformed_handshake_data exception when clients use certain cipher suites.
  • CentOS 8 Erlang runtime system rebuilt on RockyLinux 8. centos8 will remain in the package name for backward compatibility.

Enhancement

  • Add Pulsar proxy support for rule engine bridging data to Pulsar.
  • Add OOM protection for Kafka producers.
  • Add command line interface emqx_ctl pem_cache clean to allow forcibly clear x509 certificate cache to reload immediately after certificate file update.
  • Refactored ExProto so that anonymous clients can also be displayed on Dashboard.
  • Topic configuration items in bridges can now use ${node} placeholders.
  • Add validation of UTF-8 strings in MQTT packets in strict mode. When set to true, invalid UTF-8 strings will cause the client to disconnect.
  • MQTT-SN gateway supports initiative to synchronize registered topics after session resumed.
  • Improve the writing precision of rule engine floating point data from 10 decimal places to 17 decimal places.
  • EMQX will prompt how to modify the initial password of Dashboard at startup.

Bug fixes

  • Fix MQTT Subscriber module not being able to use two-way SSL connection.
  • Fix the issue that PSKFile module failed to start.
  • Fix the issue that Kafka Consumer Group module could not process binary data.
  • Fix the issue that Log Trace could not be stopped.
  • Fix the issue that the alternate action could not be triggered when the action of the rule engine persisting data to Oracle and Lindorm (only synchronous operation) failed to execute.
  • Fix an issue where rule engine data persistence to Oracle failed but the success count still increased.
  • Fix the issue that some zone configurations could not be cleared.
  • Fix an issue where changes to some monitoring and alarm configurations were invalid after restarting.
  • Fix the issue that Schema Registry is not available in the cluster environment.
  • Fix the issue that the LwM2M client list query API returned incorrect data in a cluster environment, which resulted in the inability to access the LwM2M gateway module management page.
  • Fix the issue that the JT/T 808 location report frame was parsed incorrectly.
  • Fix the issue that the el8 installation package cannot be started on Amazon Linux 2022, the error content is errno=13 Permission denied.
  • Fix an issue where the client could not reconnect if the connection process was blocked in some cases. Now waiting for more than 15 seconds without a response will force the old connection process to be closed.
  • Fix the issue of query resource request timeout when rule engine resource is unavailable.
  • Fix the issue of {error, eexist} error when re-run after hot upgrade failed.
  • Fix an issue where publishing to a non-existing topic alias would crash the connection.
  • Fix 500 error when querying lwm2m client list on another node via HTTP API.
  • Fix HTTP API for subscribing topics crashes when invalid QoS are passed in.
  • Fix the issue that the connection count was not updated because the related resources were not released when the connection process accessed through the ExProto exited abnormally.
  • Fix an issue where the value of server_keepalive configuration item would be incorrectly applied to MQTT v3.1.1 clients.
  • Fix Stomp client not firing $event/client_connection event messages.
  • Fix the issue that the system memory alarm was incorrectly activated when EMQX was started.
  • Fixed an issue where messages that failed to be delivered due to unregistered topics were not retransmitted when topics were successfully registered with the MQTT-SN client.
  • Fix EMQX startup output error log when duplicate plugins are configured in loaded_plugins file.
  • Fix MongoDB related features outputting excessive error logs when configured incorrectly.
  • Add format check for Dashboard User and AppID, special characters such as / are not allowed.
  • Corrected the reason code in the DISCONNECT packet returned when kicking the client to 0x98.
  • Auto subscriptions will ignore empty topics.

e4.4.1

Release Date: 2022-02-18

NOTE: 4.4.1 is in sync with: 4.3.7. The compare base of this change set is 4.4.0.

Important changes

  • A cluster-wide total connections calculation bug was fixed in in Enterprise edition 4.4.1. Previously only the individual node’s local number of connections were checked against the max number of connections allowed by the license. After this fix, the total number of connections is aggregated cluster-wide every 5 seconds. An extra 10% overrun is allowed to compensate the delays in aggregation.
  • The slow subscription feature is improved. It supports counting the time spent in the process of message transmission, and recording and displaying time-consuming clients and topics. Users planning to upgrade should be aware of the possibility that this change may cause clients to reach the license limit and not be able to connect.
  • Rules engine supports Lindorm database
  • Support client-level message drop metrics
  • Optimize online Trace log display on Dashboard, support syntax highlighting

Minor changes

  • Support alarm about the usage rate of license connections. By default, the number of connections reaches 80% of the allowed number of licenses, and the alarm is raised. When it is less than 75%, the alarm is cleared. User can also customize in emqx.conf: license.connection_high_watermark_alarm , license.connection_low_watermark_alarm
  • Support alarm about license expiration, when the validity period is less than 30 days, the alarm will be raised
  • Rule engine supports the configuration of rules and actions for the event of abnormal loss of client messages to enhance the user’s custom processing capabilities in this scenario
  • Improve the relevant metrics during the execution of the rule engine SQL matching
  • Fuzzy search on client supports special characters such as *, (, )
  • Improve ACL-related metrics to solve the issue that the count does not increase due to hitting the ACL cache
  • Added connected_at field to webhook event notifications
  • Log client state before terminating client due to holding the lock too long

Bug fixes

  • Fix the issue that data import and export were not available in some cases
  • The module update mechanism is improved to solve the issue that the module is unavailable after the update fails
  • Fix the issue that the rule engine did not perform type checking when executing the size comparison statement
  • Fix the issue that the related counts are cleared after updating the rule engine action
  • Fixed the issue that the metrics interface does not return authentication metrics such as client.acl.deny by default
  • Fixed the issue that the subscription query interface did not return paginated data
  • Fix the issue of parsing failure when STOMP handles TCP sticky packets
  • Fix the issue where the session creation time option was not available when filtering clients
  • Fix the issue where memory alarms might not be triggered after restarting
  • Fix the crash of import data when user data exists in emqx_auth_mnesia plugin

e4.4.0

Release Date: 2021-12-21

EMQX Enterprise 4.4.0 mainly includes the following changes:

Important changes

  • Starting from 4.4, EMQX releases are named with Erlang/OTP release in the package name. e.g. emqx-ee-4.4.0-otp24.1.5-3-centos7-arm64.rpm

  • For Debian/Ubuntu users, Debian/Ubuntu package (deb) installed EMQX now now run on systemd. This is to use systemd’s supervision functionality to ensure that EMQX service restarts after a crash. The package installation service upgrade from init.d to systemd has been verified, but it is still recommended that you verify and confirm again before deploying to the production environment, at least to ensure that systemd is available in your system

  • Rule engine InfluxDB integration adds support for InfluxDB v2 API, rule engine supports InfluxDB 2.0 and InfluxDB Cloud now

  • Rule engine adds support for SAP Event Mesh

  • Rule engine adds support for MatrixDB

  • MongoDB integration supports DNS SRV and TXT Records resolution, which can seamlessly connect with MongoDB Altas

  • Supports trace online, users can complete the tracking operation of the client and topic on the Dashboard, and view or download the trace log

  • Supports slow subscription statistics, which can be used to find abnormal situations such as message blockage in the production environment in time

  • Support dynamic modification of MQTT Keep Alive to adapt to different energy consumption strategies

  • Support 4.3 to 4.4 rolling upgrade of clustered nodes. See upgrade document for more dtails.

  • TLS for cluster backplane (RPC) connections. See clustering document for details.

Minor changes

  • Dashboard supports viewing the number of active client connections

  • Dashboard supports relative paths and custom access paths

  • Dashboard remove tab navigation

  • Support configuring whether to write integer data to InfluxDB as floating point type

  • Supports configuring whether to forward retained messages with empty payload to suit users who are still using MQTT v3.1. The relevant configurable item is retainer.stop_publish_clear_msg

  • Multi-language hook extension (ExHook) supports dynamic cancellation of subsequent forwarding of client messages

  • Rule engine SQL supports the use of single quotes in FROM clause, for example: SELECT * FROM't/#'

  • Optimize the use and interaction of the built-in access control file module

  • Change the default value of the max_topic_levels configurable item to 128. Previously, it had no limit (configured to 0), which may be a potential DoS threat

  • Improve the error log content when the Proxy Protocol message is received but the proxy_protocol configuration is not turned on

  • Add additional message attributes to the message reported by the gateway. Messages from gateways such as CoAP, LwM2M, Stomp, ExProto, etc., when converted to EMQX messages, add fields such as protocol name, protocol version, user name, client IP, etc., which can be used for multi-language hook extension (ExHook)

  • HTTP client performance improvement

  • Add openssl-1.1 to RPM dependency

Bug fixes

  • Fix the issue that the client process becomes unresponsive due to the blockage of RPC calls between nodes

  • Fix the issue that the lock management process ekka_locker crashes after killing the suspended lock owner

  • Fix the issue of garbled data when writing data to RocketMQ asynchronously

  • Fix the issue of inaccurate statistics of RocketMQ

  • Fix the display error of the Dashboard monitoring page when the number of cluster nodes exceeds seven

  • Fix the issue that the rule engine may have a higher failure rate when saving data to MySQL

  • Fix the issue that the Clickhouse-based offline messaging feature of the rule engine is unavailable

  • Fix the issue that the Max Returned Count option in the MongoDB-based offline message feature of the rule engine cannot be used

  • Fix the issue that the Path parameter of WebHook action in rule engine cannot use the rule engine variable

  • Fix MongoDB authentication module cannot use Replica Set mode and other issues

  • Fix the issue of out-of-sequence message forwarding between clusters. The relevant configurable item is rpc.tcp_client_num

  • Fix the issue of incorrect calculation of memory usage

  • Fix MQTT bridge malfunction when remote host is unreachable (hangs the connection)

  • Fix the issue that HTTP headers may be duplicated