Quick start#

Installation#

See Installation overview.

Connect to the MQTT Broker and Otel Server#

The mapping between MQTT and Otel is configured via a file called Manifest.yaml. Here’s an example of a simple configuration file that connects to an MQTT broker at http://mymqtt-broker.net:32007 and an OpenTelemetry collector at http://my-otel-collector.net:32014:

 1Version: 1.0
 2
 3MqttBroker:
 4  - Name: "My broker"
 5    Endpoint:
 6      Port: 32007
 7      Address: "mymqtt-broker.net"
 8      EnableTls: false
 9
10OtelServer:
11  - Name: "My Otel server"
12    ServiceName: "my-service"
13    ServiceNamespace: "my-service-namespace"
14    Endpoint:
15      Protocol: "http"
16      Port: 32014
17      Address: "my-otel-collector.net"
18      EnableTls: false

This assumes no credentials are required to log into the broker or the Otel collector. For further configuration options, see Configure MQTT Broker and Configure Otel Server.

Breakdown of the configuration#

• MQTT Broker:

  • Name: An identifier for the MQTT broker.
  • Endpoint: Describes the broker’s address and port.

• Otel Server:

  • Name: An identifier for the Otel server.
  • ServiceName: The name of the service.
  • ServiceNamespace: The namespace for the service.
  • Endpoint: Describes the Otel collector’s address and port.

Subscribe to a Topic and Generate a Metric#

Now that we have connected to the MQTT broker and Otel server, let’s subscribe to an MQTT topic and generate an Otel metric from the payload.

Suppose the server sends messages to the topic message-topic in the following JSON format:

1{
2    "Processor": 
3    {
4        "Temperature": 42.5
5    },
6    "TempUnit": "C"
7}

To extract the temperature, we will use the JSONPath syntax $.Processor.Temperature. The corresponding YAML would look like this:

 1Processors:
 2  - Name: "Processor Temperature"
 3    Description: "Provides the current processor temperature."
 4    Mqtt: 
 5      Subscriptions:
 6        - Name: "Processor information"
 7          Topic: "message-topic"
 8    Otel:
 9      Metrics:
10        - Name: "Processor.Temperature"
11          Description: "The current processor temperature."
12          SignalDataType: Float
13          Instrument: Gauge
14          Value: "JSONPATH('$.Processor.Temperature')"

This configuration subscribes to the MQTT topic message-topic and creates an Otel metric called Processor.Temperature with a float data type and a Gauge instrument. Every time a message is received for the message-topic, the temperature value parsed from the provided json and sent to the Otel endpoint.

The syntax is as following:

• Processors contains a list of processors. A processor is able to receive mqtt messages, process them and send them to the configured otel endpoint.
• The Processor consists of two parts:
  • Mqtt
    • A list of mqtt topic subscriptions, consisting of
      • A name
      • The topic to which they subscribe.
  • Otel
    • A list of metrics, that should be generated from the message payload. It consists of
      • Name and description
      • The data type of the signal that will be send to the otel endpoint
      • The otel instrument
      • The value of the metric that will be send to the otel endpoint.

Variables and Attributes#

Subscriptions can have variables, which can be used later in the rules section. Here’s an example of how to define variables:

1Mqtt:
2  Subscriptions:
3    - Name: "Processor information"
4      Topic: "metric/sensor_1234"
5      Variables:
6        - Key: "SensorName"
7          Value: "ProcessorServerA"

You can access variables in Otel rules by prefixing them with a $ sign. For example, to access the SensorName, you would use $SensorName.

Otel rules can also include attributes, which are added to the Otel signal for filtering or grouping. You can use variables inside attributes where needed. Here’s an example of how to add attributes:

 1Otel:
 2  Attributes:
 3    - Key: SensorName
 4      Value: $SensorName
 5    - Key: Location
 6      Value: "Main server room"
 7  Metrics:
 8    - Name: "Processor.Temperature"
 9      Description: "The current processor temperature."
10      Attributes:
11        - Key: MeasurementQuality
12          Value: 10
13      SignalDataType: Float
14      Instrument: Gauge
15      Value: "JSONPATH('$.Processor.Temperature')"

The attributes directly added under the Metrics section will be added to all metrics. The attributes added to the “Processor.Temperature” metric will only be added to this metric.

Resulting Signal Attributes:#

Working with Expressions#

We’ve already used an expression to parse the payload with JSONPATH('$.Processor.Temperature'). However, you can also perform mathematical transformations. For example, to convert the temperature from Celsius to Fahrenheit, you can use this expression:

1Value: "(JSONPATH('$.Processor.Temperature') * 1.8) + 32.0"

Standard mathematical operations like +, -, *, /, and functions such as SQRT, Sin, Cos, Tan, and constants like [Pi] are supported.

Available Functions#

For more details, please refer to the documentation.

Log Messages and Transformation#

Log messages work similarly to metrics. Let’s say you receive a log message payload in the following format from MQTT:

2026-02-26T10:28:34Z [Info] [ServerA] Temperature value read successfully.

Rather than sending the raw message to Otel, we can transform it into a structured log format using a GROK expression:

%{TIMESTAMP_ISO8601:otel_timestamp} \[%{WORD:otel_loglevel}\] \[%{WORD:server_name}\] %{GREEDYDATA:otel_message}

This can be read as:

• Parse an ISO8061 timestamp and name it otel_timestamp
• Read a space and a [ (needs to be escaped as [) adn discard the information
• Read a word and name it otel_loglevel
• Read ] [ and discard the information
• Read a word and name it server_name
• Read ] [ and discard the information
• Read the remaining part of the message and name it otel_message

This expression can then be used in a Transform expression inside the Logs section:

 1Processors:
 2  - Name: "Server logs"
 3    Description: "Collect all log messages from the server."
 4    Mqtt:
 5      Subscriptions:
 6        - Name: "Server logs"
 7          Topic: "message-log-topic"
 8    Otel:
 9      Attributes:
10        - Key: Location
11          Value: MainServerRoom
12      Logs:
13        - Name: "Logging"
14          PayloadType: Json
15          Transform: "GROK('%{TIMESTAMP_ISO8601:otel_timestamp} \[%{WORD:otel_loglevel}\] \[%{WORD:server_name}\] %{GREEDYDATA:otel_message}')"

You should notice, that we are using the Logs keyword now in the otel section to identify log messages. The Transform expression will convert the log into a JSON structure like this:

1{
2  "otel_timestamp": "2026-02-26T10:28:34Z",
3  "otel_loglevel": "Info",
4  "server_name": "ServerA",
5  "otel_message": "Temperature value read successfully."
6}

Since we’ve specified PayloadType: Json, Otel will interpret the top-level keys (otel_timestamp, otel_loglevel, server_name, and otel_message) as attributes in the log message. Attributes starting with “otel_” will have a special meaning so they are interpreted not as attributes but as the message body, timestamp and log level.

Subscription Groups#

To avoid repetition and reuse the same subscriptions across different metrics or logs, you can group them into Subscription Groups and refer to them later. This is useful when you have e.g. multiple devices or sensors sending data under the same topic structure but need to handle them differently in your rules.

Example Scenario#

Let’s say you have a device that sends both power consumption metrics (like current, power, voltage) and status information (like the microcontroller core temperature) in the same MQTT message. The message payload is structured as follows:

 1{
 2  "Time": "2026-04-12T09:07:04",
 3  "ENERGY": {
 4    "Power": 0.000,
 5    "Voltage": 227,
 6    "Current": 0.000
 7  },
 8  "ESP32": {
 9    "Temperature": 37.4
10  }
11}

You want to treat power metrics separately from the microcontroller status. To achieve this, you can group the subscriptions into a SubscriptionGroup for reuse:

Defining a Subscription Group#

 1SubscriptionGroups:
 2  - Name: "Power sensors"
 3    Variables:
 4      - Key: DeviceName
 5        Value: "My power sensor"
 6    Subscriptions:
 7      - Name: "Power Sensor washing machine"
 8        Topic: "sensor_1234"
 9        Variables:
10          - Key: "SensorName"
11            Value: "WashingMachine"
12      - Name: "Power Sensor dryer"
13        Topic: "sensor_9876"
14        Variables:
15          - Key: "SensorName"
16            Value: "Dryer"

Here, we define a Subscription Group called Power sensors, which includes two subscriptions: one for a washing machine and another for a dryer. Both subscriptions have associated variables that can be used later in the metrics or logs.

Using Subscription Groups in Metrics and Logs#

Once you’ve created the Power sensors group, you can refer to it in your metrics or logs as follows:

 1Processor:
 2  - Name: "Power Metrics"
 3    Description: "Provides power information from a power sensor."
 4    Mqtt: 
 5      SubscriptionGroups:
 6        - Name: "Power sensors"
 7    Otel:
 8      Attributes:
 9        - Key: Type
10          Value: "Power metrics"
11      Metrics:
12        - Name: "Energy_Power_W"
13          Description: "The current power consumption at the time of measurement in Watt."
14          SignalDataType: Float
15          Instrument: Gauge
16          Value: "JSONPATH('$.ENERGY.Power')"
17        - ...
18
19  - Name: "Processor Status"
20    Description: "Provides processor status from the power sensor."
21    Mqtt: 
22      SubscriptionGroups:
23        - Name: "Power sensors"
24    Otel:
25      Attributes:
26        - Key: Type
27          Value: "Processor metrics"
28      Metrics:
29        - Name: "ESP32_Temperature"
30          Description: "The current temperature of the ESP32 microcontroller."
31          SignalDataType: Float
32          Instrument: Gauge
33          Value: "JSONPATH('$.ESP32.Temperature')"
34        - ...

In this example:

• Power Metrics: We create a metric for power data (e.g., Power, Voltage), subscribing to the Power sensors group.
• Processor Status: We create another metric for processor data (e.g., Temperature), also subscribing to the same Power sensors group.

Both metrics will use different attributes.

Grouping Devices with Different Topics#

Sometimes, devices may use different MQTT topics but contain the same identifier. For example, you might have multiple topics for each sensor:

To manage these different topics, you can group them under a common Subscription Group and then specify ParentPath and SubPath to correctly target the topics.

Defining Subscription Groups with ParentPath and SubPath#

 1SubscriptionGroups:
 2  - Name: "Power sensors"
 3    Subscriptions:
 4      - Name: "Power Sensor 1"
 5        Topic: "1234"
 6      - Name: "Power Sensor 2"
 7        Topic: "9876"
 8
 9Processors:
10  - Name: "Power Metrics"
11    Description: "Provides power information from a power sensor."
12    Mqtt: 
13      SubscriptionGroups:
14        - Name: "Power sensors"
15          ParentPath: "tele"
16          SubPath: "sensor"
17    Otel:
18      Metrics:
19        - Name: "Energy_Power_W"
20          Description: "The current power consumption at the time of measurement in Watt."
21          SignalDataType: Float
22          Instrument: Gauge
23          Value: "JSONPATH('$.ENERGY.Power')"
24        - ...
25
26  - Name: "Sensor Logs"
27    Description: "Collect all log messages from the sensors."
28    Mqtt:
29      SubscriptionGroups:
30        - Name: "Power sensors"
31          ParentPath: "stat"
32          SubPath: "logs"
33    Otel:
34      Logs:
35        - Name: "Logging"
36          PayloadType: Json
37          Transform: "GROK('%{TIME:otel_timestamp} %{WORD:category}: %{GREEDYDATA:otel_message}')"

Explanation:#

• ParentPath: This specifies the top-level directory or prefix of the topic. For example, tele for telemetry data or stat for status/log data.
• SubPath: This specifies the specific subtopic or suffix that targets a specific part of the topic.

Example with the above configuration:#

• The Power Metrics rule will subscribe to topics like tele_1234_sensor and tele_9876_sensor using the ParentPath tele and SubPath sensor.
• The Sensor Logs rule will subscribe to topics like stat_1234_logs and stat_9876_logs using the ParentPath stat and SubPath logs.

Final Thoughts#

By using Subscription Groups, you can easily reuse configurations across different rules, making your setup more modular and scalable. Grouping devices and topics this way allows you to handle complex MQTT topic structures efficiently.

Complete example manifest#

Here is a complete minimal example manifest using logs, and metrics:

 1Version: 1.0
 2
 3MqttBroker:
 4  - Name: "My broker"
 5    Endpoint:
 6      Port: 32007
 7      Address: "mymqtt-broker.net"
 8      EnableTls: false
 9
10OtelServer:
11  - Name: "My Otel server"
12    ServiceName: "my-service"
13    ServiceNamespace: "my-service-namespace"
14    Endpoint:
15      Protocol: "http"
16      Port: 32014
17      Address: "my-otel-collector.net"
18      EnableTls: false
19
20Processors:
21  - Name: "Processor Temperature"
22    Description: "Provides the current processor temperature."
23    Mqtt: 
24      Subscriptions:
25        - Name: "Processor information"
26          Topic: "message-topic"
27    Otel:
28      Metrics:
29        - Name: "Processor.Temperature"
30          Description: "The current processor temperature."
31          SignalDataType: Float
32          Instrument: Gauge
33          Value: "JSONPATH('$.Processor.Temperature')"
34
35  - Name: "Server logs"
36    Description: "Collect all log messages from the server."
37    Mqtt:
38      Subscriptions:
39        - Name: "Server logs"
40          Topic: "message-log-topic"
41    Otel:
42      Attributes:
43        - Key: Location
44          Value: MainServerRoom
45      Logs:
46        - Name: "Logging"
47          PayloadType: Json
48          Transform: "GROK('%{TIME:otel_timestamp} [%{WORD:otel_loglevel}] [%{WORD:server_name}] %{GREEDYDATA:otel_message}')"