Expressions and transformations#
Expressions and transformations are the tools with which message data (payload, properties, or topics), received from mqtt can be processed. The difference between the two is:
- Expressions are used to extract or calculate a value from a payload
- Transformations are used to transform a payload into a different format, e.g. from text to json.
Parameters supporting expressions or transformation can be identified by the tag expressions or transformations.
In fields that supports text input, there is a special syntax for embedding expressions inside a text field. The expression must then be wrapped
inside a $(expression). So for example to include the payload inside a text, you can use This is the payload: $(Payload()).. Parameters supporting
embedded expressions can be identified by the tag:
embedded expressions.
All parameters that support embedded expressions support variables as well.
Expressions#
The basics#
Expressions are mainly used at the Value parameter inside a Processors.Otel.Metrics.Metric.Value are build on top of the
NCalc library where you can find additional information on capabilities not covered in this document.
A simple expression returning the constant value 42 would be:
| |
Functions#
Most of the time you want to process a payload as delivered by a mqtt subscription. Lets take the following example json payload:
| |
To access the temperature we use the JSONPATH syntax: $.Processor.Temperature that gets the Temperature
parameter inside the Processor parameter. To do this we have to use a function called JSONPATH:
| |
That will return the value 42.5. The data type returned will be the data type defined in Processors.Otel.Metrics.Metric.SignalDataType. If you want to
change the data type to e.g. int you can add another parameter to the function stating the data type:
| |
This will return the value 42.
Important
The name of functions is case-insensitive.
Available Functions#
Mqtt message parsing
| Function | Example | Description |
|---|---|---|
JsonPath | JsonPath('$.Root') | Extracts data using JSONPATH syntax from a json payload. |
XPath | XPath('/root/child[1]') | Extracts data using XPath syntax from a xml payload |
TopicPath | TopicPath('[1]') | Extracts data using TopicPath syntax from the message topic. |
UserProperty | UserProperty('Name') | Accesses an mqtt user property via its name. If the name exists multiple times, the first match is used. If the name does not exist, the function returns an empty string. |
RegEx | RegEx('[0-9]+') | Extracts data from payload using a regular expression. If the regular expression returns more than one match, then the first match is used. |
Var | Var('MyVariable') | Returns the variable with the given name. No $ is needed before the variable name. |
Payload | Payload() | Returns the raw payload |
String functions
| Function | Example | Description |
|---|---|---|
ToLower | ToLower('My Signal') => my signal | Returns lower case value |
ToUpper | ToUpper('My Signal') => MY SIGNAL | Returns upper case value |
ToPascalCase | ToPascalCase('My Signal') => MySignal | Returns pascal case value |
ToCamelCase | ToCamelCase('My Signal') => mySignal | Returns camel case value |
ToSnakeCase | ToSnakeCase('My Signal') => my_signal | Returns snake case value |
ToKebabCase | ToKebabCase('My Signal') => my-signal | Returns kebab or hyphen case value |
ToTrainCase | ToTrainCase('My Signal') => My-Signal | Returns train case value |
Trim | Trim(' Test ') => “Test” | Removes leading and trailing whitespace |
TrimStart | TrimStart(' Test ') => “Test " | Removes leading whitespace |
TrimEnd | TrimEnd(' Test ') => " Test” | Remove trailing whitespace |
StartsWith | StartsWith('MyValue', 'My') => true | Tests, whether a string starts with the provided pattern. |
EndsWith | StartsWith('MyValue', 'Value') => true | Tests, whether a string ends with the provided pattern. |
Contains | Contains('MyValue', 'Val') => true | Tests, whether a string contains the provided pattern. |
Replace | Replace('Test', 'e', 'ee') => Teest | Replaces strings inside a string. |
MatchesWildcard | MatchesWildcard('My Signal', '*Signal') => true | Tests, whether a string matches a wildcard pattern. |
MatchesRegEx | MatchesRegEx('My Signal', '.*') => true | Tests, whether a string matches a regular expression. |
Map | Map('value', 'lookupTable') => translated value | Maps the provided value to another value, as defined in the lookup table. For details, see Mappings. |
Type conversion
| Function | Example | Description |
|---|---|---|
ToInt | ToInt(42.1) => 42 | Converts a value to integer |
ToLong | ToLong(42.1) => 42L | Converts a value to a long integer |
ToFloat | ToInt(42) => 42.0 | Converts a value to float |
ToDouble | ToInt(42) => 42.00 | Converts a value to double |
ToString | ToInt(42) => “42” | Converts a value to string |
Constants#
Some constants are always available when evaluating an expression. They are enclosed in [], e.g. [Pi].
Important
The name of constants is case-insensitive.
The following constants are allways available:
| Constant | Example | Description |
|---|---|---|
| Pi | [Pi] => 3.14… | The mathematical constant pi |
| e | [e] => 2.718… | the euler number |
In some contexts additional constants may be available, e.g. in a NameFormatter or in a ValueConverter.
Calculations#
We’ve already used an expression to parse the payload with JsonPath('$.Processor.Temperature'). However, you can also perform
mathematical calculations. For example, to convert the temperature from Celsius to Fahrenheit, you can use this expression:
| |
Standard mathematical operations like +, -, *, /, and functions such as Sqrt, Sin, Cos, Tan, and constants
like [PI] and [E] are also supported. Further details can be found at the NCalc library.
Transformations#
The basics#
Transformations work similar than expressions, but instead of extracting the needed value they transform a payload in another form for further processing.
Let’s say you receive a log message payload in the following format from MQTT:
2026-02-26T10:28:34Z [Info] [ServerA] Temperature value read successfully.Rather than sending the raw message to Otel, we can transform it into a structured log format using an extended DISSECT expression:
%{otel_timestamp:DateTime} [%{otel_loglevel}] [{server_name}] %{otel_message}This can be read as:
- Parse date and time and name it otel_timestamp
- Read a space and a [ and discard the information
- Read everything up until ] and name it otel_loglevel
- Read ] [ and discard the information
- Read everything up until ] and name it server_name
- Read ] [ and discard the information
- Read the remaining part of the message and name it otel_message
With that the payload will be transformed in a log message that looks like this:
| |
This message can than be passed to the log processor. Be careful to set PayloadType: Json for getting the expected results.
The usage is similar to expressions:
Available Functions#
| Function | Example | Description |
|---|---|---|
DISSECT | DISSECT('%{otel_message}') | Converts a payload to json using Extended dissect syntax |
GROK | GROK('%{GREEDYDATA:otel_message}') | Converts a payload to json using GROK syntax |