Creating Custom OTTL Rules

Overview

Custom OTTL (OpenTelemetry Transformation Language) rules provide a powerful mechanism for transforming your telemetry data according to your unique needs. With custom conditions and statements, you can dynamically modify attributes, drop fields, or enrich your data before it’s stored or analyzed.

Key Capabilities Include:

• Granular Control: Specify conditions to precisely determine when transformations should occur.
• Flexible Transformations: Apply custom logic to alter, add, or remove fields and attributes in your telemetry data.
• Test Before You Commit: Use the integrated sandbox and exemplar testing to ensure your logic is correct before pushing it into a production environment.

What Are Conditions and Statements?

• Conditions: Conditions are logical expressions defining when the rule applies. They are evaluated against each piece of telemetry data (e.g., each log, span, or metric event). If all specified conditions are met, the associated statements execute. If you omit conditions, the statements will run on all incoming data.

• Statements: Statements are the actions taken to transform your telemetry. For example, they can set new attribute values, remove fields, rename fields, or perform type conversions. At least one statement is required to perform any meaningful transformation.

Example: If you have a condition checking for attributes["http.status_code"] == 404, and the condition passes, a statement could set attributes["error"] = true or remove a sensitive field like attributes["user_session_id"].

Configuring Custom OTTL Rules

Example:

1. Add Conditions (Optional)

• Each condition is an OTTL expression that returns true or false.
• Click "Add condition" to create a new condition line.
• Example condition:

attributes["http.status_code"] == 404

• Multiple conditions are combined with a logical AND, meaning all must be true for the statements to run.
• If no conditions are specified, the statements apply to every piece of telemetry data.

2. Add Statements (Required)

• Statements define the transformations applied to telemetry data that meets your conditions.
• Click "Add statement" to create a new statement line.
• Example statements:

set(attributes["error"], true)
delete(attributes["user_session_id"])

• You can add multiple statements, all of which will be executed in order once the conditions are met.

3. Use the Exemplar Selector (Optional)

• Select an exemplar (sample event) from your environment or collector.
• The exemplar provides real-world data against which you can test your conditions and statements.
• This step is optional but highly recommended to ensure your logic behaves as intended.

4. OTTLSandbox Testing

• The sandbox allows you to run your conditions and statements against the selected exemplar.
• View the transformed output in real-time.
• Identify and fix any syntax errors, invalid references, or unintended side effects before finalizing the rule.

Syntax and Validation

• OTTL uses a simple, flexible syntax for conditions and statements.
• Access attributes and fields using either dot notation (for well-known fields) or attributes["field_name"] for arbitrary attributes.
• Functions such as set(), delete(), and keep_keys() can be used to manipulate attributes.
• If you reference fields not present in the exemplar or make a syntax error, the sandbox will alert you, allowing you to correct the rule before saving.

Testing and Verification

1. Start Small

• Begin with a simple condition and a single statement.
• Verify logic in the sandbox using an exemplar that should trigger the condition.

2. Iterate and Expand

• Add more conditions and statements as needed.
• Test different exemplars to ensure the rule behaves correctly across various data scenarios.

3. Refine and Deploy

• Once you’re confident your custom OTTL rules are correct, proceed to apply them to live telemetry data.
• Monitor the transformed data to ensure the rules are producing the desired outcomes.

Example Configuration

Goal: Mark all logs with http.status_code == 404 as errors and remove any sensitive user session IDs.

Steps:

1. Conditions:

attributes["http.status_code"] == 404

2. Statements:

set(attributes["error"], true)
delete(attributes["user_session_id"])

3. Test with an Exemplar:
   • Choose an exemplar log with status_code = 404 and a user_session_id attribute.
   • Run the sandbox. Confirm that error is now true and user_session_id is removed.

By following these steps, logs with a 404 status become tagged as errors and stripped of sensitive fields, aligning your telemetry pipeline with security and observability best practices.

Summary

Custom OTTL rules empower you to tailor your telemetry data to meet specific needs. Whether you’re enriching data with new fields, removing sensitive information, or applying dynamic logic based on conditions, these rules provide a high degree of flexibility and control. With the sandbox and exemplar testing at your disposal, you can confidently fine-tune your telemetry transformations before deploying them into your production environment.