Skip to content

Python framework to ease Log processing and Tracing for OpenTelemetry applications.

Notifications You must be signed in to change notification settings

anthonyrabiaza/opentelemetry-log-parser

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 

Repository files navigation

OpenTelemetry Log Parser

This Python framework will help you to:

  • Read of Standard OpenTelemetry collector Logs (JSON format)
  • Process and filter of the elements received (for Boomi but can be adapted to any other Platform)
  • Generate Logs with simple format (one line per traceId) easily readable by tools like Promtail/Loki

Alt text

Sample Input

{
    "resourceSpans": [
        {
            "instrumentationLibrarySpans": [
                {
                    "instrumentationLibrary": {
                        "name": "io.opentelemetry.servlet-2.2",
                        "version": "1.4.1"
                    },
                    "spans": [
                        {
                            "attributes": [
                                {
                                    "key": "http.user_agent",
                                    "value": {
                                        "stringValue": "synthetic-monitoring-agent/v0.4.1-0-g49cdae8 (linux amd64; 49cdae8ec92d1bd404ffb3183795b5ed83cccd3f; 2021-12-02 14:23:54+00:00; +https://github.com/grafana/synthetic-monitoring-agent)"
                                    }
                                },
                                {
                                    "key": "http.flavor",
                                    "value": {
                                        "stringValue": "1.1"
                                    }
                                },
                                {
                                    "key": "enduser.id",
                                    "value": {
                                        "stringValue": "boomi_anthonyrabiaza-XXXXXX"
                                    }
                                },
                                {
                                    "key": "thread.id",
                                    "value": {
                                        "intValue": "123"
                                    }
                                },
                                {
                                    "key": "net.peer.ip",
                                    "value": {
                                        "stringValue": "127.0.0.1"
                                    }
                                },
                                {
                                    "key": "thread.name",
                                    "value": {
                                        "stringValue": "1649569546@qtp-599887337-2"
                                    }
                                },
                                {
                                    "key": "http.status_code",
                                    "value": {
                                        "intValue": "200"
                                    }
                                },
                                {
                                    "key": "http.method",
                                    "value": {
                                        "stringValue": "POST"
                                    }
                                },
                                {
                                    "key": "http.client_ip",
                                    "value": {
                                        "stringValue": "17X.1XX.1XX.8"
                                    }
                                },
                                {
                                    "key": "http.url",
                                    "value": {
                                        "stringValue": "http://localhost:9999/ws/rest/apm/test/MQ/?instrument=true"
                                    }
                                }
                            ],
                            "endTimeUnixNano": "1642773532825710376",
                            "kind": "SPAN_KIND_SERVER",
                            "name": "/ws/rest",
                            "parentSpanId": "",
                            "spanId": "8da3cb333857dbe2",
                            "startTimeUnixNano": "1642773532616862371",
                            "status": {},
                            "traceId": "cee6c79e0273f8f8fd92b5dd9f1055df"
                        }
                    ]
                },
              {
                "instrumentationLibrary": {
                  "name": "io.opentelemetry.http-url-connection",
                  "version": "1.4.1"
                },
                "spans": [
                ]
              }
            ]
        }
    ]
}

Sample Output

2022-01-21T21:58:55.546512 traceID=cee6c79e0273f8f8fd92b5dd9f1055df span_number=9 operations="/ws/rest,HTTP GET,SELECT centos7_vmware_dev,SELECT centos7_vmware_dev,JMS_centos7_vmware_dev_myqueue send,DatabaseSend.send,JMSSend.send,ExecutionTask.call,DatabaseSend.send"

Getting Started

Prerequisites

Install the following dependencies:

pip install jsonpath-ng
pip install tqdm

Configure OpenTelemetry Collector

Please check the elements wrapped between HERE and ABOVE

exporters:
  otlp:
    endpoint: abcdefg.antsoftware.org:55680
    insecure: true
  logging:
    loglevel: info
  # HERE
  file:
    path: /opt/opentelemetry/logs/traces.log
  # ABOVE
  prometheusremotewrite:
    endpoint: "https://****@prometheus-us-central1.grafana.net/api/prom/push"
service:
  extensions: [health_check, zpages]
  pipelines:
    traces/1:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlp]
    # HERE
    traces/2:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [file]
    # ABOVE
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [prometheusremotewrite]
(...)

Execute the Framework

python3 -u main.py
----------------------------------------------------------------
OpenTelemetry Log Parser
  https://github.com/anthonyrabiaza/opentelemetry-log-parser
  Press Ctrl+C to stop the process
----------------------------------------------------------------
Opening traces.log
Writing to traces.log
2022-01-21 21:14:57.833536: Receiving and processing traces...
 64%|███████████████████████████████████████████████████████████████████████████████████████████████████████████▌                                                            | 64/100 [06:10<04:13,  7.04s/it]

Execute the Framework in background

./otel-log-parser.sh

Promtail configuration

server:
  http_listen_port: 0
  grpc_listen_port: 0
positions:
  filename: /tmp/positions.yaml
client:
  url: https://****@logs-prod-us-central1.grafana.net/api/prom/push
scrape_configs:
- job_name: otelcol-parsed
  static_configs:
  - targets:
      - localhost
    labels:
      job: otelcol-parsed-logs
      __path__: /opt/opentelemetry/logs/traces_parsed*.log
  pipeline_stages:
  - match:
      selector: '{job="otelcol-parsed"}'
      stages:
      - regex:
          expression: '.*traceID=(?P<traceID>[0-9a-zA-Z]+) spanNumber=(?P<spanNumber>[0-9]+) operations=(?P<operations>.+)'
      - labels:
          traceID:
          spanNumber:
          operations:

Example of View in Grafana Loki and Tempo

Alt text

About

Python framework to ease Log processing and Tracing for OpenTelemetry applications.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published