OpenTelemetry Collector Demo

Introduction

OpenTelemetry Collector+Jaeger+Prometheus的可观测案例

• Gateway ：Nginx
• 前端：Java SpringBoot Web + OpenTelemetry + Jaeger Trace Exporter + Prometheus Metric Exporter
• 后端：OpenTelemetry Collector 、Jaeger UI 、Prometheus UI

深入了解可观测体系下Traces、Metrics采集、运行原理
演示地址 http://www.opentelemetry.cool/collector/demo

• 

Architecture

Tracing 效果图

Prometheus 采集Metric

框架列表

Library/Framework	Versions	备注
opentelemetry-collector-contrib	0.74.0	​
opentelemetry-api	1.9.1	​
opentelemetry-sdk	1.9.1	​
opentelemetry-exporter-jaeger	1.9.1	​
jaegertracing	all-in-one:1.29	docker镜像
spring-boot	2.6.2

Linux

• Docker 环境，三个服务部署在一台服务器上，网络Host模式

## down
docker-compose -f /path/docker-compose-collector.yml  down
## start
docker-compose -f /path/docker-compose-collector.yml  up -d

Windows

具体看这个Issue： windows系统无法对docker容器进行端口映射的问题

Mac

Mac 用Docker Host 模式不支持的，请把 OpenTelemetry Collector、Jaeger 端口映射方式部署
There is no docker0 bridge on the host

采样 Sampling

只上报异常链路采样Demo

processors:
  batch:
  tail_sampling:
    ##decision_wait: 10s
    ##num_traces: 100
    ##expected_new_traces_per_sec: 100
    policies:
      [
          {
            name: test-policy-5,
            type: status_code,
            status_code: {status_codes: [ERROR]}
          }
      ]
service:
  extensions: [pprof,health_check]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [tail_sampling]
      exporters: [jaeger]

采样的效果

• 采样前:
• 采样后: Processor 处理器添加了尾部取样 processors: [tail_sampling]

开启Collector 下的Logging Exporter

• Collector 添加 Exporter: [logging]

exporters:
 ##设置exporters上报 collector 控制台信息
 logging:
  loglevel: "debug"
  
  pipelines:
    traces:
      receivers: [otlp]
      processors: [tail_sampling]
      exporters: [jaeger,logging]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [prometheus,logging]

• 生效后: Collector 的日志控制台打印完整的上报链路结构

更多Collector 插件的用法

• filterprocessor：使用案例

开启日志 debug 模式

Version 0.36 and above:

Set the log level in the config service::telemetry::logs

service:
  telemetry:
    logs:
      level: "debug"

重新编译Java 项目

Maven 编译Java 程序

mvn package -DskipTests=true

• target下把对应jar包重命名java-demo.jar，放到Dockerfile文件同级目录
• Docker默认网络方式是Host模式、将Spring Boot 的配置文件otel-collector-java/src/main/resources/application-test.yml 的 host 设置成你的内网 IP，配置SDK 上报Jaeger服务器地址

trace:
 sdk:
  exporter:
   host: "127.0.0.1"
   port: "14250"

区分环境部署

• Dockerfile ENTRYPOINT 中添加环境变量

"-Dspring.profiles.active=test"

Documentation

https://github.com/open-telemetry/opentelemetry-java-instrumentation
https://www.jaegertracing.io/docs/1.29/getting-started/
https://opentelemetry.io/docs/
https://github.com/open-telemetry/opentelemetry-java
https://github.com/open-telemetry/opentelemetry-collector
opentelemetry-collector tailsamplingprocessor

联系

如果有什么疑问和建议，欢迎提交issues，我会第一时间回复