Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: opentelemetry and prometheus #2543

Merged
merged 4 commits into from
Jul 5, 2024
Merged

feat: opentelemetry and prometheus #2543

merged 4 commits into from
Jul 5, 2024

Conversation

zzzming
Copy link
Collaborator

@zzzming zzzming commented Jul 4, 2024

This PR introduces as the first phase of OpenTelemetry implementation.

  • Opentelemetry in the existing TelemetryService
  • Optionally Prometheus can be enabled
  • Prometheus port is configured. The default is 9090.
  • Metrics collected by OpenTelemetry can be exposed through Prometheus
  • A FastAPI middleware that collects all http related metrics

What's not implemented in this PR

  • Other exporters such as Console, Jaeger, Zipkin, or any other customer exporters.

An example of prometheus scraping

$curl localhost:9090/metrics
# HELP python_gc_objects_collected_total Objects collected during gc
# TYPE python_gc_objects_collected_total counter
python_gc_objects_collected_total{generation="0"} 59153.0
python_gc_objects_collected_total{generation="1"} 31939.0
python_gc_objects_collected_total{generation="2"} 4232.0
# HELP python_gc_objects_uncollectable_total Uncollectable objects found during GC
# TYPE python_gc_objects_uncollectable_total counter
python_gc_objects_uncollectable_total{generation="0"} 0.0
python_gc_objects_uncollectable_total{generation="1"} 0.0
python_gc_objects_uncollectable_total{generation="2"} 0.0
# HELP python_gc_collections_total Number of times this generation was collected
# TYPE python_gc_collections_total counter
python_gc_collections_total{generation="0"} 1881.0
python_gc_collections_total{generation="1"} 171.0
python_gc_collections_total{generation="2"} 10.0
# HELP python_info Python platform information
# TYPE python_info gauge
python_info{implementation="CPython",major="3",minor="11",patchlevel="9",version="3.11.9"} 1.0
# HELP target_info Target metadata
# TYPE target_info gauge
target_info{service_name="langflow",telemetry_sdk_language="python",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.25.0"} 1.0
# HELP requests_bytes_total The number of requests
# TYPE requests_bytes_total counter
requests_bytes_total 1.0
# HELP http_server_active_requests measures the number of concurrent HTTP requests that are currently in-flight
# TYPE http_server_active_requests gauge
http_server_active_requests{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860"} 0.0
# HELP http_server_duration_milliseconds Duration of HTTP client requests.
# TYPE http_server_duration_milliseconds histogram
http_server_duration_milliseconds_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="0.0",net_host_port="7860"} 0.0
http_server_duration_milliseconds_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="5.0",net_host_port="7860"} 0.0
http_server_duration_milliseconds_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="10.0",net_host_port="7860"} 0.0
http_server_duration_milliseconds_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="25.0",net_host_port="7860"} 1.0
http_server_duration_milliseconds_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="50.0",net_host_port="7860"} 2.0
http_server_duration_milliseconds_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="75.0",net_host_port="7860"} 2.0
http_server_duration_milliseconds_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="100.0",net_host_port="7860"} 2.0
http_server_duration_milliseconds_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="250.0",net_host_port="7860"} 2.0
http_server_duration_milliseconds_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="500.0",net_host_port="7860"} 2.0
http_server_duration_milliseconds_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="750.0",net_host_port="7860"} 2.0
http_server_duration_milliseconds_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="1000.0",net_host_port="7860"} 2.0
http_server_duration_milliseconds_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="2500.0",net_host_port="7860"} 2.0
http_server_duration_milliseconds_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="5000.0",net_host_port="7860"} 2.0
http_server_duration_milliseconds_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="7500.0",net_host_port="7860"} 2.0
http_server_duration_milliseconds_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="10000.0",net_host_port="7860"} 2.0
http_server_duration_milliseconds_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="+Inf",net_host_port="7860"} 2.0
http_server_duration_milliseconds_count{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",net_host_port="7860"} 2.0
http_server_duration_milliseconds_sum{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",net_host_port="7860"} 52.0
# HELP http_server_response_size_bytes measures the size of HTTP response messages (compressed).
# TYPE http_server_response_size_bytes histogram
http_server_response_size_bytes_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="0.0",net_host_port="7860"} 0.0
http_server_response_size_bytes_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="5.0",net_host_port="7860"} 0.0
http_server_response_size_bytes_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="10.0",net_host_port="7860"} 0.0
http_server_response_size_bytes_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="25.0",net_host_port="7860"} 0.0
http_server_response_size_bytes_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="50.0",net_host_port="7860"} 0.0
http_server_response_size_bytes_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="75.0",net_host_port="7860"} 2.0
http_server_response_size_bytes_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="100.0",net_host_port="7860"} 2.0
http_server_response_size_bytes_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="250.0",net_host_port="7860"} 2.0
http_server_response_size_bytes_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="500.0",net_host_port="7860"} 2.0
http_server_response_size_bytes_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="750.0",net_host_port="7860"} 2.0
http_server_response_size_bytes_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="1000.0",net_host_port="7860"} 2.0
http_server_response_size_bytes_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="2500.0",net_host_port="7860"} 2.0
http_server_response_size_bytes_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="5000.0",net_host_port="7860"} 2.0
http_server_response_size_bytes_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="7500.0",net_host_port="7860"} 2.0
http_server_response_size_bytes_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="10000.0",net_host_port="7860"} 2.0
http_server_response_size_bytes_bucket{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",le="+Inf",net_host_port="7860"} 2.0
http_server_response_size_bytes_count{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",net_host_port="7860"} 2.0
http_server_response_size_bytes_sum{http_flavor="1.1",http_host="127.0.0.1:7860",http_method="GET",http_scheme="http",http_server_name="localhost:7860",http_target="/health_check",net_host_port="7860"} 136.0

@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. python Pull requests that update Python code labels Jul 4, 2024
Copy link
Contributor

github-actions bot commented Jul 4, 2024

Pull Request Validation Report

This comment is automatically generated by Conventional PR

Whitelist Report

Whitelist Active Result
Pull request is a draft and should be ignored
Pull request is made by a whitelisted user and should be ignored
Pull request is submitted by a bot and should be ignored
Pull request is submitted by administrators and should be ignored

Result

Pull request does not satisfy any enabled whitelist criteria. Pull request will be validated.

Validation Report

Validation Active Result
All commits in this pull request has valid messages
Pull request does not introduce too many changes
Pull request has a valid title
Pull request has mentioned issues
Pull request has valid branch name
Pull request should have a non-empty body

Result

Pull request satisfies all enabled pull request rules.

Last Modified at 04 Jul 24 21:19 UTC

@github-actions github-actions bot added the enhancement New feature or request label Jul 4, 2024
Copy link
Contributor

@ogabrielluiz ogabrielluiz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awesome. Thanks, @zzzming

LGTM

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Jul 5, 2024
@ogabrielluiz ogabrielluiz enabled auto-merge (squash) July 5, 2024 12:37
@github-actions github-actions bot added enhancement New feature or request and removed enhancement New feature or request labels Jul 5, 2024
@ogabrielluiz ogabrielluiz merged commit c444a69 into main Jul 5, 2024
29 checks passed
@ogabrielluiz ogabrielluiz deleted the feat/telemetry branch July 5, 2024 12:50
carlosrcoelho pushed a commit that referenced this pull request Jul 5, 2024
* opentelemetry and prometheus

* set env override in create_app

* add prometheus_client to pyproject

* update top level poetry.lock
ogabrielluiz pushed a commit to yaitec/langflow that referenced this pull request Jul 9, 2024
* opentelemetry and prometheus

* set env override in create_app

* add prometheus_client to pyproject

* update top level poetry.lock
nicoloboschi pushed a commit to datastax/ragstack-ai-langflow that referenced this pull request Jul 10, 2024
* opentelemetry and prometheus

* set env override in create_app

* add prometheus_client to pyproject

* update top level poetry.lock

(cherry picked from commit c444a69)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request lgtm This PR has been approved by a maintainer python Pull requests that update Python code size:M This PR changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants