1.16.1

Fixed

• Tracing: Fix concurrent-ruby future propagation without active_trace (#3242)
• Tracing: Fix host injection error handling (#3240)

Read the full changeset and the release milestone.

1.16.0

This release includes a security change for the Tracing Redis integration:

Currently, the Datadog Agent removes command arguments from the resource name. However there are cases, like Redis compressed keys, where this obfuscation cannot correctly remove command arguments. To safeguard that situation, the resource name set by the tracer will only be the command (e.g. SET) with no arguments. To retain the previous behavior and keep arguments in the span resource, with the potential risk of some command arguments not being fully obfuscated, set DD_REDIS_COMMAND_ARGS=true or set the option c.instrument :redis, command_args: true.

Added

• Tracing: Propagate trace through Concurrent::Promises.future (#1522)
• Core: Name Datadog::Core::Remote::Worker thread (#3207)

Changed

• Tracing: Redis - Omit command arguments from span.resource by default (#3235)
• Ci-app: Bump datadog-ci dependency from 0.2.0 to 0.3.0 (#3223)

Fixed

• Appsec: ASM parse response body (#3153)
• Appsec: ASM make sure to append content type and length information (#3204)
• Appsec: Make sure function that checks content-type header value accepts nil content-type header value (#3234)
• Profiling: Shut down profiler if any components failed (#3197)
• Tracing: Fix ActiveSupport instrumentation of custom cache stores (#3206)

Read the full changeset and the release milestone.

1.15.0

Highlights

Timeline view for Profiler beta

As of ddtrace 1.15.0, the Profiler now supports gathering data for the new Timeline view.

The Timeline view allows you to look at time-based patterns and work distribution over the period of a single profile: you can look at what individual threads were doing, and when 🎉

You can use the timeline view both when looking at individual profiles, as well as when scoped to a given trace.

You can enable it:

• Using an environment variable by setting DD_PROFILING_EXPERIMENTAL_TIMELINE_ENABLED=true
• Or via code by adding to your Datadog.configure block:

Datadog.configure do |c|
  # … existing configuration …
  c.profiling.advanced.experimental_timeline_enabled = true
end

Give it a try, let us know what you think!

(Note: We do not recommend enabling this feature prior to 1.15.0!)

google-protobuf dependency is no longer needed by the Profiler

As of ddtrace version 1.15.0, the google-protobuf gem is no longer needed to enable the Profiler.

If you've added this gem to your Gemfile/gems.rb file as part of enabling the Profiler, you can
remove it now. (If you're curious, we've internally replaced this dependency with the libdatadog gem.)

Configure blocking responses for AppSec via configuration or Remote Configuration

As of dd-trace-rb 1.15.0, AppSec supports configuring the blocking response.

You can configure the blocking response via:

• Using the ENV variables: DD_APPSEC_HTTP_BLOCKED_TEMPLATE_HTML=#{file_name}, and DD_APPSEC_HTTP_BLOCKED_TEMPLATE_JSON=#{file_name}
• Via code by adding to your Datadog.configure block:

Datadog.configure do |c|
  # … existing configuration …
  c.appsec.block.templates.html = "#{file_name}"
  c.appsec.block.templates.json = "#{file_name}"
end

• Using the Remote configuration UI. This option allows you to configure the status code and the blocking behaviour. You can redirect malicious attackers to custom pages.
  You can find more information on the official documentation

Configure agentless mode for CI visibility

If you are using CI visibility with a cloud CI provider without access to the underlying worker nodes, such as GitHub Actions or CircleCI, configure the library to use the Agentless mode.

For this, set the following environment variables:

• DD_CIVISIBILITY_AGENTLESS_ENABLED=true
• DD_API_KEY=<your_api_key>

Additionally, configure which Datadog site you want to send your data to:

• DD_SITE (default: datadoghq.com)

You can also enable agentless mode with Datadog.configure block:

Datadog.configure do |c|
  # … existing configuration …
  c.ci.agentless_mode_enabled = true
  # don't forget to set DD_API_KEY env variable!
end

Added

• Enable allocation counting feature by default for some Ruby 3 versions (#3176)
• Detect WebMock Cucumber and Rails.env to disable telemetry and remote configuration for development environment (#3065 , #3062 , #3145)
• Profiling: Import java-profiler PID controller and port it to C (#3190)
• Profiling: Record allocation type when sampling objects (#3096)
• Profiling: Include ruby vm type in profiler allocation samples (#3074)
• Tracing: Support Rack 3 (#3132 )
• Tracing: Support Opensearch 3 (#3189)
• Tracing: grpc adds client_error_handler option (#3095)
• Tracing: Add async option for test_mode configuration (#3158)
• Tracing: Implements _dd.base_service tag (#3018)
• Appsec: Allow blocking response template configuration via ENV variables (#2975)
• Appsec: ASM API security. Schema extraction (#3131, #3166, #3177)
• Appsec: Enable configuring blocking response via Remote Configuration (#3099)
• Ci-app: Validate git tags (#3100)
• Ci-app: Add agentless mode (#3186 )

Changed

• Appsec: Skip passing waf addresses when the value is empty (#3188)
• Profiling: Restore support for Ruby 3.3 (#3167)
• Profiling: Add approximate thread state categorization for timeline (#3162)
• Profiling: Wire up allocation sampling into CpuAndWallTimeWorker (#3103)
• Tracing: dalli disable memcached command tag by default (#3171)
• Tracing: Use first valid extracted style for distributed tracing (#2879 )
• Tracing: Rename configuration option on_set to after_set (#3107)
• Tracing: Rename experimental_default_proc to default_proc (#3091)
• Tracing: Use peer.service for sql comment propagation (#3127)
• Ci-app: Fix Datadog::CI::Environment to support the new CI specs (#3080)
• Bump datadog-ci dependency to 0.2 (#3186)
• Bump debase-ruby_core_source dependency to 3.2.2 (#3163)
• Upgrade libdatadog 5 (#3169, #3104)
• Upgrade libddwaf-rb 1.11.0 (#3087 )
• Update AppSec rules to 1.8.0 (#3140, #3139)

Fixed

• Profiling: Add workaround for incorrect invoke location when logging gem is in use (#3183)
• Profiling: Fix missing endpoint profiling when request_queuing is enabled in rack instrumentation (#3109)
• Appsec: Span tags reporting the number of WAF failed loaded rules (#3106)
• Tracing: Fix tagging with empty data (#3102)
• Tracing: Fix rails.cache.backend span tag with multiple stores (#3060)

Removed

• Profiling: Remove legacy profiler codepath (#3172)
• Ci-app: Remove CI module and add a dependency on datadog-ci gem (#3128)
• Tracing: Remove depends_on option from configuration DSL (#3085)
• Tracing: Remove delegate_to option from configuration DSL (#3086)

Read the full changeset and the release milestone.

1.14.0

Added

• Cucumber 8.0.0 support, test CI visibility with cucumber versions 6-8 (#3061)
• Tracing: Add ddsource to #to_log_format (#3025)
• Core: include peer service configurations in telemetry payload (#3056)
• Tracing: Improve quantization (#3041)

Changed

• Profiling: Disable profiler on Ruby 3.3 due to incompatibility (#3054)
• Core: EnvironmentLogger adjustments (#3020, #3057)

Fixed

• Appsec: Fix ASM setting for automated user events. (#3070)
• Tracing: Fix ActiveRecord adapter name for Rails 7 (#3051)

Read the full changeset and the release milestone.

1.13.1

Fixed

• Tracing: net/http instrumentation excludes query string for http.url tag (#3045)
• Tracing: Remove log_tags warning when given hash for log injection (#3022)
• Tracing: Fix OpenSearch integration loading (#3019)
• Core: Fix default hostname/port when mixing http and uds configuration (#3037)
• Core: Disable Telemetry and Remote Configuration in development environments (#3039)
• Profiling: Improve Datadog::Profiling::HttpTransport error logging (#3038)
• Docs: Document known issues with hanging Resque workers (#3033)

Read the full changeset and the release milestone.

1.13.0

Automated User Login Events for AppSec (Click to expand for more information)

Starting from ddtrace version 1.13.0, Automated User Login Events are enabled by default and currently only available for Devise.

What to expect from Automated User Login Events?

When enabled, any login or signup events will be automatically monitored. We will fill in the span with information extracted from the event.

Modes

The Automated User Login Events are configured to run in "safe" mode by default.

Safe mode focuses on preventing personally identifiable information (PII) from being reported. During safe mode, we only extract the ID from the user object. We report only IDs in the GUID format to avoid disclosing private information. In cases where IDs are not GUID, events will be reported without metadata.

Here is an example of a login event in safe mode:

{
  "appsec.events.users.login.success.track" => true,
  "usr.id" => "5ff35d04-445a-4d00-b6f1-31960153eaf6"
}

Automated User Login Events also provide an "extended" mode.

The extended mode aims to extract the maximum amount of information from the user. In extended mode, we report the ID regardless of its format and try to extract the user's email and username. Depending on the user application, we may be able to extract the user's email and username.

Here is an example of a login event in extended mode:

{
  "appsec.events.users.login.success.track" => true,
  "appsec.events.users.login.success.email" => "john@doe.com",
  "appsec.events.users.login.success.username" => "John Doe",
  "usr.id" => "5ff35d04-445a-4d00-b6f1-31960153eaf6"
}

In cases where safe or extended mode is unable to extract information, we encourage users to manually monitor user login events using our public API.

How to configure Automated User Login Events:

• Use DD_APPSEC_AUTOMATED_USER_EVENTS_TRACKING to disable or change the mode.
  • To disable, use DD_APPSEC_AUTOMATED_USER_EVENTS_TRACKING=disable.
  • To change the mode, use DD_APPSEC_AUTOMATED_USER_EVENTS_TRACKING=safe|extended.
• Add to your Datadog.configure block to configure via code:

Datadog.configure do |c|
  c.appsec.track_user_events.enabled = true | false
  c.appsec.track_user_events.mode = "safe" | "extended"
end

Configuration options are now type checked (Click to expand for details)

Starting with ddtrace 1.13, configuration values are type checked when ddtrace is starting up.

In older versions, when setting configuration via Datadog.configure, no validation was performed on values. We've found this could lead to misleading or incorrect configurations, where something seems to be accepted but then did not work as expected. To address this, type checking is now applied to make sure settings are correct.

We've tested this extensively and don't expect this to impact correct configuration setups.

But, if you do have an incorrect configuration, this will now be flagged using an ArgumentError that includes a descriptive message for the setting name and the expected type.

Alternatively, we've provided a way to skip the type checking, by setting the Environment variable DD_EXPERIMENTAL_SKIP_CONFIGURATION_VALIDATION=true. We don't recommend using this option, since it hides configuration problems and thus may lead to ddtrace misbehaving.

As usual, your feedback is welcome -- let us know if we missed anything.

(Feature added in #2983, #2987, #2998 and #2999)

Added

• Core: Add support for Option precedence (#2915)
• Core: Add support for unsetting options (#2972)
• Core: Gauge primitive RubyVM::YJIT.runtime_stats, if YJIT is enabled (#2711, #2959)
• Core: Add Telemetry app-client-configuration-change event (#2977)
• Core: Improve our SafeDup module (#2960)
• Tracing: Add OpenSearch Integration (#2940)
• Tracing: Implement peer.service tag to integrations (#2982)
• Tracing: Add mandatory rpc and grpc tags for grpc integration (#2620, #2962)
• Tracing: Include _dd.profiling.enabled tag (#2913)
• Tracing: Support host injection (#2941, #3007)
• Tracing: Implement Dynamic Configuration for tracing (#2848, #2973)
• Tracing: Add for dynamic log injection configuration (#2992)
• Tracing: Add sampling configuration with DD_TRACE_SAMPLING_RULES (#2968)
• Tracing: Add HTTP header tagging with DD_TRACE_HEADER_TAGS for clients and servers (#2946, #2935)
• Profiling: Add fallback name/invoke location for unnamed threads started in native code (#2993)
• Profiling: Use invoke location as a fallback for nameless threads in the profiler (#2950)
• Profiling: Add fallback name for main thread in profiling data (#2939)
• Ci-app: Add Minitest CI integration (#2932)
• Appsec: Devise integration and automatic user events (#2877)
• Appsec: Handle disabled tracing and appsec events (#2572)
• Appsec: Automate user events check for UUID in safe mode (#2952)
• Docs: Add Ruby 3.2 support to compatibility matrix (#2971)

Changed

• Core: Set maximum supported Ruby version (#2497)
• Core: Prevent telemetry requests from being traced (#2961)
• Core: Add env and type to Core configuration option (#2983, #2988, #2994)
• Core: Remove lazy from Core configuration option (#2931, #2999)
• Profiling: Bump libdatadog dependency to version 3 (#2948)
• Profiling: Improve error message when ddtrace_profiling_loader fails to load (#2957)
• Tracing: Improve log injection runtime conditionals (#2926, #2882)

Fixed

• Core: Fix polynomial-time regular expressions (#2814)
• Core: Fix environment variable for dynamic configuration polling interval (#2967)
• Core: Reduce remote configuration error logging noise (#3011)
• Tracing: Fix manual log injection for 128 bit trace_id (#2974)
• Tracing: Ensure the GRPC client interceptor return the response (#2928)
• Tracing: Remove dynamic input used in regular expression (#2867)
• Tracing: Fix distributed tracing header formats (#3005 )
• Profiling: Fix profiler libmysqlclient version detection with mysql2-aurora gem (#2956)
• Profiling: Automatically enable profiler "no signals" workaround for passenger web server (#2978)

Read the full changeset and the release milestone.

1.12.1

Highlights

Remote configuration

For remote configuration to operate properly its reported service name must match the main application integration service name.

We highly recommend setting the main service name (set via DD_SERVICE or Datadog.configure { |c| c.service = '...' }) and the Rack integration service name to match. Note that when other integrations such as Rails are present and Rack integration configuration is not set, the Rack integration service name will default to the same integration name.

In general, we recommend using DD_SERVICE or c.service instead of c.remote.service if at all possible, but for situations where this is not possible, this version provides an override via a new configuration setting:

Datadog.configure { |c| c.remote.service = '...' }`

It must be made to match the Rack integration service name, which can be found appearing on rack.request tracing spans.

Examples

Case 1

Datadog.configure do |c|
  c.service = 'foo' # or DD_SERVICE env var
  c.tracing.instrument :rails, service_name: 'foo_web' # Rack will inherit this name
end

We recommend:

• either changing c.service to 'foo_web'
• or :rails, service_name: to 'foo'

If none of these changes are possible, as a last resort add c.remote.service = 'foo_web'.

Case 2

Datadog.configure do |c|
  # no c.service = '...' nor DD_SERVICE env var
  c.tracing.instrument :rails, service_name: 'foo_web'
end

We recommend:

• either setting c.service (or DD_SERVICE) to 'foo_web'
• or setting c.service (or DD_SERVICE) to 'foo' and :rails, service_name: to 'foo'

If none of these changes are possible, as a last resort add c.remote.service = 'foo_web'.

Case 3

Datadog.configure do |c|
  # no c.service = '...' nor DD_SERVICE env var
  c.tracing.instrument :rack, service_name: 'foo_rack'
  c.tracing.instrument :rails, service_name: 'foo_rails'
end

We recommend:

• either setting c.service (or DD_SERVICE) to 'foo_rack'
• or setting c.service (or DD_SERVICE) to 'foo' and :rack, service_name: to 'foo'

If none of these changes are possible, as a last resort add c.remote.service = 'foo_rack'.

Added

• Appsec: Add appsec.blocked tag to span (#2895)
• Profiling: Add workaround for legacy profiler incompatibility with ruby-cloud-profiler gem (#2891)
• Core: Allow setting remote configuration service name (#2853)

Changed

• Appsec: Change the value format for the WAF address server.request.query (#2903)
• Profiling: Log pkg-config command when building profiling native extension

Fixed

• Appsec: Update blocked response content_type resolution (#2900)
• Appsec: Ensure to use service entry span. (#2898)
• Tracing: Fix AWS integration constant loading (#2896)

Read the full changeset and the release milestone.

1.12.0

Highlights

Remote configuration

• We fixed an issue with Passlist.

• We have added support for Custom In-App Waf Rules

• In previous versions, there was an issue with identifying the service name correctly. Everything, by default now should work as expected.

Mounted applications

• Appsec now supports mounted apps. That means you can instrument multiple services with appsec.
  Using the example of a Sinatra app mounted as part of a Rails application, you can now instrument both with AppSec. The configuration below should work now.

require 'datadog/appsec'

Datadog.configure do |c|
  c.appsec.enabled = true
  c.appsec.instrument :rails
  c.appsec.instrument :sinatra
end

Added

• Profiling: Add support for profiling Ruby 3.3.0-preview1 (#2860)
• Appsec: Appsec support nested apps (#2836)
• Appsec: Appsec add support for custom rules (#2856)
• Appsec: Update appsec static rules to 1.7.0 version (#2869)
• Appsec: Tag AppSec on Rack top-level span (#2858)
• Profiling: Implement "no signals" workaround and enable CPU Profiling 2.0 for all customers (#2873)
• Ci: Update CI Visibility spec (#2874)
• Appsec: Added missing waf addresses to request operation (#2883)

Changed

• Tracing: Consistent APM Span tags for AWS SDK Requests (#2730)
• Tracing: Change default service_name values Part 2 (#2765)
• Profiling: Bump debase-ruby_core_source dependency to 3.2.1 (#2875)

Fixed

• Telemetry: Disable for non-HTTP agent connection (#2815)
• Tracing: Fix circular requires (#2826)
• Tracing: Update comment about Datadog::Tracing::Distributed::Ext to correct modules (#2840)
• Appsec: Check if :appsec setting is present before accessing it in remote component (#2854)
• Telemetry: Do not send Dependency hash when version is present (#2855)
• Core: Fix symbol configuration for env and service (#2864)
• Tracing: Fix sql comment propagation full mode when tracing is disabled (#2866)
• Appsec: Use relative URI for server.request.uri.raw (#2890)

Read the full changeset and the release milestone.

1.11.1

Fixed

• Appsec: Remove misreported ASM_CUSTOM_RULES capability (#2829)
• Appsec: Fix block response content negotiation (#2824)
• Appsec: Fix incorrect remote configuration payload key handling (#2822)

Read the full changeset and the release milestone.

1.11.0

Highlights

Remote Configuration

The Remote Configuration feature is now in General Availability.

What to expect from Remote Configuration?

ASM can now receive live updates via Remote Configuration, such as:

• protection rules, including request blocking
• IP and path passlists
• IP and user denylists

Remote Configuration will be progressively expanded to support other Datadog products such as APM.

How to configure Remote Configuration?

• Make sure ASM is enabled (see our documentation)
• Upgrade the Datadog agent to 7.42.0 or above (7.43.0 or above recommended)
• Configure the agent to allow Remote Configuration via e.g the environment variableDD_REMOTE_CONFIGURATION_ENABLED=true

To opt out of Remote Configuration, it can be done via either the environment variable DD_REMOTE_CONFIGURATION_ENABLED=false or Datadog.configure { |c| c.remote.enabled = false }.

Read more about Remote Configuration in our documentation

Ruby CPU Profiling 2.0

The new CPU Profiling 2.0 feature is now in General Availability.

What to expect from Ruby CPU Profiling 2.0?

• Finer-grained profiling data due to our sampling engine rewritten in C+Rust. The profiler will be able to run more often and get more information while keeping the same 2% overhead target you're used to, and with a lower impact on latency. Especially when looking at the "Code Hotspots" panel for a distributed trace, expect more and finer grained profiles.
• Thread id information now includes the operating system thread id for Ruby 3.1+, so you'll be able to correlate your thread information when looking at other system monitoring tools
• Thread names are now collected and you're able to filter your profiles by these names
• The profiler now exposes a Datadog::Profiling.allocation_count API that can be used to measure how many objects were allocated in parts of your application
• Experimental support for capturing CPU and Wall-time spent doing Garbage Collection. This is disabled by default as we're still improving the performance of this feature and fixing a few incompatibilities with Ruby Ractors. You can enable it by adding DD_PROFILING_FORCE_ENABLE_GC=true or c.profiling.advanced.force_enable_gc_profiling = true to the instructions seen above.

...with more and faster improvements to come soon!

If you're using the profiler, the new feature will be automatically enabled, except in the following cases:

• When running on Ruby 2.5 and below
• When the mysql2 or rugged gems are installed

Known issues:

• Rare incompatibilities with native extensions/libraries.

  Ruby CPU Profiling 2.0 gathers profiling data by sending SIGPROF unix signals to Ruby applications. This is a common approach used by many other profilers, and it may cause system calls performed by native extensions/libraries to be interrupted with an EINTR error code (reference).

  Most native extensions/libraries are unaffected by this issue, but we know that the mysql2 and rugged gems can trigger these issues (details). When either of these gems is detected, the new feature is not enabled.

  We expect these occurrences to be rare, and will be working to both improve the ecosystem as well as to deploy countermeasures in the profiler itself to avoid triggering these issues.

• Ruby 2.5 and below are missing an API that allows the profiler to detect the currently-active Ruby thread. We plan to ship a workaround for this issue soon.

• The disabled-by-default experimental support for capturing CPU and Wall-time spent doing Garbage Collection is incompatible with Ractors due to Ruby upstream bugs (https://bugs.ruby-lang.org/issues/18464 and https://bugs.ruby-lang.org/issues/19112). We plan to work with the Ruby developers to incorporate fixes for these issues.

• The disabled-by-default experimental support for capturing CPU and Wall-time spent doing Garbage Collection can cause a lot of overhead in Ruby applications with high object allocation rates.

Telemetry

The Telemetry feature is now in General Availability and enabled by default. It will allow Datadog to provide more efficient support.

Added

• Add remote configuration, enabled by default (#2674, #2678, #2686, #2687, #2688, #2689, #2696, #2705, #2731, #2732, #2733, #2739, #2756, #2769, #2771, #2773, #2789, #2805, #2794)
• AppSec: Add response headers passing to WAF (#2701)
• Tracing: Distributed tracing for Sidekiq (#2513)
• Tracing: Add Roda integration (#2368)
• Profiling: Support disabling endpoint names collection in new profiler (#2698)
• Tracing: Support Sidekiq 7 (#2810)
• Core: Add support for Unix Domain Socket (UDS) configuration via DD_TRACE_AGENT_URL (#2806)
• Core: Enable Telemetry by default (#2762)

Changed

• Core: Allow 1 as true value in environment variables (#2710)
• Profiling: Enable CPU Profiling 2.0 by default (#2702)
• Tracing: Improve controller instrumentation and deprecate option exception_controller (#2726)
• Tracing: Implement Span Attribute Schema Environment Variable (#2727)
• Tracing: Change default service_name values (gated by feature flag) (#2760)

Fixed

• Bug: Tracing: Fix w3c propagation special character handling (#2720)
• Performance: Tracing: Use +@ instead of dup for duplicating strings (#2704)
• Profiling: Avoid triggering allocation sampling during sampling (#2690)
• Integrations: Tracing: Fix Rails < 3 conditional check in Utils#railtie_supported? (#2695)
• Profiling: Do not auto-enable new profiler when rugged gem is detected (#2741)
• Tracing: Fix using SemanticLogger#log(severity, message, progname) (#2748) (@rqz13)
• Profiling: Improve detection of mysql2 gem incompatibilities with profiler (#2770)
• AppSec: Remove check for ::Rack::Request.instance_methods.include?(:each_header) at load time (#2778)
• Tracing: Fix quadratic backtracking on invalid URI (#2788)
• Community: Correctly set mutex (#2757) (@ixti)

Read the full changeset and the release milestone.