Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Serverless Mini Agent] Install Protoc for Serverless Release #630

Merged
merged 3 commits into from
Sep 16, 2024

Conversation

duncanpharvey
Copy link
Contributor

What does this PR do?

Installs protoc for Serverless Release.

Motivation

Follows changes in #606 to install protoc.

Additional Notes

Needed for upcoming release of 0.7.0.

How to test the change?

See https://datadoghq.atlassian.net/wiki/spaces/SLS/pages/2977497119/Serverless+Mini+Agent#Testing

@duncanpharvey duncanpharvey requested a review from a team as a code owner September 16, 2024 13:48
@duncanpharvey duncanpharvey changed the title [Serverless Mini Agent] Intall Protoc for Serverless Release [Serverless Mini Agent] Install Protoc for Serverless Release Sep 16, 2024
@pr-commenter
Copy link

pr-commenter bot commented Sep 16, 2024

Benchmarks

Comparison

Benchmark execution time: 2024-09-16 17:52:27

Comparing candidate commit cf46799 in PR branch duncan-harvey/serverless-release-install-protoc with baseline commit 910f604 in branch main.

Found 3 performance improvements and 4 performance regressions! Performance is the same for 44 metrics, 2 unstable metrics.

scenario:credit_card/is_card_number/37828224631000521389798

  • 🟥 execution_time [+7.613µs; +7.948µs] or [+7.978%; +8.330%]
  • 🟥 throughput [-807806.116op/s; -773471.410op/s] or [-7.707%; -7.380%]

scenario:credit_card/is_card_number_no_luhn/37828224631000521389798

  • 🟥 execution_time [+7.470µs; +7.806µs] or [+7.813%; +8.163%]
  • 🟥 throughput [-790975.535op/s; -757010.243op/s] or [-7.562%; -7.238%]

scenario:redis/obfuscate_redis_string

  • 🟩 execution_time [-6.717µs; -6.302µs] or [-15.114%; -14.181%]

scenario:sql/obfuscate_sql_string

  • 🟩 execution_time [-15.721µs; -15.645µs] or [-17.346%; -17.263%]

scenario:tags/replace_trace_tags

  • 🟩 execution_time [-92.569ns; -82.757ns] or [-3.298%; -2.949%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz cf46799 1726508537 duncan-harvey/serverless-release-install-protoc
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 37.173µs 37.930µs ± 1.463µs 37.257µs ± 0.028µs 37.321µs 41.092µs 41.133µs 41.959µs 12.62% 1.712 0.962 3.85% 0.103µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [37.727µs; 38.133µs] or [-0.534%; +0.534%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz cf46799 1726508537 duncan-harvey/serverless-release-install-protoc
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 506.866µs 507.595µs ± 0.349µs 507.549µs ± 0.247µs 507.838µs 508.242µs 508.462µs 508.585µs 0.20% 0.451 -0.306 0.07% 0.025µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1966239.665op/s 1970077.261op/s ± 1353.301op/s 1970254.032op/s ± 958.705op/s 1971158.644op/s 1972016.026op/s 1972484.805op/s 1972907.111op/s 0.13% -0.448 -0.310 0.07% 95.693op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 465.378µs 466.258µs ± 0.367µs 466.245µs ± 0.242µs 466.490µs 466.835µs 467.306µs 467.387µs 0.24% 0.343 0.279 0.08% 0.026µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2139556.033op/s 2144736.773op/s ± 1689.647op/s 2144796.813op/s ± 1115.633op/s 2145892.451op/s 2147265.552op/s 2148352.329op/s 2148789.174op/s 0.19% -0.338 0.273 0.08% 119.476op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 179.723µs 180.416µs ± 0.368µs 180.387µs ± 0.221µs 180.602µs 180.984µs 181.611µs 181.644µs 0.70% 1.024 1.564 0.20% 0.026µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5505262.964op/s 5542763.564op/s ± 11295.002op/s 5543650.230op/s ± 6794.317op/s 5550647.409op/s 5557555.260op/s 5561694.386op/s 5564112.074op/s 0.37% -1.008 1.519 0.20% 798.677op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 43.382µs 43.559µs ± 0.078µs 43.557µs ± 0.056µs 43.609µs 43.695µs 43.736µs 43.768µs 0.48% 0.172 -0.423 0.18% 0.006µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 22847885.013op/s 22957202.839op/s ± 41173.350op/s 22958315.417op/s ± 29559.700op/s 22988793.084op/s 23019258.604op/s 23039940.878op/s 23050997.834op/s 0.40% -0.164 -0.428 0.18% 2911.395op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 49.928µs 50.120µs ± 0.143µs 50.065µs ± 0.062µs 50.239µs 50.375µs 50.450µs 50.590µs 1.05% 0.934 -0.274 0.28% 0.010µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 19766629.654op/s 19952394.942op/s ± 56827.423op/s 19973885.306op/s ± 24556.775op/s 19992385.458op/s 20016781.978op/s 20022037.699op/s 20028902.172op/s 0.28% -0.927 -0.292 0.28% 4018.306op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [507.546µs; 507.643µs] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1969889.706op/s; 1970264.815op/s] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [466.207µs; 466.309µs] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2144502.604op/s; 2144970.941op/s] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [180.365µs; 180.467µs] or [-0.028%; +0.028%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5541198.186op/s; 5544328.943op/s] or [-0.028%; +0.028%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [43.549µs; 43.570µs] or [-0.025%; +0.025%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [22951496.609op/s; 22962909.070op/s] or [-0.025%; +0.025%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [50.100µs; 50.140µs] or [-0.040%; +0.040%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [19944519.207op/s; 19960270.676op/s] or [-0.039%; +0.039%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz cf46799 1726508537 duncan-harvey/serverless-release-install-protoc
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 1.199µs 1.371µs ± 0.053µs 1.383µs ± 0.034µs 1.412µs 1.441µs 1.463µs 1.464µs 5.80% -0.777 0.702 3.83% 0.004µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [1.364µs; 1.379µs] or [-0.532%; +0.532%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz cf46799 1726508537 duncan-harvey/serverless-release-install-protoc
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.696µs 2.719µs ± 0.010µs 2.720µs ± 0.007µs 2.726µs 2.733µs 2.745µs 2.750µs 1.09% 0.037 -0.046 0.37% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.718µs; 2.720µs] or [-0.052%; +0.052%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz cf46799 1726508537 duncan-harvey/serverless-release-install-protoc
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 8.912ms 8.942ms ± 0.012ms 8.942ms ± 0.007ms 8.948ms 8.964ms 8.975ms 8.980ms 0.42% 0.346 0.392 0.14% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [8.940ms; 8.944ms] or [-0.019%; +0.019%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz cf46799 1726508537 duncan-harvey/serverless-release-install-protoc
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 74.816µs 74.947µs ± 0.151µs 74.926µs ± 0.024µs 74.954µs 75.027µs 75.178µs 76.899µs 2.63% 10.976 137.232 0.20% 0.011µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [74.926µs; 74.968µs] or [-0.028%; +0.028%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz cf46799 1726508537 duncan-harvey/serverless-release-install-protoc
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 17.581µs 23.309µs ± 10.061µs 18.955µs ± 0.273µs 20.143µs 36.898µs 37.743µs 98.494µs 419.61% 4.020 24.045 43.05% 0.711µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [21.915µs; 24.704µs] or [-5.982%; +5.982%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz cf46799 1726508537 duncan-harvey/serverless-release-install-protoc
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 270.144µs 273.864µs ± 1.598µs 273.855µs ± 1.170µs 274.966µs 276.362µs 277.485µs 278.067µs 1.54% 0.077 -0.448 0.58% 0.113µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 3596256.917op/s 3651576.903op/s ± 21303.878op/s 3651561.921op/s ± 15531.341op/s 3667675.699op/s 3684965.407op/s 3697405.219op/s 3701734.646op/s 1.37% -0.050 -0.454 0.58% 1506.412op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 25.757µs 25.825µs ± 0.048µs 25.818µs ± 0.024µs 25.842µs 25.906µs 25.985µs 26.092µs 1.06% 1.554 5.064 0.19% 0.003µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 38325244.658op/s 38722713.872op/s ± 71885.116op/s 38732454.839op/s ± 36056.187op/s 38768096.292op/s 38816027.669op/s 38821170.261op/s 38824297.711op/s 0.24% -1.528 4.912 0.19% 5083.045op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 15.286µs 15.324µs ± 0.028µs 15.317µs ± 0.019µs 15.345µs 15.379µs 15.397µs 15.406µs 0.58% 0.728 -0.310 0.18% 0.002µs 1 200
normalization/normalize_name/normalize_name/good throughput 64910096.766op/s 65255776.757op/s ± 118760.432op/s 65287207.018op/s ± 82860.970op/s 65351848.398op/s 65403722.631op/s 65413971.740op/s 65421042.623op/s 0.20% -0.722 -0.325 0.18% 8397.631op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [273.642µs; 274.085µs] or [-0.081%; +0.081%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [3648624.391op/s; 3654529.416op/s] or [-0.081%; +0.081%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [25.818µs; 25.831µs] or [-0.026%; +0.026%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [38712751.286op/s; 38732676.457op/s] or [-0.026%; +0.026%] None None None
normalization/normalize_name/normalize_name/good execution_time [15.320µs; 15.328µs] or [-0.025%; +0.025%] None None None
normalization/normalize_name/normalize_name/good throughput [65239317.703op/s; 65272235.810op/s] or [-0.025%; +0.025%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz cf46799 1726508537 duncan-harvey/serverless-release-install-protoc
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 293.061ns 306.479ns ± 13.166ns 301.630ns ± 6.651ns 314.600ns 334.711ns 336.264ns 347.074ns 15.07% 0.999 -0.145 4.29% 0.931ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [304.654ns; 308.303ns] or [-0.595%; +0.595%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz cf46799 1726508537 duncan-harvey/serverless-release-install-protoc
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 1.208µs 1.209µs ± 0.001µs 1.208µs ± 0.000µs 1.209µs 1.212µs 1.212µs 1.215µs 0.54% 2.418 6.461 0.09% 0.000µs 1 200
credit_card/is_card_number/ throughput 823046823.236op/s 827316248.913op/s ± 762700.654op/s 827531584.714op/s ± 240121.008op/s 827769904.292op/s 827886101.968op/s 827925316.263op/s 827962949.934op/s 0.05% -2.411 6.406 0.09% 53931.080op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 109.043µs 111.148µs ± 0.749µs 111.203µs ± 0.449µs 111.595µs 112.152µs 113.313µs 114.008µs 2.52% 0.562 1.669 0.67% 0.053µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 8771280.100op/s 8997424.645op/s ± 60431.251op/s 8992567.653op/s ± 36273.736op/s 9037535.630op/s 9084317.323op/s 9118232.881op/s 9170678.244op/s 1.98% -0.496 1.519 0.67% 4273.135op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 102.131µs 103.697µs ± 0.741µs 103.690µs ± 0.340µs 103.979µs 105.284µs 105.877µs 105.962µs 2.19% 0.693 1.115 0.71% 0.052µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 9437358.318op/s 9643953.527op/s ± 68613.815op/s 9644097.559op/s ± 31679.395op/s 9681311.198op/s 9749640.945op/s 9783462.823op/s 9791391.873op/s 1.53% -0.637 1.027 0.71% 4851.729op/s 1 200
credit_card/is_card_number/37828224631 execution_time 1.208µs 1.209µs ± 0.002µs 1.209µs ± 0.001µs 1.211µs 1.212µs 1.213µs 1.219µs 0.89% 1.926 6.670 0.13% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 820084773.187op/s 826945394.070op/s ± 1086880.447op/s 827400616.966op/s ± 361847.018op/s 827731861.952op/s 827872735.949op/s 827980166.200op/s 827988744.804op/s 0.07% -1.906 6.495 0.13% 76854.053op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 98.048µs 99.913µs ± 0.840µs 99.892µs ± 0.516µs 100.381µs 101.329µs 102.164µs 103.342µs 3.45% 0.568 1.091 0.84% 0.059µs 1 200
credit_card/is_card_number/378282246310005 throughput 9676646.160op/s 10009392.774op/s ± 83784.007op/s 10010797.996op/s ± 51770.556op/s 10065821.173op/s 10135922.339op/s 10193840.569op/s 10199094.484op/s 1.88% -0.499 0.933 0.83% 5924.424op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 101.853µs 103.198µs ± 0.762µs 103.046µs ± 0.243µs 103.295µs 104.266µs 106.350µs 106.756µs 3.60% 2.782 9.042 0.74% 0.054µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 9367152.294op/s 9690662.127op/s ± 70148.656op/s 9704435.414op/s ± 22858.457op/s 9726807.573op/s 9758621.463op/s 9785363.015op/s 9818052.260op/s 1.17% -2.709 8.700 0.72% 4960.259op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 20.235µs 21.161µs ± 0.379µs 21.119µs ± 0.264µs 21.410µs 21.811µs 22.082µs 22.110µs 4.69% 0.255 -0.288 1.79% 0.027µs 1 200
credit_card/is_card_number/x371413321323331 throughput 45227428.126op/s 47271938.303op/s ± 844056.820op/s 47349612.780op/s ± 592285.776op/s 47922850.280op/s 48588976.160op/s 49079993.732op/s 49418210.047op/s 4.37% -0.167 -0.316 1.78% 59683.830op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 1.208µs 1.208µs ± 0.000µs 1.208µs ± 0.000µs 1.209µs 1.209µs 1.210µs 1.210µs 0.15% 0.850 0.672 0.04% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 826342599.464op/s 827563646.540op/s ± 311625.844op/s 827598721.764op/s ± 225705.010op/s 827824385.357op/s 827956644.135op/s 828032151.000op/s 828080051.408op/s 0.06% -0.848 0.664 0.04% 22035.275op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 92.748µs 95.925µs ± 0.698µs 96.123µs ± 0.327µs 96.361µs 96.734µs 96.967µs 97.022µs 0.94% -1.590 3.465 0.73% 0.049µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 10306928.553op/s 10425383.907op/s ± 76789.405op/s 10403384.902op/s ± 35361.038op/s 10453519.231op/s 10564143.192op/s 10692614.617op/s 10781897.010op/s 3.64% 1.655 3.760 0.73% 5429.831op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 85.213µs 87.071µs ± 0.651µs 87.101µs ± 0.413µs 87.525µs 88.070µs 88.431µs 88.841µs 2.00% -0.301 0.073 0.75% 0.046µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 11256064.205op/s 11485504.878op/s ± 86052.599op/s 11480946.206op/s ± 54726.278op/s 11534864.359op/s 11626588.442op/s 11717961.567op/s 11735274.860op/s 2.22% 0.345 0.108 0.75% 6084.838op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 1.208µs 1.210µs ± 0.001µs 1.210µs ± 0.001µs 1.211µs 1.212µs 1.212µs 1.213µs 0.27% 0.122 -1.536 0.12% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 824417669.082op/s 826581981.526op/s ± 1013517.959op/s 826604269.823op/s ± 950093.246op/s 827555042.438op/s 827861698.471op/s 827938907.920op/s 828018944.161op/s 0.17% -0.120 -1.537 0.12% 71666.542op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 80.654µs 83.080µs ± 0.719µs 83.044µs ± 0.454µs 83.484µs 84.233µs 84.893µs 85.367µs 2.80% 0.203 0.568 0.86% 0.051µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 11714061.501op/s 12037532.053op/s ± 103971.952op/s 12041863.689op/s ± 65815.958op/s 12107622.747op/s 12200178.751op/s 12263468.615op/s 12398685.575op/s 2.96% -0.137 0.574 0.86% 7351.927op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 101.979µs 103.256µs ± 0.794µs 103.034µs ± 0.233µs 103.367µs 105.148µs 106.295µs 108.223µs 5.04% 3.057 11.360 0.77% 0.056µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 9240196.723op/s 9685196.465op/s ± 72823.183op/s 9705513.222op/s ± 22009.547op/s 9724345.629op/s 9738701.698op/s 9765682.679op/s 9805985.048op/s 1.04% -2.967 10.669 0.75% 5149.377op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 20.298µs 21.103µs ± 0.354µs 21.065µs ± 0.239µs 21.331µs 21.663µs 21.887µs 22.285µs 5.79% 0.319 0.059 1.68% 0.025µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 44872609.967op/s 47400102.966op/s ± 792414.472op/s 47472272.712op/s ± 537217.306op/s 47914501.187op/s 48704268.912op/s 49055979.592op/s 49264966.697op/s 3.78% -0.223 -0.059 1.67% 56032.165op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [1.209µs; 1.209µs] or [-0.013%; +0.013%] None None None
credit_card/is_card_number/ throughput [827210545.938op/s; 827421951.888op/s] or [-0.013%; +0.013%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [111.044µs; 111.252µs] or [-0.093%; +0.093%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [8989049.455op/s; 9005799.835op/s] or [-0.093%; +0.093%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [103.594µs; 103.800µs] or [-0.099%; +0.099%] None None None
credit_card/is_card_number/ 378282246310005 throughput [9634444.312op/s; 9653462.742op/s] or [-0.099%; +0.099%] None None None
credit_card/is_card_number/37828224631 execution_time [1.209µs; 1.209µs] or [-0.018%; +0.018%] None None None
credit_card/is_card_number/37828224631 throughput [826794762.893op/s; 827096025.247op/s] or [-0.018%; +0.018%] None None None
credit_card/is_card_number/378282246310005 execution_time [99.797µs; 100.030µs] or [-0.117%; +0.117%] None None None
credit_card/is_card_number/378282246310005 throughput [9997781.117op/s; 10021004.432op/s] or [-0.116%; +0.116%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [103.092µs; 103.303µs] or [-0.102%; +0.102%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [9680940.198op/s; 9700384.056op/s] or [-0.100%; +0.100%] None None None
credit_card/is_card_number/x371413321323331 execution_time [21.108µs; 21.213µs] or [-0.248%; +0.248%] None None None
credit_card/is_card_number/x371413321323331 throughput [47154960.146op/s; 47388916.461op/s] or [-0.247%; +0.247%] None None None
credit_card/is_card_number_no_luhn/ execution_time [1.208µs; 1.208µs] or [-0.005%; +0.005%] None None None
credit_card/is_card_number_no_luhn/ throughput [827520458.195op/s; 827606834.885op/s] or [-0.005%; +0.005%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [95.828µs; 96.022µs] or [-0.101%; +0.101%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [10414741.634op/s; 10436026.181op/s] or [-0.102%; +0.102%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [86.981µs; 87.161µs] or [-0.104%; +0.104%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [11473578.816op/s; 11497430.941op/s] or [-0.104%; +0.104%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [1.210µs; 1.210µs] or [-0.017%; +0.017%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [826441517.685op/s; 826722445.368op/s] or [-0.017%; +0.017%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [82.980µs; 83.179µs] or [-0.120%; +0.120%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [12023122.540op/s; 12051941.565op/s] or [-0.120%; +0.120%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [103.146µs; 103.366µs] or [-0.107%; +0.107%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [9675103.872op/s; 9695289.057op/s] or [-0.104%; +0.104%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [21.054µs; 21.152µs] or [-0.233%; +0.233%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [47290281.942op/s; 47509923.991op/s] or [-0.232%; +0.232%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz cf46799 1726508537 duncan-harvey/serverless-release-install-protoc
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.342µs 3.183µs ± 1.546µs 3.030µs ± 0.023µs 3.050µs 3.118µs 13.813µs 17.666µs 482.97% 7.952 63.852 48.46% 0.109µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [2.969µs; 3.397µs] or [-6.733%; +6.733%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz cf46799 1726508537 duncan-harvey/serverless-release-install-protoc
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 139.458µs 140.837µs ± 0.425µs 140.835µs ± 0.201µs 141.046µs 141.458µs 141.887µs 142.494µs 1.18% -0.027 2.272 0.30% 0.030µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [140.778µs; 140.895µs] or [-0.042%; +0.042%] None None None

Baseline

Omitted due to size.

@codecov-commenter
Copy link

codecov-commenter commented Sep 16, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 73.01%. Comparing base (910f604) to head (cf46799).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #630      +/-   ##
==========================================
- Coverage   73.03%   73.01%   -0.02%     
==========================================
  Files         252      252              
  Lines       36093    36093              
==========================================
- Hits        26359    26352       -7     
- Misses       9734     9741       +7     
Components Coverage Δ
crashtracker 20.53% <ø> (ø)
datadog-alloc 98.73% <ø> (ø)
data-pipeline 90.12% <ø> (ø)
data-pipeline-ffi 0.00% <ø> (ø)
ddcommon 83.08% <ø> (ø)
ddcommon-ffi 69.52% <ø> (ø)
ddtelemetry 59.10% <ø> (ø)
ipc 83.63% <ø> (ø)
profiling 84.26% <ø> (ø)
profiling-ffi 77.42% <ø> (ø)
serverless 0.00% <ø> (ø)
sidecar 40.12% <ø> (ø)
sidecar-ffi 0.00% <ø> (ø)
spawn-worker 50.36% <ø> (ø)
tinybytes 91.66% <ø> (ø)
trace-mini-agent 72.49% <ø> (ø)
trace-normalization 98.25% <ø> (ø)
trace-obfuscation 95.73% <ø> (ø)
trace-protobuf 77.67% <ø> (ø)
trace-utils 92.88% <ø> (ø)

@duncanpharvey duncanpharvey merged commit a5d9cf0 into main Sep 16, 2024
30 checks passed
@duncanpharvey duncanpharvey deleted the duncan-harvey/serverless-release-install-protoc branch September 16, 2024 18:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants