Skip to content

7.28.0
Compare
Choose a tag to compare
@siddharthlatest siddharthlatest released this 26 Jun 18:57
· 443 commits to dev since this release
7.28.0
ec53083

Reliability and Performance

This release includes major reliability ⎍ and performance enhancements 🚀 and as a result of the adoption of a rigorous benchmarking framework based on esrally by our team. 🤓

You should absolutely consider updating to this release! Read more on the details of the improvements below:

  • Latency improvements: Arc is now very close to being optimal as a reverse proxy to an Elasticsearch upstream. This release includes performance enhancements based on profiling Arc's CPU and memory usage and should show a 50ms to 1s latency improvement [1] per request.
  • Memory usage improvements: We've been able to get peak memory usage [2] shrink from 1.2GB+ to ~300MB (over 4x improvement). A major chunk of this improvement comes from a decision to have Arc save logs to the filesystem instead of sending to the Elasticsearch upstream. A fluentbit process now separately logs these to the Elasticsearch upstream, and controls the logging throughput using a combination of throttling, windowing and sending logs in bulk to have minimal impact on the actual throughput of the search applications.
  • Reliability improvements: Earlier versions of Arc couldn't complete the benchmarking tests without crashing. Arc now reliably completes benchmarking without adding a significant throughput or latency overhead over directly using an Elasticsearch upstream.

See the detailed benchmark results report - Baseline uses an Elasticsearch upstream and the contender is Arc v7.28.0 release.

Features

  • An enhancement in how Arc creates an index's search template covers some important gaps for a general-purpose search strategy and at the same time, improves the index's storage efficiency by up to 10% #123
  • Introduction of a fluentbit based logging daemon to ensure that search application bursts get a higher priority than logs
  • Add support for copying synonyms and search relevancy strategy across indexes
  • Adds support for setting all Arc generated indices as hidden (which has been introduced starting Elasticsearch v7.7.0)

Fixes

  • Search relevancy: Field weights can have floating values (previously, only integer values were supported)
  • Default sniffing to false (no sniffing) instead of true as most Elasticsearch hosting services offer load balancing with an option to set sniffing (when you're hosting on your own and/or want to bypass an additional load balancer layer b/w Arc and Elasticsearch)

[1] Latency improvements came as a result of making our usage of bcrypt more efficient (check once per unique user encountered) v/s check per request, and improving how each middleware interacts with the request and response bodies in a request's lifecycle. The most latency improvements are related to advanced (noss) features of search relevancy and query rules.

[2] Peak memory usage was measured after benchmarking with esrally on a custom track by performing a series of bulk requests, index-stats requests, node-stats requests, term queries, phrase queries, and uncached aggregations.

Assets 6
Loading