Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reduce TransitEvent struct size #570

Merged
merged 14 commits into from
Sep 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,8 @@
backend thread has increased throughput around 5%
- Simplified the `TransitEventBuffer` in the backend worker thread, resulting in a minor throughput improvement of
approximately 1%.
- Optimised the size of the `TransitEvent` structure, reducing its memory footprint and improving backend thread
throughput by an additional 3%.
- Added an optional fsync interval to control the minimum time between consecutive fsync calls, reducing disk wear from
frequent fsync operations. This option is only applicable when fsync is
enabled. ([#557](https://github.com/odygrd/quill/issues/557))
Expand Down
1 change: 0 additions & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,6 @@ set(HEADER_FILES
include/quill/core/DynamicFormatArgStore.h
include/quill/core/Codec.h
include/quill/core/Filesystem.h
include/quill/core/FormatBuffer.h
include/quill/core/FrontendOptions.h
include/quill/core/InlinedVector.h
include/quill/core/LoggerBase.h
Expand Down
8 changes: 4 additions & 4 deletions benchmarks/backend_throughput/quill_backend_throughput.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,15 @@ static constexpr size_t total_iterations = 4'000'000;
int main()
{
// main thread affinity
quill::detail::set_cpu_affinity(0);
quill::detail::set_cpu_affinity(1);

quill::BackendOptions backend_options;
backend_options.cpu_affinity = 5;
backend_options.sleep_duration = std::chrono::nanoseconds{0};

// Start the logging backend thread and give it some tiem to init
quill::Backend::start(backend_options);

std::this_thread::sleep_for(std::chrono::milliseconds{100});

// Create a file sink to write to a file
std::shared_ptr<quill::Sink> file_sink = quill::Frontend::create_or_get_sink<quill::FileSink>(
"quill_backend_total_time.log",
Expand All @@ -41,7 +40,8 @@ int main()
"%(time) [%(thread_id)] %(short_source_location) %(log_level) %(message)", "%H:%M:%S.%Qns",
quill::Timezone::LocalTime, false});

quill::Frontend::preallocate();
LOG_INFO(logger, "preallocate");
logger->flush_log(0);

// start counting the time until backend worker finishes
auto const start_time = std::chrono::steady_clock::now();
Expand Down
11 changes: 4 additions & 7 deletions benchmarks/hot_path_latency/quill_hot_path_rdtsc_clock.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -28,18 +28,13 @@ void quill_benchmark(std::vector<uint16_t> const& thread_count_array,
/** - MAIN THREAD START - Logger setup if any **/

/** - Setup Quill **/
// main thread affinity
quill::detail::set_cpu_affinity(0);

quill::BackendOptions backend_options;
backend_options.cpu_affinity = 5;
backend_options.sleep_duration = std::chrono::nanoseconds{0};

// Start the logging backend thread and give it some tiem to init
quill::Backend::start(backend_options);

std::this_thread::sleep_for(std::chrono::milliseconds{100});

// wait for the backend thread to start
std::this_thread::sleep_for(std::chrono::seconds(1));

Expand All @@ -62,9 +57,11 @@ void quill_benchmark(std::vector<uint16_t> const& thread_count_array,

/** LOGGING THREAD FUNCTIONS - on_start, on_exit, log_func must be implemented **/
/** those run on a several thread(s). It can be one or multiple threads based on THREAD_LIST_COUNT config */
auto on_start = []() {
auto on_start = [logger]()
{
// on thread start
Frontend::preallocate();
LOG_INFO(logger, "preallocate");
logger->flush_log(0);
};

auto on_exit = [logger]()
Expand Down
8 changes: 3 additions & 5 deletions benchmarks/hot_path_latency/quill_hot_path_system_clock.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,6 @@ void quill_benchmark(std::vector<uint16_t> const& thread_count_array,
/** - MAIN THREAD START - Logger setup if any **/

/** - Setup Quill **/
// main thread affinity
quill::detail::set_cpu_affinity(0);

quill::BackendOptions backend_options;
backend_options.cpu_affinity = 5;
backend_options.sleep_duration = std::chrono::nanoseconds{0};
Expand Down Expand Up @@ -63,10 +60,11 @@ void quill_benchmark(std::vector<uint16_t> const& thread_count_array,

/** LOGGING THREAD FUNCTIONS - on_start, on_exit, log_func must be implemented **/
/** those run on a several thread(s). It can be one or multiple threads based on THREAD_LIST_COUNT config */
auto on_start = []()
auto on_start = [logger]()
{
// on thread start
Frontend::preallocate();
LOG_INFO(logger, "preallocate");
logger->flush_log(0);
};

auto on_exit = [logger]()
Expand Down
2 changes: 1 addition & 1 deletion include/quill/backend/BackendOptions.h
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ struct BackendOptions
* transit_events_hard_limit The backend will use a separate transit_event_buffer for each
* frontend thread. The capacity must be a power of two.
*/
uint32_t transit_event_buffer_initial_capacity = 64;
uint32_t transit_event_buffer_initial_capacity = 128;

/**
* The backend gives priority to reading messages from the frontend queues of all
Expand Down
Loading