Json/Structured Logs extras

[JoshBurrell]

Hey everyone,

Going off the documentation there is 2 ways of passing variables into a custom json sink.

Either using the normal LOG macro and then naming the variables
info("'{name}' scored {points} - {total_score}", name, points, total_score);

or using the LOGJ macro which will append the variables to the end of the message
warn("This is a warning, penalty applied", name, penalty);

However I want a best of both worlds. I want to be able to do something like this
info("'{name}' scored {points}", name, points, total_score);

which would result in the formatted string given name='Josh', points=3 to be "'Josh' scored 3", this is still possible however the sink/backend thread doesn't receive the total_score variable because it wasn't mentioned in the format string. Some logging libraries call this stuff extras and in an ideal world the named_args the sink gets on void write_log would contain all 3 variables.

I have tried overriding the LOGJ macro to always append a character at the end of all messages then in the sink do a substring from 0 to the appended character as a hacky way to know what the original formatted message was hoping that it would format the whole string and then I could string out the appended variables.
Eg

#define info(fmt, ...) QUILL_LOGJ_INFO(global_logger_a, fmt"|", ##__VA_ARGS__)

info("'{name}' scored {points}", name, points, total_score);

Could not format log statement. message: "'{name}' scored {points}| {name}, {points}, {total_score}"
error: "argument not found"

tldr Is there a way to have access to all variables passed into a macro in a sink while still having the the message string formatted for you

[odygrd]

Hey, I don’t think it’s possible at the moment.

The key names are derived from the format string, so if a key is missing there, the value of the argument will be ignored.

The thread issuing the log passes only the format string and argument values to the backend thread, not a separate key-value map. The backend thread extracts the key names from the format string and maps them to their corresponding values.

We could still print the value without a key, it wouldn’t be very useful, but i am okay with it.

For example, if we add it positionally to a JSON output for total_score of 100 like this:

info("'{name}' scored {points}", name, points, total_score);

json :
{“name”: "Josh", “score”: 3, “2”: 100}

non json
Josh scored 3

[odygrd]

The display of key names as placeholders has been implemented in commit 069aad7 and will be included in the next release.

You can find the detailed description in the changelog for version 7.2.0: CHANGELOG.md.

To see it in action, you can run the example here: json_file_logging.cpp.