Added config options to define json fields and line_format for loki logs

[0x49b]

Changes

Added configuration variables to define a json map to extract fields and to define the line_format of Loki logs. By passing a string with json in LOGGING_PLUGIN_JSON_MAP like {"timestamp": "[\"@timestamp\"]"} the json field timestamp gets extracted into timestamp what then can be used to define a custom line_format in LOGGING_PLUGIN_LINE_FORMAT like
"{{.timestamp}}: {{.message}}"

Mentioned in #1204

/kind feature

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you review them:

• Has Docs included if any changes are user facing
• Has Tests included if any functionality added or changed
• Tested your changes locally (if this is a code change)
• Follows the commit message standard
• Meets the Tekton contributor standards (including functionality, content, code)
• Has a kind label. You can add a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
• Release notes block below has been updated with any user-facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings)
• Release notes contain the string "action required" if the change requires additional action from users switching to the new release

Release Notes

Make the Loki LogQL more configurable by letting the user decide a json map for fields to be extracted and also define the following line_format (#1204) 

[linux-foundation-easycla]

• ✅ login: 0x49b / name: 0x49b (329ba04, 22545fb)
• ✅ login: 0x49b / name: Florian Thiévent (1bbf44d, 66528dd, a6fe1d1, d6e09c3, f66685c)
• ❌ - login: @bhanuprasad14 / name: Test User . The commit (a0dfa2d) is not authorized under a signed CLA. Please click here to be authorized. For further assistance with EasyCLA, please submit a support request ticket.

[khrm]

I need to test this locally with LokiStack operator. It looks like a good feature.

[aThorp96]

This seems like a good change to me. Thanks for the contribution!

A couple asks:

• This is configured using a vendor-agnostic variable LOGGING_PLUGIN_LINE_FORMAT. It makes sense to me that this feature should be implemented for the other logging plugin formats supported by LOGS_TYPE or at least issues created to port support to the other log types (I believe the full list of log types are Loki, GCS, S3, "blob" (same as S3?), and Splunk).
• Please also add unit tests and documentation (docs/logging-supprot.md) for the new configuration options. It's worth noting in that documentation that LOGGING_PLUGIN_JSON_MAP has no effect on the logging output unless used in conjunction with LOGGING_PLUGIN_LINE_FORMAT.

Along a similar lines, since I am not very familiar with Loki: since there's the json key=value component to the query, does LOGGING_PLUGIN_LINE_FORMAT work when used without LOGGING_PLUGIN_JSON_MAP, or do they have to be used together in all cases?

[0x49b]

@aThorp96 Thank you for your review. I will add tests and documentation for these two config variables. I can add Feature Requests for the other log formats besides Loki to have that feature for the others too, but I'm not really able to help because I do not know them (GCS, S3, Splunk, etc) well enough.
What also can work to include the Log Vendor Type in the config variable, so it is better for users if config is vendor-agnostic like in this case LOGGING_PLUGIN_LOKI_JSON_MAP and LOGGING_PLUGIN_LOKI_LINE_FORMAT.

The LOGGING_PLUGIN_LINE_FORMAT only works if there are fields set in LOGGING_PLUGIN_JSON_MAP. With the documentation I will add, this should be clear for users that they work in conjunction.

[khrm]

I tested this and it works fine with operator.

[khrm]

Can you please squash these commits? I will give lgtm then/

[tekton-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign vdemeester after the PR has been reviewed.
You can assign the PR to them by writing /assign @vdemeester in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[khrm]

@0x49b Can you please sign easy cla? And squash the commits?

[0x49b]

EasyCLa for me (0x49b) is done, but the one for @bhanuprasad14 is not. Not a Colleague of mine... Squash I will do