Krkn telemetry integration

[tsebastiani]

Adding telemetry data collection to krkn.

[tsebastiani]

/funtest

[chaitanyaenr]

Testing update: successfully ran hog scenarios with telemetry enabled. Here is the output stored in S3:

{
    "scenarios": [
        {
            "startTimeStamp": 1689037675,
            "endTimeStamp": 1689037762.1072097,
            "scenario": "scenarios/arcaflow/cpu-hog/input.yaml",
            "exitStatus": 0,
            "parametersBase64": "",
            "parameters": {
                "input_list": [
                    {
                        "cpu_count": 1,
                        "cpu_load_percentage": 80,
                        "cpu_method": "all",
                        "duration": "30s",
                        "kubeconfig": "anonymized",
                        "namespace": "default",
                        "node_selector": {}
                    }
                ]
            }
        },
        {
            "startTimeStamp": 1689037762,
            "endTimeStamp": 1689037797.4187903,
            "scenario": "scenarios/arcaflow/memory-hog/input.yaml",
            "exitStatus": 0,
            "parametersBase64": "",
            "parameters": {
                "input_list": [
                    {
                        "duration": "30s",
                        "kubeconfig": "anonymized",
                        "namespace": "default",
                        "node_selector": {},
                        "vm_bytes": "10%",
                        "vm_workers": 2
                    }
                ]
            }
        }
    ]
}

Nice addition. Thinking about a couple of enhancements:

• Use uuid generated during the run as name of the document as an identifier
• Alerts triggered pre and post chaos ( separate entries )
• Thinking if OCP metadata can be part of the same document to make it easy to tie things together. Thoughts?

[chaitanyaenr]

Testing update:

Warnings around capturing builds

2023-07-27 19:27:00,903 [WARNING] CustomObjectsApi -> (404)
Reason: Not Found
{},"status":"Failure","message":"builds.config.openshift.io \"status\" not found","reason":"NotFound","details":{"name":"status","group":"config.openshift.io","kind":"builds"},"code":404}

Maybe we should print the details about the bucket where the data is stored at the end of the run for the users with access to be able to find it. Thoughts?

[chaitanyaenr]

@tsebastiani let's rebase this PR.

[tsebastiani]

/funtest

[tsebastiani]

/funtest

[tsebastiani]

/funtest

[tsebastiani]

/funtest

[tsebastiani]

/funtest

[chaitanyaenr]

LGTM!

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: chaitanyaenr, tsebastiani

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment