Add support to run io hog scenario

This commit also enables the scenario in prow.

README.md

@@ -14,6 +14,7 @@ Scenario   | Description | Working
 [time skew](docs/time-scenarios.md) | Skews the time and date | :heavy_check_mark: |
 [Node cpu hog](docs/node-cpu-hog.md) | Hogs CPU on the targeted nodes | :heavy_check_mark: |
 [Node memory hog](docs/node-memory-hog.md) | Hogs memory on the targeted nodes | :heavy_check_mark:  |
+[Node IO hog](docs/node-io-hog.md) | Hogs io on the targeted nodes | :heavy_check_mark: |
 [Namespace failures](docs/namespace-scenarios.md) | Fails the components in a namespace by deleting it | :heavy_check_mark: | 
 [Application outages](docs/application-outages.md) | Isolates application Ingress/Egress traffic to observe the impact on dependent applications and recovery/initialization timing | :heavy_check_mark: |
 [Power Outages](docs/power-outages.md) | Shuts down the cluster for the specified duration and turns it back on to check the cluster health | :heavy_check_mark: |

docker-compose.yaml

@@ -50,6 +50,11 @@ services:
         context: ./
         dockerfile: ./node-memory-hog/Dockerfile
       image: quay.io/redhat-chaos/krkn-hub:node-memory-hog
+    node-io-hog:
+      build:
+        context: ./
+        dockerfile: ./node-io-hog/Dockerfile
+      image: quay.io/redhat-chaos/krkn-hub:node-io-hog
     namespace-scenarios:
       build:
         context: ./

docs/node-cpu-hog.md

@@ -40,7 +40,7 @@ NODE_SELECTORS | Node selectors where the scenario containers will be scheduled
 
 **NOTE** In case of using custom metrics profile or alerts profile when `CAPTURE_METRICS` or `ENABLE_ALERTS` is enabled, mount the metrics profile from the host on which the container is run using podman/docker under `/root/kraken/config/metrics-aggregated.yaml` and `/root/kraken/config/alerts`. For example:
 ```
-$ podman run --name=<container_name> --net=host --env-host=true -v <path-to-custom-metrics-profile>:/root/kraken/config/metrics-aggregated.yaml -v <path-to-custom-alerts-profile>:/root/kraken/config/alerts -v <path-to-kube-config>:/root/.kube/config:Z -d quay.io/redhat-chaos/krkn-hub:container-scenarios
+$ podman run --name=<container_name> --net=host --env-host=true -v <path-to-custom-metrics-profile>:/root/kraken/config/metrics-aggregated.yaml -v <path-to-custom-alerts-profile>:/root/kraken/config/alerts -v <path-to-kube-config>:/root/.kube/config:Z -d quay.io/redhat-chaos/krkn-hub:node-cpu-hog
 ```
 
 #### Demo

docs/node-io-hog.md

@@ -0,0 +1,45 @@
+### Node IO hog scenario
+This scenario hogs the IO on the specified node on a Kubernetes/OpenShift cluster for a specified duration. For more information refer the following [documentation](https://github.com/redhat-chaos/krkn/blob/main/docs/arcaflow_scenarios/io_hog.md).
+
+#### Run
+If enabling [Cerberus](https://github.com/redhat-chaos/krkn#kraken-scenario-passfail-criteria-and-report) to monitor the cluster and pass/fail the scenario post chaos, refer [docs](https://github.com/redhat-chaos/krkn-hub/tree/main/docs/cerberus.md). Make sure to start it before injecting the chaos and set `CERBERUS_ENABLED` environment variable for the chaos injection container to autoconnect.
+
+```
+$ podman run --name=<container_name> --net=host --env-host=true -v <path-to-kube-config>:/root/.kube/config:Z -d quay.io/redhat-chaos/krkn-hub:node-io-hog
+$ podman logs -f <container_name or container_id> # Streams Kraken logs
+$ podman inspect <container-name or container-id> --format "{{.State.ExitCode}}" # Outputs exit code which can considered as pass/fail for the scenario
+```
+
+```
+$ docker run $(./get_docker_params.sh) --name=<container_name> --net=host -v <path-to-kube-config>:/root/.kube/config:Z -d quay.io/redhat-chaos/krkn-hub:node-io-hog
+OR 
+$ docker run -e <VARIABLE>=<value> --net=host -v <path-to-kube-config>:/root/.kube/config:Z -d quay.io/redhat-chaos/krkn-hub:node-io-hog
+
+$ docker logs -f <container_name or container_id> # Streams Kraken logs
+$ docker inspect <container-name or container-id> --format "{{.State.ExitCode}}" # Outputs exit code which can considered as pass/fail for the scenario
+```
+
+
+#### Supported parameters
+
+The following environment variables can be set on the host running the container to tweak the scenario/faults being injected:
+
+ex.) 
+`export <parameter_name>=<value>`
+
+See list of variables that apply to all scenarios [here](all_scenarios_env.md) that can be used/set in addition to these scenario specific variables
+
+Parameter               | Description                                                           | Default
+----------------------- | -----------------------------------------------------------------     | ------------------------------------                   |
+TOTAL_CHAOS_DURATION    | Set chaos duration (in sec) as desired                                | 180                                  |
+IO_BLOCK_SIZE | string size of each write in bytes. Size can be from 1 byte to 4m | 1m |
+IO_WORKERS | Number of stressorts | 5 |
+IO_WRITE_BYTES | string writes N bytes for each hdd process. The size can be expressed as % of free space on the file system or in units of Bytes, KBytes, MBytes and GBytes using the suffix b, k, m or g | 10m |
+NAMESPACE | Namespace where the scenario container will be deployed | default |
+NODE_SELECTORS | Node selectors where the scenario containers will be scheduled in the format "`<selector>=<value>`". __NOTE__: This value can be specified as a list of node selectors separated by "`;`". Will be instantiated a container per each node selector with the same scenario options. This option is meant to run one or more stress scenarios simultaneously on different nodes, kubernetes will schedule the pods on the target node accordingly with the selector specified. Specifying the same selector multiple times will  instantiate as many scenario container as the number of times the selector is specified on the same node| "" |                             |
+
+
+**NOTE** In case of using custom metrics profile or alerts profile when `CAPTURE_METRICS` or `ENABLE_ALERTS` is enabled, mount the metrics profile from the host on which the container is run using podman/docker under `/root/kraken/config/metrics-aggregated.yaml` and `/root/kraken/config/alerts`. For example:
+```
+$ podman run --name=<container_name> --net=host --env-host=true -v <path-to-custom-metrics-profile>:/root/kraken/config/metrics-aggregated.yaml -v <path-to-custom-alerts-profile>:/root/kraken/config/alerts -v <path-to-kube-config>:/root/.kube/config:Z -d quay.io/redhat-chaos/krkn-hub:node-io-hog
+```

docs/node-memory-hog.md

@@ -40,7 +40,7 @@ NODE_SELECTORS | Node selectors where the scenario containers will be scheduled
 
 **NOTE** In case of using custom metrics profile or alerts profile when `CAPTURE_METRICS` or `ENABLE_ALERTS` is enabled, mount the metrics profile from the host on which the container is run using podman/docker under `/root/kraken/config/metrics-aggregated.yaml` and `/root/kraken/config/alerts`. For example:
 ```
-$ podman run --name=<container_name> --net=host --env-host=true -v <path-to-custom-metrics-profile>:/root/kraken/config/metrics-aggregated.yaml -v <path-to-custom-alerts-profile>:/root/kraken/config/alerts -v <path-to-kube-config>:/root/.kube/config:Z -d quay.io/redhat-chaos/krkn-hub:container-scenarios
+$ podman run --name=<container_name> --net=host --env-host=true -v <path-to-custom-metrics-profile>:/root/kraken/config/metrics-aggregated.yaml -v <path-to-custom-alerts-profile>:/root/kraken/config/alerts -v <path-to-kube-config>:/root/.kube/config:Z -d quay.io/redhat-chaos/krkn-hub:node-memory-hog
 ```
 
 #### Demo

node-io-hog/Dockerfile

@@ -0,0 +1,21 @@
+# Dockerfile for kraken
+
+FROM quay.io/redhat-chaos/krkn:latest
+
+MAINTAINER Red Hat Chaos Engineering Team
+
+ENV KUBECONFIG /root/.kube/config
+
+# Install dependencies
+RUN yum install -y which wget
+RUN wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/bin/yq && chmod +x /usr/bin/yq
+# Copy configurations
+COPY metrics_config.yaml.template /root/kraken/config/kube_burner.yaml.template
+COPY config.yaml.template /root/kraken/config/config.yaml.template
+COPY node-io-hog/env.sh /root/env.sh
+COPY env.sh /root/main_env.sh
+COPY node-io-hog/run.sh /root/run.sh
+COPY node-io-hog/input.yaml.template /root/kraken/scenarios/arcaflow/io-hog/input.yaml.template
+COPY common_run.sh /root/common_run.sh
+
+ENTRYPOINT /root/run.sh

node-io-hog/README.md

@@ -0,0 +1,3 @@
+# Node IO Hog Scenario Docs
+
+See [doc](https://github.com/redhat-chaos/krkn-hub/blob/main/docs/node-io-hog.md) for how to run and all the variables listed

node-io-hog/env.sh

@@ -0,0 +1,12 @@
+#!/bin/bash
+
+# Vars and respective defaults
+export TOTAL_CHAOS_DURATION=${TOTAL_CHAOS_DURATION:="180"}
+export IO_BLOCK_SIZE=${IO_BLOCK_SIZE:="1m"}
+export IO_WORKERS=${IO_WORKERS:="5"}
+export IO_WRITE_BYTES=${IO_WRITE_BYTES:="10m"}
+export NAMESPACE=${NAMESPACE:="default"}
+export NODE_SELECTORS=${NODE_SELECTORS:=""}
+
+export SCENARIO_TYPE=${SCENARIO_TYPE:=arcaflow_scenarios}
+export SCENARIO_FILE=${SCENARIO_FILE:=scenarios/arcaflow/io-hog/input.yaml}

node-io-hog/input.yaml.template

@@ -0,0 +1,13 @@
+duration: $TOTAL_CHAOS_DURATION
+io_block_size: $IO_BLOCK_SIZE
+io_workers: $IO_WORKERS
+io_write_bytes: $IO_WRITE_BYTES
+kubeconfig: ""
+namespace: $NAMESPACE
+node_selector:
+  $SELECTOR: $SELECTOR_VALUE
+target_pod_folder: /hog-data
+target_pod_volume:
+  hostPath:
+    path: /tmp
+  name: node-volume

node-io-hog/run.sh

@@ -0,0 +1,30 @@
+#!/bin/bash
+
+set -ex
+
+ROOT_FOLDER="/root"
+KRAKEN_FOLDER="$ROOT_FOLDER/kraken"
+SCENARIO_FOLDER="$KRAKEN_FOLDER/scenarios/arcaflow/io-hog"
+
+# Source env.sh to read all the vars
+source $ROOT_FOLDER/main_env.sh
+source $ROOT_FOLDER/env.sh
+
+source $ROOT_FOLDER/common_run.sh
+
+setup_arcaflow_env "$SCENARIO_FOLDER"
+# Substitute config with environment vars defined
+envsubst < $KRAKEN_FOLDER/config/config.yaml.template > $KRAKEN_FOLDER/config/io_config.yaml
+checks
+config_setup
+
+# Run Kraken
+cd $KRAKEN_FOLDER
+
+cat config/io_config.yaml
+
+cat scenarios/arcaflow/io-hog/input.yaml
+
+python3.9 run_kraken.py --config=config/io_config.yaml
+
+

prow/io-hog/prow_run.sh

@@ -0,0 +1,43 @@
+#!/bin/bash
+
+set -ex
+
+ls
+
+# Source env.sh to read all the vars
+source env.sh
+
+export KUBECONFIG=$KRKN_KUBE_CONFIG
+# Move kraken from root dir to tmp to avoid permissions issues in prow until fixed in base image
+cp -r /root/kraken /tmp/kraken
+krkn_loc=/tmp/kraken
+SCENARIO_FOLDER="$krkn_loc/scenarios/arcaflow/cpu-hog"
+
+# cluster details
+echo "Printing cluster details"
+oc version
+cat $KRKN_KUBE_CONFIG
+oc config view
+echo "Printing node info"
+for node in $(oc get nodes | awk 'NR!=1{print $1}'); do oc get node/$node -o yaml; done
+
+# Copy config to kraken
+cp node-io-hog/input.yaml.template $SCENARIO_FOLDER/input.yaml.template
+
+source node-io-hog/env.sh
+source env.sh
+source common_run.sh
+
+setup_arcaflow_env "$SCENARIO_FOLDER"
+checks
+
+# Substitute config with environment vars defined
+#envsubst < node-io-hog/input.yaml.template> node-io-hog/io_hog_scenario.yaml
+#export SCENARIO_FILE="node-io-hog/io_hog_scenario.yaml"
+export SCENARIO_FILE="$SCENARIO_FOLDER/input.yaml"
+envsubst < config.yaml.template > io_hog_config.yaml
+
+# Run Kraken
+cat io_hog_config.yaml
+cat $SCENARIO_FOLDER/input.yaml
+python3.9 $krkn_loc/run_kraken.py --config=io_hog_config.yaml