Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tool and workflow to generate and regenerate tutorials from documentation #7

Merged
merged 47 commits into from
Jun 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
163c9b4
Start generation
jdbaldry Jun 17, 2024
1d44bf1
Demonstrate with regexps
jdbaldry Jun 18, 2024
668b21c
Start rendering Markdown from Markdown
jdbaldry Jun 18, 2024
fbd3bd0
Remove between ignore markers
jdbaldry Jun 18, 2024
df204a9
Add intro transformer
jdbaldry Jun 18, 2024
bc5b88e
Add exec and copy transformations
jdbaldry Jun 18, 2024
2b4501e
Transform localhost and partial URLs
jdbaldry Jun 18, 2024
8da17aa
Consolidate action transformers
jdbaldry Jun 19, 2024
ce27fcb
Replace figure shortcodes with images
jdbaldry Jun 19, 2024
5bb7252
Ensure heading hierarchy starts with H1
jdbaldry Jun 19, 2024
f9e7d54
Remove vendored Goldmark HTML renderer implementation
jdbaldry Jun 19, 2024
2e91509
Lint
jdbaldry Jun 19, 2024
7443ce9
Include path
jdbaldry Jun 19, 2024
9f6fa8a
Update generated tutorial
jdbaldry Jun 19, 2024
a0d9835
Simplify and make ignore transformer more efficient
jdbaldry Jun 19, 2024
c32009f
Add include directive
jdbaldry Jun 19, 2024
a6e8809
Regenerate with include directive support
jdbaldry Jun 19, 2024
48d70d0
Add exec action
jdbaldry Jun 19, 2024
d7b0d62
Amended single quotes to see if issue fixes
Jayclifford345 Jun 19, 2024
e054d32
remove indentation to try and fix markdown issue
Jayclifford345 Jun 19, 2024
38c69d0
Copy paste reformat
Jayclifford345 Jun 19, 2024
6a46b65
added missng bash and indented code blocks
Jayclifford345 Jun 19, 2024
273c8c5
Testing supports fix
Jayclifford345 Jun 19, 2024
033aeef
Add action directive to all inline code
jdbaldry Jun 19, 2024
71756fd
Regenerate
jdbaldry Jun 20, 2024
4c4084d
Cleanup
jdbaldry Jun 20, 2024
341bdea
Add command line argument handling
jdbaldry Jun 20, 2024
72660e5
Add workflow to regenerate tutorials
jdbaldry Jun 20, 2024
a40daae
Merge remote-tracking branch 'origin/main' into jdb/2024-06-killercod…
jdbaldry Jun 20, 2024
3219c62
Reuse blank workflow for testing
jdbaldry Jun 20, 2024
28bc61e
Fix checkout
jdbaldry Jun 20, 2024
6c396ea
Fix path to go.mod
jdbaldry Jun 20, 2024
7c6e186
Fix repository quoting
jdbaldry Jun 20, 2024
2119fcf
Fix typo
jdbaldry Jun 20, 2024
d5dafab
Fix working directory for build
jdbaldry Jun 20, 2024
8251754
Fix path to transformer binary
jdbaldry Jun 20, 2024
29efb1b
Make script executable
jdbaldry Jun 20, 2024
9b33063
Lint
jdbaldry Jun 20, 2024
718f288
Add GITHUB_TOKEN environment variable
jdbaldry Jun 20, 2024
f03fce5
Fill in the subject
jdbaldry Jun 20, 2024
bd68bc7
Ignore transformer binary
jdbaldry Jun 20, 2024
a99b4bd
Use empty body
jdbaldry Jun 20, 2024
a7e73c8
Write index.json and intro file if present
jdbaldry Jun 20, 2024
5ad8c79
Add preprocessing substitution step
jdbaldry Jun 20, 2024
c11e350
Regenerate
jdbaldry Jun 20, 2024
7a31820
Add documentation
jdbaldry Jun 20, 2024
978813a
Document step
jdbaldry Jun 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 29 additions & 30 deletions .github/workflows/blank.yml
Original file line number Diff line number Diff line change
@@ -1,36 +1,35 @@
# This is a basic workflow to help you get started with Actions

name: CI

# Controls when the workflow will run
name: Regenerate tutorials
on:
# Triggers the workflow on push or pull request events but only for the "main" branch
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]

# Allows you to run this workflow manually from the Actions tab
schedule:
- cron: '0 7 * * 1-5'
workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
main:
if: github.repository == 'grafana/killercoda'
runs-on: ubuntu-latest

# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v4

# Runs a single command using the runners shell
- name: Run a one-line script
run: echo Hello, world!

# Runs a set of commands using the runners shell
- name: Run a multi-line script
run: |
echo Add other actions to build,
echo test, and deploy your project.
with:
repository: grafana/loki
# Change to `main` after this branch is merged.
ref: jdb/2024-06-killercoda-migration
path: loki
- uses: actions/checkout@v4
with:
ref: jdb/2024-06-killercoda-migration
path: killercoda
- uses: actions/setup-go@v5
with:
go-version-file: killercoda/tools/transformer/go.mod
- run: go build ./
working-directory: killercoda/tools/transformer
- run: >
./transformer
"${GITHUB_WORKSPACE}/loki/docs/sources/get-started/quick-start.md"
"${GITHUB_WORKSPACE}/killercoda/loki/loki-quickstart"
working-directory: killercoda/tools/transformer
- run: ./scripts/manage-pr.bash
env:
GH_TOKEN: ${{ github.token }}
shell: bash
working-directory: killercoda
39 changes: 19 additions & 20 deletions loki/loki-quickstart/index.json
Original file line number Diff line number Diff line change
@@ -1,24 +1,23 @@
{
"title": "Loki Quickstart Demo",
"description": "This sandbox provides an online enviroment for testing the Loki quickstart demo.",
"details": {
"intro": {
"text": "intro.md"
"title": "Loki Quickstart Demo",
"description": "This sandbox provides an online enviroment for testing the Loki quickstart demo.",
"details": {
"intro": {
"text": "intro.md"
},
"steps": [
{
"text": "step1.md"
},
"steps": [
{
"text": "step1.md"
},
{
"text": "step2.md"
}
],
"finish": {
"text": "finished.md"
{
"text": "step2.md"
}
],
"finish": {
"text": "finished.md"
}
,
"backend": {
"imageid": "ubuntu"
}
}
},
"backend": {
"imageid": "ubuntu"
}
}
19 changes: 12 additions & 7 deletions loki/loki-quickstart/intro.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,24 @@
![Loki Quickstart](../../assets/loki-ile.png)


# Quickstart to run Loki locally

If you want to experiment with Loki, you can run Loki locally using the Docker Compose file that ships with Loki. It runs Loki in a [monolithic deployment](https://grafana.com/docs/loki/latest/get-started/deployment-modes/#monolithic-mode) mode and includes a sample application to generate logs.
If you want to experiment with Loki, you can run Loki locally using the Docker Compose file that ships with Loki. It runs Loki in a [monolithic deployment](https://grafana.com/docs/loki/latest/get-started/deployment-modes/#monolithic-mode) mode and includes a sample application to generate logs.

The Docker Compose configuration instantiates the following components, each in its own container:

- **flog** a sample application which generates log lines. [flog](https://github.com/mingrammer/flog) is a log generator for common log formats.
- **flog** a sample application which generates log lines.
[flog](https://github.com/mingrammer/flog) is a log generator for common log formats.

- **Grafana Alloy** which scrapes the log lines from flog, and pushes them to Loki through the gateway.
- **Gateway** (NGINX) which receives requests and redirects them to the appropriate container based on the request's URL.

- **Gateway** (nginx) which receives requests and redirects them to the appropriate container based on the request’s URL.

- One Loki **read** component (Query Frontend, Querier).

- One Loki **write** component (Distributor, Ingester).

- One Loki **backend** component (Index Gateway, Compactor, Ruler, Bloom Compactor (Experimental), Bloom Gateway (Experimental)).

- **Minio** an S3-compatible object store which Loki uses to store its index and chunks.

- **Grafana** which provides visualization of the log lines captured within Loki.

![Loki Quickstart](https://grafana.com/media/docs/loki/get-started-flog-v3.png)
![Getting started sample application](https://grafana.com/media/docs/loki/get-started-flog-v3.png)
75 changes: 40 additions & 35 deletions loki/loki-quickstart/step1.md
Original file line number Diff line number Diff line change
@@ -1,49 +1,54 @@

# Step 1: Installing Loki and collecting sample logs
# Install Loki and collecting sample logs

**To install Loki locally, follow these steps:**

1. Create a directory called `evaluate-loki` for the demo environment. Make `evaluate-loki` your current working directory:
1. Create a directory called `evaluate-loki`{{copy}} for the demo environment.
Make `evaluate-loki`{{copy}} your current working directory:

```bash
mkdir evaluate-loki
cd evaluate-loki
```{{exec}}

1. Download `loki-config.yaml`{{copy}}, `alloy-local-config.yaml`{{copy}}, and `docker-compose.yaml`{{copy}}:

```bash
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/loki-config.yaml -O loki-config.yaml
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/alloy-local-config.yaml -O alloy-local-config.yaml
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/docker-compose.yaml -O docker-compose.yaml
```{{exec}}

1. Deploy the sample Docker image.

```bash
mkdir evaluate-loki
cd evaluate-loki
```{{execute}}
With `evaluate-loki`{{copy}} as the current working directory, start the demo environment using `docker compose`{{copy}}:

2. Download `loki-config.yaml`, `alloy-local-config.yaml`, and `docker-compose.yaml`:
```bash
docker-compose up -d
```{{exec}}

```bash
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/loki-config.yaml -O loki-config.yaml
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/alloy-local-config.yaml -O alloy-local-config.yaml
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/docker-compose.yaml -O docker-compose.yaml
```{{execute}}

3. Deploy the sample Docker image.
At the end of the command, you should see something similar to the following:

With `evaluate-loki` as the current working directory, start the demo environment using `docker compose`:
```console
Creating evaluate-loki_flog_1 ... done
Creating evaluate-loki_minio_1 ... done
Creating evaluate-loki_read_1 ... done
Creating evaluate-loki_write_1 ... done
Creating evaluate-loki_gateway_1 ... done
Creating evaluate-loki_alloy_1 ... done
Creating evaluate-loki_grafana_1 ... done
Creating evaluate-loki_backend_1 ... done
```

```bash
docker-compose up -d
```{{execute}}

You should see something similar to the following:
1. (Optional) Verify that the Loki cluster is up and running.

```bash
✔ Network evaluate-loki_loki Created 0.1s
✔ Container evaluate-loki-minio-1 Started 0.6s
✔ Container evaluate-loki-flog-1 Started 0.6s
✔ Container evaluate-loki-backend-1 Started 0.8s
✔ Container evaluate-loki-write-1 Started 0.8s
✔ Container evaluate-loki-read-1 Started 0.8s
✔ Container evaluate-loki-gateway-1 Started 1.1s
✔ Container evaluate-loki-grafana-1 Started 1.4s
✔ Container evaluate-loki-alloy-1 Started 1.4s
```
- The read component returns `ready`{{copy}} when you browse to [http://localhost:3101/ready]({{TRAFFIC_HOST1_3101}}/ready).
The message `Query Frontend not ready: not ready: number of schedulers this worker is connected to is 0`{{copy}} shows until the read component is ready.

4. (Optional) Verify that the Loki cluster is up and running.
- The read component returns `ready` when you point a web browser at [http://localhost:3101/ready]({{TRAFFIC_HOST1_3101}}/ready). The message `Query Frontend not ready: not ready: number of schedulers this worker is connected to is 0` will show prior to the read component being ready.
- The write component returns `ready` when you point a web browser at [http://localhost:3102/ready]({{TRAFFIC_HOST1_3102}}/ready). The message `Ingester not ready: waiting for 15s after being ready` will show prior to the write component being ready.
- The write component returns `ready`{{copy}} when you browse to [http://localhost:3102/ready]({{TRAFFIC_HOST1_3102}}/ready).
The message `Ingester not ready: waiting for 15s after being ready`{{copy}} shows until the write component is ready.

5. (Optional) Verify that Grafana Alloy is running.
- Grafana Alloy's UI can be accessed at [http://localhost:12345]({{TRAFFIC_HOST1_12345}}).
1. (Optional) Verify that Grafana Alloy is running.

- You can access the Grafana Alloy UI at [http://localhost:12345]({{TRAFFIC_HOST1_12345}}).
Loading