Release 1.6.0 (#2502)

* Updating changelog and release notes for 1.6.0

* updating version in documentation

README.md

@@ -29,9 +29,9 @@ Loki differs from Prometheus by focusing on logs instead of metrics, and deliver
 
 ## Getting started
 
-* [Installing Loki](https://github.com/grafana/loki/tree/v1.5.0/docs/installation/README.md)
-* [Installing Promtail](https://github.com/grafana/loki/tree/v1.5.0/docs/clients/promtail/installation.md)
-* [Getting Started Guide](https://github.com/grafana/loki/tree/v1.5.0/docs/getting-started/README.md)
+* [Installing Loki](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/installation/README.md)
+* [Installing Promtail](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/clients/promtail/installation.md)
+* [Getting Started Guide](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/getting-started/README.md)
 
 ## Upgrading
 
@@ -40,6 +40,7 @@ Loki differs from Prometheus by focusing on logs instead of metrics, and deliver
 ### Documentation
 
 * [master](./docs/README.md)
+* [v1.6.0](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/_index.md)
 * [v1.5.0](https://github.com/grafana/loki/tree/v1.5.0/docs/README.md)
 * [v1.4.1](https://github.com/grafana/loki/tree/v1.4.1/docs/README.md)
 * [v1.4.0](https://github.com/grafana/loki/tree/v1.4.0/docs/README.md)
@@ -48,18 +49,18 @@ Loki differs from Prometheus by focusing on logs instead of metrics, and deliver
 * [v1.1.0](https://github.com/grafana/loki/tree/v1.1.0/docs/README.md)
 * [v1.0.0](https://github.com/grafana/loki/tree/v1.0.0/docs/README.md)
 
-Commonly used sections (from the latest release v1.5.0):
-
-- [API documentation](https://github.com/grafana/loki/tree/v1.5.0/docs/api.md) for alternative ways of getting logs into Loki.
-- [Labels](https://github.com/grafana/loki/blob/master/docs/getting-started/labels.md)
-- [Operations](https://github.com/grafana/loki/tree/v1.5.0/docs/operations) for important aspects of running Loki.
-- [Promtail](https://github.com/grafana/loki/tree/v1.5.0/docs/clients/promtail) is an agent which can tail your log files and push them to Loki.
-- [Pipelines](https://github.com/grafana/loki/tree/v1.5.0/docs/clients/promtail/pipelines.md) for detailed log processing pipeline documentation
-- [Docker Logging Driver](https://github.com/grafana/loki/tree/v1.5.0/docs/clients/docker-driver) is a docker plugin to send logs directly to Loki from Docker containers.
-- [LogCLI](https://github.com/grafana/loki/tree/v1.5.0/docs/getting-started/logcli.md) on how to query your logs without Grafana.
-- [Loki Canary](https://github.com/grafana/loki/tree/v1.5.0/docs/operations/loki-canary.md) for monitoring your Loki installation for missing logs.
-- [Troubleshooting](https://github.com/grafana/loki/tree/v1.5.0/docs/getting-started/troubleshooting.md) for help around frequent error messages.
-- [Loki in Grafana](https://github.com/grafana/loki/tree/v1.5.0/docs/getting-started/grafana.md) for how to set up a Loki datasource in Grafana and query your logs.
+Commonly used sections (from the latest release v1.6.0):
+
+- [API documentation](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/api/_index.md) for alternative ways of getting logs into Loki.
+- [Labels](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/getting-started/labels.md)
+- [Operations](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/operations/_index.md) for important aspects of running Loki.
+- [Promtail](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/clients/promtail/_index.md) is an agent which can tail your log files and push them to Loki.
+- [Pipelines](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/clients/promtail/pipelines.md) for detailed log processing pipeline documentation
+- [Docker Logging Driver](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/clients/docker-driver/_index.md) is a docker plugin to send logs directly to Loki from Docker containers.
+- [LogCLI](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/getting-started/logcli.md) on how to query your logs without Grafana.
+- [Loki Canary](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/operations/loki-canary.md) for monitoring your Loki installation for missing logs.
+- [Troubleshooting](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/getting-started/troubleshooting.md) for help around frequent error messages.
+- [Loki in Grafana](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/getting-started/grafana.md) for how to set up a Loki datasource in Grafana and query your logs.
 
 ## Getting Help
 

docs/sources/clients/aws/ec2/_index.md

@@ -91,7 +91,7 @@ Next we'll download, install and give executable right to [Promtail](../../promt
 
 ```bash
 mkdir /opt/promtail && cd /opt/promtail
-curl -O -L "https://github.com/grafana/loki/releases/download/v1.5.0/promtail-linux-amd64.zip"
+curl -O -L "https://github.com/grafana/loki/releases/download/v1.6.0/promtail-linux-amd64.zip"
 unzip "promtail-linux-amd64.zip"
 chmod a+x "promtail-linux-amd64"
 ```
@@ -231,7 +231,7 @@ Jul 08 15:48:57 ip-172-31-45-69.us-east-2.compute.internal systemd[1]: Started P
 Jul 08 15:48:57 ip-172-31-45-69.us-east-2.compute.internal systemd[1]: Starting Promtail...
 Jul 08 15:48:57 ip-172-31-45-69.us-east-2.compute.internal promtail-linux-amd64[2732]: level=warn ts=2020-07-08T15:48:57.559085451Z caller=filetargetmanager.go:98 msg="WARNING!!! entry_parser config is deprecated, please change to pipeline_stages"
 Jul 08 15:48:57 ip-172-31-45-69.us-east-2.compute.internal promtail-linux-amd64[2732]: level=info ts=2020-07-08T15:48:57.559869071Z caller=server.go:179 http=[::]:3100 grpc=[::]:35127 msg="server listening on addresses"
-Jul 08 15:48:57 ip-172-31-45-69.us-east-2.compute.internal promtail-linux-amd64[2732]: level=info ts=2020-07-08T15:48:57.56029474Z caller=main.go:67 msg="Starting Promtail" version="(version=1.5.0, branch=HEAD, revision=12c7eab8)"
+Jul 08 15:48:57 ip-172-31-45-69.us-east-2.compute.internal promtail-linux-amd64[2732]: level=info ts=2020-07-08T15:48:57.56029474Z caller=main.go:67 msg="Starting Promtail" version="(version=1.6.0, branch=HEAD, revision=12c7eab8)"
 ```
 
 You can now verify in Grafana that Loki has correctly received your instance logs by using the [LogQL](../../../logql/) query `{zone="us-east-2"}`.

docs/sources/clients/aws/ecs/_index.md

@@ -98,7 +98,7 @@ curl https://raw.githubusercontent.com/grafana/loki/master/docs/aws/ecs/ecs-task
 ```json
  {
     "essential": true,
-    "image": "grafana/fluent-bit-plugin-loki:1.5.0-amd64",
+    "image": "grafana/fluent-bit-plugin-loki:1.6.0-amd64",
     "name": "log_router",
     "firelensConfiguration": {
         "type": "fluentbit",

docs/sources/clients/aws/ecs/ecs-task.json

@@ -2,7 +2,7 @@
     "containerDefinitions": [
         {
             "essential": true,
-            "image": "grafana/fluent-bit-plugin-loki:1.5.0-amd64",
+            "image": "grafana/fluent-bit-plugin-loki:1.6.0-amd64",
             "name": "log_router",
             "firelensConfiguration": {
                 "type": "fluentbit",

docs/sources/clients/aws/eks/values.yaml

@@ -17,7 +17,7 @@ initContainer:
 
 image:
   repository: grafana/promtail
-  tag: 1.5.0
+  tag: 1.6.0
   pullPolicy: IfNotPresent
   ## Optionally specify an array of imagePullSecrets.
   ## Secrets must be manually created in the namespace.

docs/sources/clients/promtail/installation.md

@@ -15,7 +15,7 @@ Every release includes binaries for Promtail which can be found on the
 
 ```bash
 # modify tag to most recent version
-docker pull grafana/promtail:1.5.0
+docker pull grafana/promtail:1.6.0
 ```
 
 ## Helm

docs/sources/configuration/_index.md

@@ -1101,11 +1101,16 @@ redis:
   [enable_tls: <boolean> | default = false]
 
 fifocache:
-  # Number of entries to cache in-memory.
-  # CLI flag: -<prefix>.fifocache.size
-  [size: <int> | default = 0]
+  # Maximum memory size of the cache in bytes. A unit suffix (KB, MB, GB) may be
+  # applied.
+  # CLI flag: -<prefix>.fifocache.max-size-bytes
+  [max_size_bytes: <string> | default = ""]
 
-  # The expiry duration for the in-memory cache.
+  # Maximum number of entries in the cache.
+  # CLI flag: -<prefix>.fifocache.max-size-items
+  [max_size_items: <int> | default = 0]
+
+  # The expiry duration for the cache.
   # CLI flag: -<prefix>.fifocache.duration
   [validity: <duration> | default = 0s]
 ```

docs/sources/installation/docker.md

@@ -18,28 +18,28 @@ For production, we recommend installing with Tanka or Helm.
 Copy and paste the commands below into your command line.
 
 ```bash
-wget https://raw.githubusercontent.com/grafana/loki/v1.5.0/cmd/loki/loki-local-config.yaml -O loki-config.yaml
-docker run -v $(pwd):/mnt/config -p 3100:3100 grafana/loki:1.5.0 -config.file=/mnt/config/loki-config.yaml
-wget https://raw.githubusercontent.com/grafana/loki/v1.5.0/cmd/promtail/promtail-docker-config.yaml -O promtail-config.yaml
-docker run -v $(pwd):/mnt/config -v /var/log:/var/log grafana/promtail:1.5.0 -config.file=/mnt/config/promtail-config.yaml
+wget https://raw.githubusercontent.com/grafana/loki/v1.6.0/cmd/loki/loki-local-config.yaml -O loki-config.yaml
+docker run -v $(pwd):/mnt/config -p 3100:3100 grafana/loki:1.6.0 -config.file=/mnt/config/loki-config.yaml
+wget https://raw.githubusercontent.com/grafana/loki/v1.6.0/cmd/promtail/promtail-docker-config.yaml -O promtail-config.yaml
+docker run -v $(pwd):/mnt/config -v /var/log:/var/log grafana/promtail:1.6.0 -config.file=/mnt/config/promtail-config.yaml
 ```
 
 When finished, `loki-config.yaml` and `promtail-config.yaml` are downloaded in the directory you chose. Docker containers are running Loki and Promtail using those config files.
 
 Navigate to http://localhost:3100/metrics to view the metrics and http://localhost:3100/ready for readiness.
 
-As of v1.5.0, image is configured to run by default as user loki with  UID `10001` and GID `10001`. You can use a different user, specially if you are using bind mounts, by specifying the UID with a `docker run` command and using `--user=UID` with numeric UID suited to your needs.
+As of v1.6.0, image is configured to run by default as user loki with  UID `10001` and GID `10001`. You can use a different user, specially if you are using bind mounts, by specifying the UID with a `docker run` command and using `--user=UID` with numeric UID suited to your needs.
 
 **Windows**
 
 Copy and paste the commands below into your terminal. Note that you will need to replace the `<placeholders>` in the commands with your local path.
 
 ```bash
 cd "<local-path>"
-wget https://raw.githubusercontent.com/grafana/loki/v1.5.0/cmd/loki/loki-local-config.yaml -O loki-config.yaml
-docker run -v <local-path>:/mnt/config -p 3100:3100 grafana/loki:1.5.0 --config.file=/mnt/config/loki-config.yaml
-wget https://raw.githubusercontent.com/grafana/loki/v1.5.0/cmd/promtail/promtail-docker-config.yaml -O promtail-config.yaml
-docker run -v <local-path>:/mnt/config -v /var/log:/var/log grafana/promtail:1.5.0 --config.file=/mnt/config/promtail-config.yaml
+wget https://raw.githubusercontent.com/grafana/loki/v1.6.0/cmd/loki/loki-local-config.yaml -O loki-config.yaml
+docker run -v <local-path>:/mnt/config -p 3100:3100 grafana/loki:1.6.0 --config.file=/mnt/config/loki-config.yaml
+wget https://raw.githubusercontent.com/grafana/loki/v1.6.0/cmd/promtail/promtail-docker-config.yaml -O promtail-config.yaml
+docker run -v <local-path>:/mnt/config -v /var/log:/var/log grafana/promtail:1.6.0 --config.file=/mnt/config/promtail-config.yaml
 ```
 
 When finished, `loki-config.yaml` and `promtail-config.yaml` are downloaded in the directory you chose. Docker containers are running Loki and Promtail using those config files.
@@ -51,6 +51,6 @@ Navigate to http://localhost:3100/metrics to view the output.
 Run the following commands in your command line. They work for Windows or Linux systems.
 
 ```bash
-wget https://raw.githubusercontent.com/grafana/loki/v1.5.0/production/docker-compose.yaml -O docker-compose.yaml
+wget https://raw.githubusercontent.com/grafana/loki/v1.6.0/production/docker-compose.yaml -O docker-compose.yaml
 docker-compose -f docker-compose.yaml up
 ```

docs/sources/operations/loki-canary.md

@@ -116,7 +116,7 @@ Loki Canary is also provided as a Docker container image:
 
 ```bash
 # change tag to the most recent release
-$ docker pull grafana/loki-canary:1.5.0
+$ docker pull grafana/loki-canary:1.6.0
 ```
 
 ### Kubernetes

docs/sources/operations/upgrade.md

@@ -12,23 +12,102 @@ On this page we will document any upgrade issues/gotchas/considerations we are a
 
 ## Master / Unreleased
 
-#### Documentation changes:
+## 1.6.0
 
-Configuration document has been re-orderd a bit and for all the config, corresponding `CLI` flag is provided.
+### IMPORTANT: Ksonnet Port Change and Removal of NET_BIND_SERVICE Capability from docker image
 
-S3 config now supports exapnded config. Example can be found here [s3_expanded_config](../configuration/examples.md#s3-expanded-config)
+In 1.5.0 we changed the Loki user to not run as root which created problems binding to port 80.
+To address this we updated the docker image to add the NET_BIND_SERVICE capability to the loki process
+which allowed Loki to bind to port 80 as a non root user, so long as the underlying system allowed that 
+linux capability.
 
-### New Ingester GRPC API special rollout procedure in microservices mode
+This has proved to be a problem for many reasons and in PR [2294](https://github.com/grafana/loki/pull/2294/files)
+the capability was removed.
 
-A new ingester GRPC API has been added allowing to speed up metric queries, to ensure a rollout without query errors make sure you upgrade all ingesters first.
+It is now no longer possible for the Loki to be started with a port less than 1024 with the published docker image.
+
+The default for Helm has always been port 3100, and Helm users should be unaffect unless they changed the default.
+
+**Ksonnet users however should closely check their configuration, in PR 2294 the loki port was changed from 80 to 3100**
+
+
+### IMPORTANT: If you run Loki in microservices mode, special rollout instructions
+
+A new ingester GRPC API has been added allowing to speed up metric queries, to ensure a rollout without query errors **make sure you upgrade all ingesters first.**
 Once this is done you can then proceed with the rest of the deployment, this is to ensure that queriers won't look for an API not yet available.
 
 If you roll out everything at once, queriers with this new code will attempt to query ingesters which may not have the new method on the API and queries will fail.
 
 This will only affect reads(queries) and not writes and only for the duration of the rollout.
 
+### IMPORTANT: Scrape config changes to both Helm and Ksonnet will affect labels created by Promtail
+
+PR [2091](https://github.com/grafana/loki/pull/2091) Makes several changes to the promtail scrape config:
+
+````
+This is triggered by https://github.com/grafana/jsonnet-libs/pull/261
+
+The above PR changes the instance label to be actually unique within
+a scrape config. It also adds a pod and a container target label
+so that metrics can easily be joined with metrics from cAdvisor, KSM,
+and the Kubelet.
+
+This commit adds the same to the Loki scrape config. It also removes
+the container_name label. It is the same as the container label
+and was already added to Loki previously. However, the
+container_name label is deprecated and has disappeared in K8s 1.16,
+so that it will soon become useless for direct joining.
+````
+
+TL;DR
+
+The following label have been changed in both the Helm and Ksonnet Promtail scrape configs:
+
+`instance` -> `pod`  
+`container_name` -> `container`
+
+
+### Experimental boltdb-shipper changes
+
+PR [2166](https://github.com/grafana/loki/pull/2166) now forces the index to have a period of exactly `24h`:
+
+Loki will fail to start with an error if the active schema or upcoming schema are not set to a period of `24h`
+
+You can add a new schema config like this:
+
+```yaml
+schema_config:
+  configs:
+    - from: 2020-01-01      <----- This is your current entry, date will be different
+      store: boltdb-shipper
+      object_store: aws
+      schema: v11
+      index:
+        prefix: index_
+        period: 168h
+    - from: [INSERT FUTURE DATE HERE]   <----- Add another entry, set a future date
+      store: boltdb-shipper
+      object_store: aws
+      schema: v11
+      index:
+        prefix: index_
+        period: 24h   <--- This must be 24h
+```
+If you are not on `schema: v11` this would be a good oportunity to make that change _in the new schema config_ also.
+
+**NOTE** If the current time in your timezone is after midnight UTC already, set the date one additional day forward.
+
+There was also a significant overhaul to how boltdb-shipper internals, this should not be visible to a user but as this 
+feature is experimental and under development bug are possible!
+
+The most noticeable change if you look in the storage, Loki no longer updates an existing file and instead creates a
+new index file every 15mins, this is an important move to make sure objects in the object store are immutable and
+will simplify future operations like compaction and deletion.
+
 ### Breaking CLI flags changes
 
+The following CLI flags where changed to improve consistency, they are not expected to be widely used
+
 ```diff
 - querier.query_timeout
 + querier.query-timeout