Merge pull request #40 from tum-gis/bump/release-3.0.1-2.1.2

Release candidate for 3.1.0-beta1

CHANGELOG.md

@@ -8,6 +8,20 @@ Versions are prefixed with `sddi-ckan-` due to usage of
 [chart-releaser-action](https://github.com/helm/chart-releaser-action).
 For releases `< 1.0.0` minor version step indicate breaking changes.
 
+## [sddi-ckan-3.1.0-beta1] - 2024-04-22
+
+## Added
+
+- ClamAV service for virus scanning of uploaded files. tum-gis/sddi-ckan-k8s#38
+- Ingress security headers. tum-gis/sddi-ckan-k8s#37
+
+### Changed
+
+- Limits and requests for the CKAN StatefulSet.
+- - Bump SDDI CKAN Image `2.0.0` --> `2.1.2`, see
+  [CHANGELOG](https://github.com/tum-gis/ckan-docker/blob/2.0.0/CHANGELOG.md) for more.
+  - Adds ClamAV extension.
+
 ## [sddi-ckan-3.0.0] - 2024-03-22
 
 ## Added
@@ -375,6 +389,7 @@ is displayed when navigating to the _Datasets_ view of CKAN.
 
 [Unreleased]: https://github.com/tum-gis/sddi-ckan-k8s/compare/sddi-ckan-3.0.0...HEAD
 
+[sddi-ckan-3.1.0-beta1]: https://github.com/tum-gis/sddi-ckan-k8s/compare/sddi-ckan-3.0.0...sddi-ckan-3.1.0-beta1
 [sddi-ckan-3.0.0]: https://github.com/tum-gis/sddi-ckan-k8s/compare/sddi-ckan-2.0.0...sddi-ckan-3.0.0
 [sddi-ckan-2.0.0]: https://github.com/tum-gis/sddi-ckan-k8s/compare/sddi-ckan-1.2.2...sddi-ckan-2.0.0
 [sddi-ckan-1.2.2]: https://github.com/tum-gis/sddi-ckan-k8s/compare/sddi-ckan-1.2.1...sddi-ckan-1.2.2

charts/sddi-ckan/Chart.yaml

@@ -10,8 +10,8 @@ sources:
   - https://www.asg.ed.tum.de/en/gis/projects/smart-district-data-infrastructure
   - https://github.com/tum-gis/ckan-docker
 
-version: 3.0.0
-appVersion: "2.0.0"
+version: "3.1.0-beta1"
+appVersion: "2.1.2"
 kubeVersion: ">= 1.23.0-0"
 
 maintainers:
@@ -48,6 +48,10 @@ dependencies:
     condition: cert-manager.enabled
     version: "^1"
     repository: https://charts.jetstack.io
+  - name: clamav
+    condition: clamav.enabled
+    version: "~2.8.0"
+    repository: https://wiremind.github.io/wiremind-helm-charts
   - name: common
     repository: oci://registry-1.docker.io/bitnamicharts
     tags:

charts/sddi-ckan/README.md

@@ -1,6 +1,6 @@
 # sddi-ckan
 
-![Version: 3.0.0](https://img.shields.io/badge/Version-3.0.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 2.0.0](https://img.shields.io/badge/AppVersion-2.0.0-informational?style=flat-square)
+![Version: 3.1.0-beta1](https://img.shields.io/badge/Version-3.1.0--beta1-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 2.1.2](https://img.shields.io/badge/AppVersion-2.1.2-informational?style=flat-square)
 
 Helm Chart for a SDDI enabled CKAN catalog. See [CHANGELOG](https://github.com/tum-gis/sddi-ckan-k8s/blob/main/CHANGELOG.md) for changes.
 
@@ -32,6 +32,7 @@ Kubernetes: `>= 1.23.0-0`
 |  | solr | * |
 | https://charts.jetstack.io | cert-manager(cert-manager) | ^1 |
 | https://kubernetes.github.io/ingress-nginx | ingress-nginx(ingress-nginx) | ^4 |
+| https://wiremind.github.io/wiremind-helm-charts | clamav | ~2.8.0 |
 | oci://registry-1.docker.io/bitnamicharts | common | 2.x.x |
 
 ## Values
@@ -40,6 +41,14 @@ Kubernetes: `>= 1.23.0-0`
 |-----|------|---------|-------------|
 | cert-manager.enabled | bool | `false` | Enable/disable cert-manager. |
 | certIssuer.enabled | bool | `true` | Enable/disable namespace Issuers for cert-manager. |
+| clamav.clamdConfig | string | `"###############\n# General\n###############\n\nDatabaseDirectory /data\nTemporaryDirectory /tmp\nLogTime yes\n# CUSTOM: Use pid file in tmp\nPidFile /tmp/clamd.pid\nLocalSocket /tmp/clamd.sock\n# CUSTOM: Set local socket group to defined group id\nLocalSocketGroup 2000\nTCPSocket 3310\nForeground yes\nStreamMaxLength 4000M\nLogVerbose yes\nBytecodeTimeout 1000\n\n###############\n# Results\n###############\n\nDetectPUA yes\nExcludePUA NetTool\nExcludePUA PWTool\nHeuristicAlerts yes\nBytecode yes\n\n###############\n# Scan\n###############\n\nScanPE yes\nDisableCertCheck yes\nScanELF yes\nAlertBrokenExecutables yes\nScanOLE2 yes\nScanPDF yes\nScanSWF yes\nScanMail yes\nPhishingSignatures yes\nPhishingScanURLs yes\nScanHTML yes\nScanArchive yes\n\n###############\n# Scan\n###############\n\nMaxScanSize 150M\nMaxFileSize 30M\nMaxRecursion 10\nMaxFiles 15000\nMaxEmbeddedPE 10M\nMaxHTMLNormalize 10M\nMaxHTMLNoTags 2M\nMaxScriptNormalize 5M\nMaxZipTypeRcg 1M\nMaxPartitions 128\nMaxIconsPE 200\nPCREMatchLimit 10000\nPCRERecMatchLimit 10000\n"` |  |
+| clamav.enabled | bool | `true` | Enable/disable [ClamAV](https://www.clamav.net/) virus scanning of uploaded files. |
+| clamav.freshclamConfig | string | `"###############\n# General\n###############\n\nDatabaseDirectory /data\nUpdateLogFile /dev/stdout\nLogTime yes\n# CUSTOM: Use pid file in tmp\nPidFile /tmp/freshclam.pid\n# CUSTOM: Set defined user\nDatabaseOwner 2000\n\n###############\n# Updates\n###############\n\nDatabaseMirror database.clamav.net\nScriptedUpdates yes\nNotifyClamd /etc/clamav/clamd.conf\nBytecode yes\n"` |  |
+| clamav.fullnameOverride | string | `"clamav"` |  |
+| clamav.resources.limits.cpu | string | `"4000m"` | [k8s: Resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
+| clamav.resources.limits.memory | string | `"8Gi"` | [k8s: Resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
+| clamav.resources.requests.cpu | string | `"1500m"` | [k8s: Resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
+| clamav.resources.requests.memory | string | `"2Gi"` | [k8s: Resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
 | datapusher.enabled | bool | `true` | Enable/disable Datapusher |
 | fullnameOverride | string | `""` | Override fullname |
 | global.datapusher.db.auth.password | string | `"changeMe"` | Jobs database password. If set, this values will overwrite the value in the Datapusher chart. |
@@ -75,3 +84,5 @@ Kubernetes: `>= 1.23.0-0`
 | redis.enabled | bool | `true` | Enable/disable Redis instance. Disable, if an external Redis instance is used. |
 | solr.enabled | bool | `true` | Enable/disable Apache Solr instance. Disable, if an external Solr instance is used. |
 
+----------------------------------------------
+Autogenerated from chart metadata using [helm-docs v1.13.1](https://github.com/norwoodj/helm-docs/releases/v1.13.1)

charts/sddi-ckan/charts/certIssuer/README.md

@@ -12,3 +12,5 @@ Namespace Issuers for CertManager.
 | enabled | bool | `true` | Enable/disable namespace [Issuers](https://cert-manager.io/docs/concepts/issuer/) for CertManager. |
 | issuerEmail | string | `"example@email.com"` | eMail address for registration with Let's Encrypt account. Note: This is overwritten by `global.ingress.certManager.issuerEmail`, if set. |
 
+----------------------------------------------
+Autogenerated from chart metadata using [helm-docs v1.13.1](https://github.com/norwoodj/helm-docs/releases/v1.13.1)

charts/sddi-ckan/charts/ckan/Chart.yaml

@@ -9,8 +9,8 @@ sources:
   - https://github.com/tum-gis/ckan-docker
   - https://github.com/keitaroinc/docker-ckan
 
-version: 3.0.1
-appVersion: "2.0.0"
+version: "3.1.0-beta1"
+appVersion: "2.1.2"
 
 maintainers:
   - email: b.willenborg@tum.de

charts/sddi-ckan/charts/ckan/README.md

@@ -1,6 +1,6 @@
 # ckan
 
-![Version: 3.0.1](https://img.shields.io/badge/Version-3.0.1-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 2.0.0](https://img.shields.io/badge/AppVersion-2.0.0-informational?style=flat-square)
+![Version: 3.1.0-beta1](https://img.shields.io/badge/Version-3.1.0--beta1-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 2.1.2](https://img.shields.io/badge/AppVersion-2.1.2-informational?style=flat-square)
 
 A Helm chart for SDDI enabled CKAN.
 
@@ -49,6 +49,11 @@ A Helm chart for SDDI enabled CKAN.
 | autoscaling.targetCPUUtilizationPercentage | string | `nil` | [HorizontalPodAutoscaling](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/). |
 | autoscaling.targetMemoryUtilizationPercentage | string | `nil` | [HorizontalPodAutoscaling](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/). |
 | backgroundImage | string | `"../base/images/hero.jpg"` | Set URL or path to [CKAN SDDI background image](https://github.com/tum-gis/ckanext-grouphierarchy-sddi#personalisation). |
+| clamav.enabled | bool | `true` | [CKAN config enable ClamAV] |
+| clamav.host | string | `"clamav"` | [CKAN config ClamAV host] |
+| clamav.port | int | `3310` | [CKAN config ClamAV port] |
+| clamav.timeout | int | `360` | [CKAN config ClamAV connection timeout] |
+| clamav.uploadUnscanned | string | `"False"` | [CKAN config ClamAV upload unscanned files] |
 | component | string | `"ckan"` | Role of CKAN in this chart |
 | datapusher.apiToken | string | `nil` | Datapusher API token, see [CKAN Datapusher settings](https://docs.ckan.org/en/latest/maintaining/configuration.html#datapusher-settings) |
 | datapusher.callback_url_base | string | `"http://ckan:5000/"` | This should be set to cluster internal ckan service domain. [CKAN DataPusher settings](https://docs.ckan.org/en/latest/maintaining/configuration.html#ckan-datapusher-callback-url-base) |
@@ -87,10 +92,11 @@ A Helm chart for SDDI enabled CKAN.
 | ingress.certManager.issuerName | string | `"letsencrypt-staging"` | Name of the Issuer to use. For certManager.type = namespace `letsencrypt-staging`, `letsencrypt-prod` and `self-signed` are available. |
 | ingress.certManager.issuerType | string | `"namespace"` | Type of [cert-manager](https://cert-manager.io/docs/) Issuer: Use either "namespace" or "cluster". |
 | ingress.className | string | `"nginx"` | Name of the [IngressClass](https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-class) to use in Ingress routes. |
+| ingress.configurationSnippet | string | `"more_set_headers \"X-Frame-Options: DENY\";\nmore_set_headers \"X-Xss-Protection: 0\";\nmore_set_headers \"X-Content-Type-Options: nosniff\";\nmore_set_headers \"Content-Security-Policy: object-src 'none'; child-src 'self'; frame-ancestors 'none'; base-uri 'none'; upgrade-insecurerequests; blockall-mixed-content; require-trustedtypes-for 'script'\";\n"` |  |
 | ingress.cors.enabled | bool | `true` | Enable/disable [CORS](https://de.wikipedia.org/wiki/Cross-Origin_Resource_Sharing). See [ingress-nginx cors settings](https://github.com/kubernetes/ingress-nginx/blob/main/docs/user-guide/nginx-configuration/annotations.md#enable-cors) for details on CORS configuration and default settings. Use `ingress.annotations` to overwrite the default configuration annotations. |
 | ingress.domains | list | `[]` | List of [FQDNs](https://de.wikipedia.org/wiki/Fully-Qualified_Host_Name) for this Ingress. Note: All FQDNs will be used for Ingress hosts and TLS certificate. Note: Set `siteUrl` accordingly! |
 | ingress.enabled | bool | `true` | Enable/disable Ingress. |
-| ingress.stickySessions.enabled | bool | `true` | Enable/disable sticks sessions, see [Nginx Ingress Controller Sticky sessions](https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie/). |
+| ingress.stickySessions.enabled | bool | `false` | Enable/disable sticks sessions, see [Nginx Ingress Controller Sticky sessions](https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie/). |
 | ingress.stickySessions.sessionCookie.affinityMode | string | `"balanced"` | [Nginx Ingress Controller Sticky sessions](https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie/) |
 | ingress.stickySessions.sessionCookie.changeOnFailure | string | `"true"` | [Nginx Ingress Controller Sticky sessions](https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie/) |
 | ingress.stickySessions.sessionCookie.maxAge | string | `"172800"` | [Nginx Ingress Controller Sticky sessions](https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie/) |
@@ -131,7 +137,10 @@ A Helm chart for SDDI enabled CKAN.
 | readiness.timeoutSeconds | int | `10` | Timeout interval for the liveness probe |
 | redis.url | string | `"redis://redis-hl:6379/0"` | Redis endpoint for CKAN. This should be set to cluster internal Redis service domain. [CKAN configuration Redis](https://docs.ckan.org/en/latest/maintaining/configuration.html#redis-settings) |
 | replicaCount | int | `1` | Number of replicas. Only used if `autoscaling.enabled = false`. **Note:** Running multiple replicas requires to enable persistent data storage (`persistence.enabled = true`) and, if Pods run on different nodes, a storage that supports RWX. |
-| resources | object | `{}` | [k8s: Resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
+| resources.limits.cpu | string | `"500m"` | [k8s: Resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
+| resources.limits.memory | string | `"1Gi"` | [k8s: Resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
+| resources.requests.cpu | string | `"250m"` | [k8s: Resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
+| resources.requests.memory | string | `"256Mi"` | [k8s: Resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
 | sddiInitDataJson | string | `"init_data.json"` | Local path or URL to File path or URL to [CKAN SDDI `init_data.json`](https://github.com/tum-gis/ckanext-grouphierarchy-sddi/blob/main/ckanext/grouphierarchy/init_data.json). This file allows to specify pre-defined set of SDDI CKAN main categories, topics, and organizations. |
 | securityContext | object | `{}` | [k8s: Security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) |
 | service.port | int | `5000` | Service port for http |
@@ -179,3 +188,5 @@ A Helm chart for SDDI enabled CKAN.
 | volumes | list | See [`values.yml`](values.yml) for the list of default volumes. | Sets [`volumes`](https://kubernetes.io/docs/concepts/storage/volumes). Set to `[]` to disable the default volumes. Set to any list of volume definitions to overwrite the default volumes. Use `extraVolumes` to extend the default volumes. |
 | webassets.path | string | `nil` | Webassets storage path, see [CKAN webassets settings](https://docs.ckan.org/en/latest/maintaining/configuration.html#webassets-settings) This should point to the location of webassets in the CKAN image. The path may vary depending on the CKAN Docker image used. |
 
+----------------------------------------------
+Autogenerated from chart metadata using [helm-docs v1.13.1](https://github.com/norwoodj/helm-docs/releases/v1.13.1)

charts/sddi-ckan/charts/ckan/templates/ckan-configMap-env.yml

@@ -75,6 +75,14 @@ data:
   {{- if .Values.webassets.path }}
   CKAN__WEBASSETS__PATH: {{ .Values.webassets.path | quote }}
   {{- end }}
+  {{- if .Values.clamav.enabled }}
+  # CKANEXT__CLAMAV__SOCKET_PATH: /your/path/to/socket.file
+  CKANEXT__CLAMAV__UPLOAD_UNSCANNED: {{ .Values.clamav.uploadUnscanned | quote }}
+  CKANEXT__CLAMAV__SOCKET_TYPE: tcp
+  CKANEXT__CLAMAV__TCP__HOST: {{ .Values.clamav.host | quote }}
+  CKANEXT__CLAMAV__TCP__PORT: {{ .Values.clamav.port | quote }}
+  CKANEXT__CLAMAV__TIMEOUT: {{ .Values.clamav.timeout | quote }}
+  {{- end }}
   # Additional env vars from values.yaml
   {{- with .Values.extraEnv }}
     {{- toYaml . | nindent 2 }}

charts/sddi-ckan/charts/ckan/templates/ckan-ingress.yml

@@ -30,6 +30,13 @@ metadata:
     {{- end }}
     nginx.ingress.kubernetes.io/proxy-body-size: "{{ .Values.maxUploadSizeMB }}m"
     nginx.org/client-max-body-size: "{{ .Values.maxUploadSizeMB }}m"
+    nginx.ingress.kubernetes.io/proxy-connect-timeout: "360"
+    nginx.ingress.kubernetes.io/proxy-send-timeout: "360"
+    nginx.ingress.kubernetes.io/proxy-read-timeout: "360"
+    nginx.org/proxy-connect-timeout: "360"
+    nginx.org/proxy-read-timeout: "360"
+    nginx.org/proxy-send-timeout: "360"
+    nginx.ingress.kubernetes.io/configuration-snippet: {{- .Values.ingress.configurationSnippet | toYaml | indent 4 }}
     {{- if .Values.ingress.stickySessions.enabled }}
     # https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie/
     nginx.ingress.kubernetes.io/affinity: "cookie"

charts/sddi-ckan/charts/ckan/values.yaml

@@ -134,7 +134,7 @@ ingress:
   stickySessions:
     # -- Enable/disable sticks sessions, see
     # [Nginx Ingress Controller Sticky sessions](https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie/).
-    enabled: true
+    enabled: false
     sessionCookie:
       # -- [Nginx Ingress Controller Sticky sessions](https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie/)
       affinityMode: "balanced"
@@ -156,6 +156,11 @@ ingress:
   tls:
     # -- Specify a custom tls secret name. This overwrites `global.ingress.tls.secretName`.
     secretName:
+  configurationSnippet: |
+    more_set_headers "X-Frame-Options: DENY";
+    more_set_headers "X-Xss-Protection: 0";
+    more_set_headers "X-Content-Type-Options: nosniff";
+    more_set_headers "Content-Security-Policy: object-src 'none'; child-src 'self'; frame-ancestors 'none'; base-uri 'none'; upgrade-insecurerequests; blockall-mixed-content; require-trustedtypes-for 'script'";
 
 # General settings
 # -- CKAN site url. This should match a domain name of CKAN specified in `ingress.domains`/`global.ingress.domains`
@@ -493,8 +498,30 @@ activityStreams:
   # -- [CKAN config activity stream](https://docs.ckan.org/en/latest/maintaining/configuration.html#activity-streams-settings)
   emailNotifications: True
 
-# -- [k8s: Resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/)
-resources: {}
+clamav:
+  # -- [CKAN config enable ClamAV]
+  enabled: true
+  # -- [CKAN config ClamAV upload unscanned files]
+  uploadUnscanned: "False"
+  # -- [CKAN config ClamAV host]
+  host: clamav
+  # -- [CKAN config ClamAV port]
+  port: 3310
+  # -- [CKAN config ClamAV connection timeout]
+  timeout: 360
+
+resources:
+  limits:
+    # -- [k8s: Resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/)
+    cpu: 500m
+    # -- [k8s: Resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/)
+    memory: 1Gi
+  requests:
+    # -- [k8s: Resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/)
+    cpu: 250m
+    # -- [k8s: Resource management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/)
+    memory: 256Mi
+
 # -- [k8s: Assign pods to nodes](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/)
 nodeSelector: {}
 # -- [k8S: Taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/)

charts/sddi-ckan/charts/datapusher/README.md

@@ -65,3 +65,5 @@ A Helm chart for CKAN Datapusher.
 | serviceAccount.name | string | `""` | The name of the service account to use. If not set and create is true, a name is generated using the fullname template |
 | tolerations | list | `[]` | [k8S: Taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) |
 
+----------------------------------------------
+Autogenerated from chart metadata using [helm-docs v1.13.1](https://github.com/norwoodj/helm-docs/releases/v1.13.1)

charts/sddi-ckan/charts/postgis/README.md

@@ -66,3 +66,5 @@ A Helm chart for sa simple PostGIS database pre-configured for CKAN.
 | serviceAccount.name | string | `""` | The name of the service account to use. If not set and create is true, a name is generated using the fullname template |
 | tolerations | list | `[]` | [k8S: Taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) |
 
+----------------------------------------------
+Autogenerated from chart metadata using [helm-docs v1.13.1](https://github.com/norwoodj/helm-docs/releases/v1.13.1)

charts/sddi-ckan/charts/redis/README.md

@@ -55,3 +55,5 @@ A Helm chart for basic Redis for use with CKAN.
 | serviceAccount.name | string | `""` | The name of the service account to use. If not set and create is true, a name is generated using the fullname template |
 | tolerations | list | `[]` | [k8S: Taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) |
 
+----------------------------------------------
+Autogenerated from chart metadata using [helm-docs v1.13.1](https://github.com/norwoodj/helm-docs/releases/v1.13.1)