controller-manager flex volume path is hardcoded (fails on coreos)

[f0]

Is this a BUG REPORT or FEATURE REQUEST?

BUG REPORT

Versions

kubeadm version (use kubeadm version):
kubeadm version: &version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.0", GitCommit:"6e937839ac04a38cac63e6a7a306c5d035fe7b0a", GitTreeState:"clean", BuildDate:"2017-09-28T22:46:41Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Environment:

• Cloud provider or hardware configuration:
  VMWare
• OS (e.g. from /etc/os-release):
  NAME="Container Linux by CoreOS"
  ID=coreos
  VERSION=1520.2.0
  VERSION_ID=1520.2.0
  BUILD_ID=2017-09-13-2048
  PRETTY_NAME="Container Linux by CoreOS 1520.2.0 (Ladybug)"
  ANSI_COLOR="38;5;75"
  HOME_URL="https://coreos.com/"
  BUG_REPORT_URL="https://issues.coreos.com"
  COREOS_BOARD="amd64-usr"
• Kernel (e.g. uname -a):
  4.13.1-coreos

What happened?

In /etc/kubernetes/manifests/kube-controller-manager.yaml the path to the flex volume plugin dir is hardcoded to /usr/libexec/kubernetes/kubelet-plugins/volume/exec

     - hostPath:
      path: /usr/libexec/kubernetes/kubelet-plugins/volume/exec
      type: DirectoryOrCreate
      name: flexvolume-dir

On CoreOS /usr/libexec is ReadOnly, and the controller manager fails to start
I can chenge the controller manager path with a flag, but this does not change the hardcoded hostpath

What you expected to happen?

i can chage the hardcoded mount path with a config option

How to reproduce it (as minimally and precisely as possible)?

Install coreos + kubeadm init

[jamiehannaford]

To make this configurable, we'd probably need to:

• Add a new config option (something like Storage.FlexPluginsPath)
• If defined, mount this path in the manifests
• We might also need to update kubelet's --volume-plugin-dir and CM's --flex-volume-plugin-dir flags

[andrewrynhard]

@jamiehannaford please see kubernetes/kubernetes#49840. Once this merged it will allow you to override the default path with something like this:

apiServerExtraVolumes:
  - name: ca-certs
    hostPath: /usr/share/ca-certificates
    mountPath: /etc/ssl/certs
controllerManagerExtraArgs:
  flex-volume-plugin-dir: /opt/libexec/kubernetes/kubelet-plugins/volume/exec
controllerManagerExtraVolumes:
  - name: ca-certs
    hostPath: /usr/share/ca-certificates
    mountPath: /etc/ssl/certs
  - name: flexvolume-dir
    hostPath: /opt/libexec/kubernetes/kubelet-plugins/volume/exec
    mountPath: /usr/libexec/kubernetes/kubelet-plugins/volume/exec

[luxas]

@andrewrynhard will send one more PR for fixing this without user intervention and we'll backport to v1.8