Howto get kubernetes anywhere with vsphere and kubernetes 1.11.1 working behind proxy (no direct internet access)

Welcome to the kubernetes-anywhere wiki!

If you need a proxy, the deployment is a little bit harder and need pre planning the cluster infrastructure.

Deployment Network: 10.10.10.0/24

vCenter IP: 10.10.100.6
NodeIPs: 10.10.10.102, 10.10.10.103, 10.10.10.104, 10.10.10.105, 10.10.10.106
Internet Proxy: 10.10.80.98

All IPs you want to use, need to configure in different locations!

First at all, you have to manipulate the PhotonOS Template: Start the VM, login with root/kubernetes Edit /etc/sysconfig/proxy
USE_PROXY=yes
HTTP_PROXY="http://10.XXX.XXX.XXX:8080/"
HTTPS_PROXY="http://10.XXX.XXX.XXX:8080/"
NO_PROXY="127.0.0.1, localhost, 10.10.100.6,10.10.10.102, 10.10.10.103, 10.10.10.104, 10.10.10.105, 10.10.10.106"

Dont forget to delete the machine-id! echo -n > /etc/machine-id Next Step: run the docker "deployment image" with env variables for proxy:

docker pull cnastorage/kubernetes-anywhere:latest
docker run -it -v /tmp:/tmp --rm --env="PS1=[container]:\w> " \
--env="NO_PROXY=localhost,127.0.0.1,10.10.100.6,10.10.10.102,10.10.10.103,10.10.10.104,10.10.10.105,10.10.10.106" \
--env "HTTPS_PROXY=http://10.10.80.98:8080/" \
--env "HTTP_PROXY=http://10.10.80.98:8080/" \
--net=host cnastorage/kubernetes-anywhere:latest /bin/bash

Now in container shell, make config with your settings:

[container]:/opt/kubernetes-anywhere> make config

Use following Options for kubernetes v1.11.1 :
# Phase 2: Node Bootstrapping
#
.phase2.kubernetes_version="v1.11.1"
.phase2.provider="ignition"
.phase2.installer_container="docker.io/cnastorage/k8s-ignition:v1.8-dev-release"
.phase2.docker_registry="gcr.io/google-containers"

now we have to say the nodes, that docker need a proxy:
edit file: kubernetes-anywhere/phase1/vsphere/configure-vm.sh near line 60 (after echo "${nodes_dns_mappings}" >> /etc/hosts)

add these lines :
echo 'HTTP_PROXY=\"http:///10.10.80.98:8080/\"' >> /etc/default/docker <br> echo 'HTTPS_PROXY=\"http:///10.10.80.98:8080/\"' >> /etc/default/docker
echo 'NO_PROXY=\"localhost,127.0.0.1,10.10.100.6,10.10.10.102,10.10.10.103,10.10.10.104,10.10.10.105,10.10.10.106\"' >> /etc/default/docker

save the file...

then:
make deploy

at my site the cluster created successfully.


dont forget to copy phase1/vsphere/<your_defined_clustername>/kubeconfig.json before closing the deployment container

Now you have to use a seperate machine with kubectl
kubectl proxy --address='10.10.10.100' --port=8001 --accept-hosts='.*'
since any one version, the access to WebUI is following:
http://10.10.10.100:8011/api/v1/namespaces/kube-system/services/kubernetes-dashboard/proxy/#/workload?namespace=default