-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Updated documentation, cleaned up old files, reworked secrets. #2
Updated documentation, cleaned up old files, reworked secrets. #2
Conversation
Bunch of changes: - Made documentation as painless as possible - Set default network to 10.244.0.0/16, this is what kube-up does by default, along with other deployers in cluster/ - Removed templates for kubernetes output - Generating configs to files, and secrets from the files. - Removed hostpaths. This makes things brittle but portable, warning is in documentation. This is nowhere near ready yet, but lets people poke at it easier. Needs more automation around the IP range variables.
kubectl create secret generic ceph-client-key --from-file=ceph-client-key --namespace=ceph | ||
|
||
cd .. | ||
``` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can I be greedy and ask for this to be in one generate_ceph_secrets.sh
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I'll leave this for the tutorial but add in a script that just does everything.
Great job @elsonrodriguez! I appreciated the resolv.conf changes being documented too! 👍 |
|
||
kubectl create namespace ceph | ||
|
||
kubectl create secret generic ceph-conf-combined --from-file=ceph.conf --from-file=ceph.client.admin.keyring --from-file=ceph.mon.keyring --namespace=ceph |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor change but should ceph.conf
be placed in a ConfigMap instead of a secret?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The /etc/ceph directory is mounted from this secret and contains keys. ceph.conf is the only insecure thing in there.
Would need a refactor, not worth it just yet.
Also I opted to delete the DaemonSet for the MON instead of the RC, but you Ceph experts should chime in on whether or not we need MON for every OSD. |
If your pod has issues mounting, make sure mount.ceph is installed on all nodes. | ||
|
||
``` | ||
apt-get install ceph-fs-common |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To avoid having to install packages (which won't work for things like CoreOS), I wonder if it would make sense (and actually work) to modify the ceph-tools example to copy mount.ceph
to the node - https://github.com/ceph/ceph-docker/tree/master/examples/coreos/tools
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like CoreOS can mount cephfs volumes without additional configuration.
Tested on CoreOS 1032.1.0
Also need to get RBD working. EDIT: Nope, there's an upstream bug regarding Secrets and RBD: kubernetes/kubernetes#25490 Only other way is to put a keyring.conf on each node, much nope. |
c16d59e
to
9eb6a34
Compare
This lets `kubectl rollout` be used for managing everything.
I went ahead and tested the pending RBD PR and it works, so I added an RBD example. @hunter @cornelius-keller What do you think? |
Also doing some work upstream to make the salt-based kube-up providers more ceph-friendly: |
Hi @elsonrodriguez sorry I was on vacation with no mobile broadband access. Right now I am busy catching up, but I will look deeper into it as soon as possible. |
@cornelius-keller No worries. Let me know what else you think we need in order to merge this all upstream. |
Bunch of changes:
This is nowhere near ready yet, but lets people poke at it easier.
Needs more automation around the IP range variables.