I do a thing or two with containers from time to time, that involves building Mesosphere or Kubernetes setups. Often enough my OS of choice happens to be CoreOS, but as ashamed as I am I should mention that I often end up scratching my head and wondering what the hell that command was again :pensive:?!? Hope this helps me recall.
Provided, that you are using CoreOS as your VM OS, you may configure your boxes
through the so-called
cloud-config files. In several setups you will provide
etcd details in the cloud config file. This section will just give you a
general idea of where to look for your resources when introspecting CoreOS
One may query the unit-files known to a CoreOS box by running the followng command
I, for one, often end up looking at the the service files of my
which happen to be somewhere in
#cloud-config coreos: etcd: name: etcd addr: $private_ipv4:4001 bind-addr: 0.0.0.0 peer-addr: $private_ipv4:7001 cluster-active-size: 1 etcd-http-read-timeout: 86400 snapshot: true units: - name: etcd.service command: start
May result to the following conf in the
[Service] Environment="ETCD_ADDR=10.220.4.81:4001" Environment="ETCD_BIND_ADDR=0.0.0.0" Environment="ETCD_CLUSTER_ACTIVE_SIZE=1" Environment="ETCD_NAME=etcd" Environment="ETCD_PEER_ADDR=10.220.4.81:7001" Environment="ETCD_SNAPSHOT=true"
Some observation leads one to assume that there happens to be a one-to-one
correspondence between the
cloud-config content and the eventual
configuration with the key names capitalized and prefixed with
$private\_ipv4 token substituted for the machines private IP address.
According to the CoreOS documentation one should find unit files
Fleet handles orchestration on CoreOS clusters. Somewhere I once read to “think of it as systemd for the cloud” and darn… that is spot on.
As you start daemons on your box with
may start services on the cluster with
The beautiful thing, I came to learn about fleet is that it stores the settings
etcd. I discovered it after, destroying some VM’s, creating them anew and
discovering that somehow my services were magically spawned on the fresh boxes
without my intervention. Introspecting the
etcd store left me clueless but
some basic sleuthing uncovered that fleet actually keeps
track of this information in hidden etcd keys. There is actually a world of
etcd secrets to behold if you run
etcdctl ls --recursive _coreos.com).
The entire concept of clustering on CoreOS seems to require clustered machines to share a discovery repository. This is the way clustered machines seem to share information in the CoreOS world and etcd seems to be one of the popular solutions for making this happen.
Somewhere along the line a unit/service breaks and needs to be debugged.
CoreOS logs everything into
one to view the content of these binary logs.
sudo journalctl --file=/var/log/journal/REF/FILE.log
I ran into the
"Failed units: 1" notification which is presented at login.
After reviewing which services were up with
sudo systemctl list-units, I
discovered that the
gce-coreos-cloudinit.service had failed. Running
journalctl and filtering for the logs regarding a specific service using
-u SERVICENAME parameters makes for much easier reading:
sudo journalctl -u gce-coreos-cloudinit.service --file=JOURNAL_FILE.log