- Controllers: Loop that reconciles reality with the desired state
- Controller Manager: Binary that runs a bunch of controllers
- Cloud Controller Manager: Part of the Kubernetes Control Plane, responsible for exposing Cloud Provider functionality inside the Kubernetes cluster.
- Controllers in CCMs:
- Node Metadata
- Match Kubernetes `Node` to Cloud API Server (/Instance/Droplet/...) and set `spec.providerID`
- Label Kubernetes `Nodes` with metadata of the Node, including the [Instance Type](https://kubernetes.io/docs/reference/labels-annotations-taints/#nodekubernetesioinstance-type), Topology ([Region](https://kubernetes.io/docs/reference/labels-annotations-taints/#topologykubernetesioregion), [Zone](https://kubernetes.io/docs/reference/labels-annotations-taints/#topologykubernetesiozone))
- Figure out the Network addresses of the `Node` and set them in `status.addresses`.
- Node Lifecycle
- The `kubelet` is usually responsible for creating the `Node` object in Kubernetes on its first startup.
- This does not work well for the removal of the node, as the `kubelet` might not get a chance to do this. Instead, the Node Lifecycle controller regularly checks if the Node was deleted in the Cloud API, and if it was it also deletes the `Node` object in Kubernetes.
- Service
- Watches `Service` objects with `type: LoadBalancer`. Creates Cloud Load Balancers and configures them according to the `Service` and `Node` objects
- Route
- In Kubernetes, every Pod gets an IP address. This IP address needs to be available from every other pods network (by default). This is usually implemented as an Overlay Network through your CNI like Calico or Cilium.
- If you are already using some kind of Private Networking functionality of your Cloud Provider, then you can use this instead to get rid of the additional Overlay Network and let the Cloud Provider handle the connectivity.
- This is implemented by configuring Routes in the Private Network Routing Table that send all traffic to the `Nodes``spec.podCIDR` to the `Nodes` private IP.
> Goal: Get a compilable binary running k/cloud-provider with no controllers yet
### Dependencies
As all ~~good~~ software does, we [stand on the shoulders of giants](https://xkcd.com/2347/). Let us review the two main dependencies that our code will use:
Thankfully, `k8s.io/cloud-provider` provides an example entrypoint to get us started with out CCM. Lets review the code to make sure we understand what is happening:
```go
package main
// [Imports]
// The main function is called as the entrypoint of our Go binary.
func main() {
// k8s.io/cloud-provider has an elaborate way to read the configuration from flags.
// I found this very tedious to debug, but at least we do not have to do this ourselves.
klog.Fatalf("unable to initialize command options: %v", err)
}
// Can be used to add custom flags to the binary, we dont need this.
fss := cliflag.NamedFlagSets{}
// This initializes the CLI command. The arguments are interesting, so lets take a closer look:
command := app.NewCloudControllerManagerCommand(
// The options we initialized earlier.
ccmOptions,
// This is a custom function that needs to return a [cloudprovider.Interface],
// we will get to this soon.
cloudInitializer,
// This defines which controllers are started, if wanted,
// one can add additional controller loops heroe
app.DefaultInitFuncConstructors,
// Kubernetes v1.28 renamed the controllers to more sensible names, this
// map makes it so that old command-line arguments (--controllers) still work
names.CCMControllerAliases(),
// Our optional additional flags
fss,
// A [<-chanstruct{}]thatcanbeusedtoshutdowntheCCMondemand,
// we do not need this.
wait.NeverStop,
)
// Actually run the command to completion.
code := cli.Run(command)
os.Exit(code)
}
```
Now, this does not compile right now. We use the undefined `cloudInitalizer` method. The method signature we need to implement is `(*config.CompletedConfig) cloudprovider.Interface`. The sample code includes this method, but I found it overly complex for our small CCM, so we will implement it ourselves. Lets take a closer look at the interface we need to return in the next section.
This is the entrypoint into the functionality we can (and will!) implement for our CCM. The interface includes an initializer, two cloud-provider metadata methods and a bunch of Getter functions to other interfaces that implement the actual functionality.
Before we can write the `cloudInitializer` method from above, lets prepare a struct that implements the interface:
```go
package ccm
// ccm/cloud.go
import (
cloudprovider "k8s.io/cloud-provider"
)
type CloudProvider struct {}
// Let's try to assign our struct to a var of the interface we try to implement.
// This way we get nice feedback from our IDE if we break the contract.
var _ cloudprovider.Interface = CloudProvider{}
// Can be used to setup our own controllers or goroutines that need to talk to
//Kubernetes. Not needed for our implementation, so we will leave it empty.
Now that we have a struct implementing the interface, lets create our `cloudInitializer` method. We will actually do this in the same file as our struct:
```go
// ccm/cloud.go
// Just create a new struct for now, we will add some more stuff to this later.
- If this topic does not interest you, you can safely skip ahead! While we use what we built in the following parts, you can still understand what is going on without reading this article.
