One 、Operator brief introduction
stay Kubernetes We often use Deployment、DaemonSet、Service、ConfigMap And so on , These are all resources Kubernetes Built in resources for , Their creation 、 to update 、 Delete, etc Controller Manager Responsible for managing the .
Two 、Operator form
Operator(Controller+CRD）,Operator By Kubernetes Custom resources （CRD） And controller （Controller） The meta - native extension service , among CRD Define each Operator Custom resource objects that need to be created and managed , The bottom layer is actually through APIServer Interface in ETCD Register a new resource type in , After registration, you can create objects of this resource type . But just registering resources and creating resource objects doesn't make any sense ,CRD The most important thing is to cooperate with the corresponding Controller To achieve the desired state of the user-defined resource , For example, built-in Deployment Controller Used to control Department Functions of resource objects , Generate a specific number of... According to the configuration Pod Monitor its status , And make corresponding actions according to the event .
3、 ... and 、Operator Use
Users want to build a for their own custom resources Kubernetes Operator, There are many tools to choose from, such as Operator SDK、Kubebuilder, You can even use Operator SDK（HELM、Ansible、Go）. These tools create Kubernetes Operator Used to monitor custom resources , And adjust the resource status according to the change of resources .
Operator As a custom extension resource, you can Deployment Deploy to Kubernetes in , adopt List-Watch Method to monitor the changes of corresponding resources , When a user modifies anything in a custom resource ,Operator Will monitor changes to resources , And perform specific operations according to the changes , These operations usually have an impact on Kubernetes API Some resources in the .
Four 、Operator The application case
Kubebuilder Introduce
Kubebuilder It's a use. Go Language construction Kubenetes API Controller and CRD Scaffolding tools , By using Kubebuilder, Users can follow a simple programming framework , To write Operator Using examples .
- kubenetes: v1.23.8
- go
tar xf go1.18.3.linux-amd64.tar.gz
mv go /usr/local/
vim /etc/profile.d/go183.sh
export GO111MODULE=on
export GOROOT=/usr/local/go
export GOPATH=/root/gopath
export PATH=$PATH:$GOROOT/bin:$GOPATH/bin
source /etc/profile.d/go183.sh- kubebuilder: 3.5.0
yum -y install gcc
curl -L -o kubebuilder https://go.kubebuilder.io/dl/latest/$(go env GOOS)/$(go env GOARCH)
chmod +x kubebuilder && mv kubebuilder /usr/local/bin/
kubebuilder versionCase study
Welcome The case mainly realizes the use of Operator and CRD Deploy a complete application environment , You can create resources based on custom types , By creating a Welcome Type of resource , Automatically created in the background Deployment and Service, adopt Web Page access Service Render application deployment , Control management is carried out by means of user-defined controller
We created western medicine Welcome User defined resources and corresponding Controllers, Finally, we can use the following code YAML File deployment is simple Web application
apiVersion: webapp.demo.welcome.domain/v1
kind: Welcome
metadata:
name: welcome-sample
spec:
name: myfriendsWeb Application Introduction
In this case , We use Go Language HTTP Module to create a Web service , After the user visits the page, it will be loaded automatically NAME And PORT Environment variables and render index.html In static file , The code is as follows ：
package main
import (
"fmt"
"net/http"
"os"
)
func main() {
name := os.Getenv("NAME")
hello := fmt.Sprintf("Hello %s", name)
http.Handle("/hello/", http.StripPrefix("/hello/", http.FileServer(http.Dir("static"))))
f, err := os.OpenFile("./static/index.html", os.O_APPEND|os.O_WRONLY|os.O_CREATE, 0600)
if err != nil {
panic(err)
}
defer f.Close()
if _, err = f.WriteString(hello); err != nil {
panic(err)
}
port := os.Getenv("PORT")
if port == "" {
port = "8080"
}
}among ,NAME Environment variables are passed through us in Welcome As defined in name Field get , In the following controller writing, we will introduce in detail the detailed methods of obtaining fields . We will index.html Put it in Static Under the folder , And package the project file as Docker Mirror image ,Dockerfile as follows ：
FROM golang:1.17 as builder
WORKDIR /
COPY . .
COPY static
RUN CGO_ENABLED=0 GOOS=linux go build -v -o main
FROM alpine
RUN apk add --no-cache ca-certificates
COPY --from=builder /main /usr/local/main
COPY --from=builder static /static
CMD ["/usr/local/main"]Project initialization
Use Kubebuilder Command image project initialization
mkdir demo
cd demo
go mod init welcome_demo.domain
kubebuilder init --domain demo.welcome.domainAfter initializing the project ,kubebuilder The generated project interface is as follows
[[email protected] demo]# tree .
.
├── config
│ ├── default
│ │ ├── kustomization.yaml
│ │ ├── manager_auth_proxy_patch.yaml
│ │ └── manager_config_patch.yaml
│ ├── manager
│ │ ├── controller_manager_config.yaml
│ │ ├── kustomization.yaml
│ │ └── manager.yaml
│ ├── prometheus
│ │ ├── kustomization.yaml
│ │ └── monitor.yaml
│ └── rbac
│ ├── auth_proxy_client_clusterrole.yaml
│ ├── auth_proxy_role_binding.yaml
│ ├── auth_proxy_role.yaml
│ ├── auth_proxy_service.yaml
│ ├── kustomization.yaml
│ ├── leader_election_role_binding.yaml
│ ├── leader_election_role.yaml
│ ├── role_binding.yaml
│ └── service_account.yaml
├── Dockerfile
├── go.mod
├── go.sum
├── hack
│ └── boilerplate.go.txt
├── main.go
├── Makefile
├── PROJECT
└── README.md
6 directories, 25 files
establish `Welcome` Kind And its corresponding controller
[[email protected] demo]# kubebuilder create api --group webapp --kind Welcome --version v1
Create Resource [y/n]
y
Create Controller [y/n]
yType twice y, Kubebuilder Create templates for resources and controllers respectively , Here group、version、kind this 3 Attributes are combined to identify a k8s Of CRD, After creation ,Kubebuilder Add the following file ：
[[email protected] demo]# tree .
.
├── api
│ └── v1
│ ├── groupversion_info.go
│ ├── welcome_types.go
│ └── zz_generated.deepcopy.go
├── bin
│ └── controller-gen
├── config
│ ├── crd
│ │ ├── kustomization.yaml
│ │ ├── kustomizeconfig.yaml
│ │ └── patches
│ │ ├── cainjection_in_welcomes.yaml
│ │ └── webhook_in_welcomes.yaml
│ ├── default
│ │ ├── kustomization.yaml
│ │ ├── manager_auth_proxy_patch.yaml
│ │ └── manager_config_patch.yaml
│ ├── manager
│ │ ├── controller_manager_config.yaml
│ │ ├── kustomization.yaml
│ │ └── manager.yaml
│ ├── prometheus
│ │ ├── kustomization.yaml
│ │ └── monitor.yaml
│ ├── rbac
│ │ ├── auth_proxy_client_clusterrole.yaml
│ │ ├── auth_proxy_role_binding.yaml
│ │ ├── auth_proxy_role.yaml
│ │ ├── auth_proxy_service.yaml
│ │ ├── kustomization.yaml
│ │ ├── leader_election_role_binding.yaml
│ │ ├── leader_election_role.yaml
│ │ ├── role_binding.yaml
│ │ ├── service_account.yaml
│ │ ├── welcome_editor_role.yaml
│ │ └── welcome_viewer_role.yaml
│ └── samples
│ └── webapp_v1_welcome.yaml
├── controllers
│ ├── suite_test.go
│ └── welcome_controller.go
├── Dockerfile
├── go.mod
├── go.sum
├── hack
│ └── boilerplate.go.txt
├── main.go
├── Makefile
├── PROJECT
└── README.md
13 directories, 38 filesThe following two steps are required ：
- modify Resource Type
- modify Controller Logic
modify Resource Type
here Resource Type For the resource fields that need to be defined , Used in Yaml Make a statement in the document , In this case, you need to add name Fields are used for “Welcome”Kind Medium Web application , The code is as follows ：
vim api/v1/welcome_types.go
// WelcomeSpec defines the desired state of Welcome
type WelcomeSpec struct {
// INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
// Important: Run "make" to regenerate code after modifying this file
// Foo is an example field of Welcome. Edit welcome_types.go to remove/update
// Foo string `json:"foo,omitempty"`
Name string `json:"name,omitempty"`
}modify Controller Logic
stay Controller We need to go through Reconcile Method to complete Deployment and Service Deploy , And finally achieve the desired state .
vim controllers/welcome_controller.go
//+kubebuilder:rbac:groups=webapp.demo.welcome.domain,resources=welcomes,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=webapp.demo.welcome.domain,resources=welcomes/status,verbs=get;update;patch
//+kubebuilder:rbac:groups=webapp.demo.welcome.domain,resources=welcomes/finalizers,verbs=update
// Reconcile is part of the main kubernetes reconciliation loop which aims to
// move the current state of the cluster closer to the desired state.
// TODO(user): Modify the Reconcile function to compare the state specified by
// the Welcome object against the actual cluster state, and then
// perform operations to make the cluster state reflect the state specified by
// the user.
//
// For more details, check Reconcile and its Result here:
// - https://pkg.go.dev/sigs.k8s.io/[email protected]/pkg/reconcile
func (r *WelcomeReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
// _ = log.FromContext(ctx)
// TODO(user): your logic here
log := r.Log.WithValues("welcome", req.NamespacedName)
log.Info("reconcilling welcome")
return ctrl.Result{}, nil
}- The final code is as follows
/*
Copyright 2022.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package controllers
import (
"context"
"fmt"
appsv1 "k8s.io/api/apps/v1"
"k8s.io/apimachinery/pkg/api/resource"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/util/intstr"
"sigs.k8s.io/controller-runtime/pkg/log"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/runtime"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
webappv1 "welcome_demo.domain/api/v1"
)
// WelcomeReconciler reconciles a Welcome object
type WelcomeReconciler struct {
client.Client
Scheme *runtime.Scheme
}
//+kubebuilder:rbac:groups=webapp.demo.welcome.domain,resources=welcomes,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=webapp.demo.welcome.domain,resources=welcomes/status,verbs=get;update;patch
//+kubebuilder:rbac:groups=webapp.demo.welcome.domain,resources=welcomes/finalizers,verbs=update
// Reconcile is part of the main kubernetes reconciliation loop which aims to
// move the current state of the cluster closer to the desired state.
// TODO(user): Modify the Reconcile function to compare the state specified by
// the Welcome object against the actual cluster state, and then
// perform operations to make the cluster state reflect the state specified by
// the user.
//
// For more details, check Reconcile and its Result here:
// - https://pkg.go.dev/sigs.k8s.io/[email protected]/pkg/reconcile
func (r *WelcomeReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
_ = log.FromContext(ctx)
// TODO(user): your logic here
welcome := &webappv1.Welcome{}
if err := r.Client.Get(ctx, req.NamespacedName, welcome); err != nil {
return ctrl.Result{}, err
}
deployment, err := r.createWelcomeDeployment(welcome)
if err != nil {
return ctrl.Result{}, err
}
fmt.Println("create deployment success!")
svc, err := r.createService(welcome)
if err != nil {
return ctrl.Result{}, err
}
fmt.Println("create service success!")
applyOpts := []client.PatchOption{client.ForceOwnership, client.FieldOwner("welcome_controller")}
err = r.Patch(ctx, &deployment, client.Apply, applyOpts...)
if err != nil {
return ctrl.Result{}, err
}
err = r.Patch(ctx, &svc, client.Apply, applyOpts...)
if err != nil {
return ctrl.Result{}, err
}
return ctrl.Result{}, nil
}
func (r *WelcomeReconciler) createWelcomeDeployment(welcome *webappv1.Welcome) (appsv1.Deployment, error) {
defOne := int32(1)
name := welcome.Spec.Name
if name == "" {
name = "world"
}
depl := appsv1.Deployment{
TypeMeta: metav1.TypeMeta{
APIVersion: appsv1.SchemeGroupVersion.String(),
Kind: "Deployment",
},
ObjectMeta: metav1.ObjectMeta{
Name: welcome.Name,
Namespace: welcome.Namespace,
},
Spec: appsv1.DeploymentSpec{
Replicas: &defOne,
Selector: &metav1.LabelSelector{
MatchLabels: map[string]string{"welcome": welcome.Name},
},
Template: corev1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: map[string]string{"welcome": welcome.Name},
},
Spec: corev1.PodSpec{
Containers: []corev1.Container{
{
Name: "welcome",
Env: []corev1.EnvVar{
{Name: "NAME", Value: name},
},
Ports: []corev1.ContainerPort{
{ContainerPort: 8080,
Name: "http",
Protocol: "TCP",
},
},
Image: "sdfcdwefe/operatordemo:v1",
Resources: corev1.ResourceRequirements{
Requests: corev1.ResourceList{
corev1.ResourceCPU: *resource.NewMilliQuantity(100, resource.DecimalSI),
corev1.ResourceMemory: *resource.NewMilliQuantity(100000, resource.BinarySI),
},
},
},
},
},
},
},
}
return depl, nil
}
func (r *WelcomeReconciler) createService(welcome *webappv1.Welcome) (corev1.Service, error) {
svc := corev1.Service{
TypeMeta: metav1.TypeMeta{
APIVersion: corev1.SchemeGroupVersion.String(),
Kind: "Service"},
ObjectMeta: metav1.ObjectMeta{
Name: welcome.Name,
Namespace: welcome.Namespace,
},
Spec: corev1.ServiceSpec{
Ports: []corev1.ServicePort{
{Name: "http",
Port: 8080,
Protocol: "TCP",
TargetPort: intstr.FromString("http")},
},
Selector: map[string]string{"welcome": welcome.Name},
Type: corev1.ServiceTypeLoadBalancer,
},
}
return svc, nil
}
// SetupWithManager sets up the controller with the Manager.
func (r *WelcomeReconciler) SetupWithManager(mgr ctrl.Manager) error {
return ctrl.NewControllerManagedBy(mgr).
For(&webappv1.Welcome{}).
Complete(r)
}
Welcome Should be deployed
- Generate CRD resources
[[email protected] demo]# make manifests
/root/demo/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
[[email protected] demo]# tree .
.
├── api
│ └── v1
│ ├── groupversion_info.go
│ ├── welcome_types.go
│ └── zz_generated.deepcopy.go
├── bin
│ └── controller-gen
├── config
│ ├── crd
│ │ ├── bases
│ │ │ └── webapp.demo.welcome.domain_welcomes.yaml
│ │ ├── kustomization.yaml
│ │ ├── kustomizeconfig.yaml
│ │ └── patches
│ │ ├── cainjection_in_welcomes.yaml
│ │ └── webhook_in_welcomes.yaml
│ ├── default
│ │ ├── kustomization.yaml
│ │ ├── manager_auth_proxy_patch.yaml
│ │ └── manager_config_patch.yaml
│ ├── manager
│ │ ├── controller_manager_config.yaml
│ │ ├── kustomization.yaml
│ │ └── manager.yaml
│ ├── prometheus
│ │ ├── kustomization.yaml
│ │ └── monitor.yaml
│ ├── rbac
│ │ ├── auth_proxy_client_clusterrole.yaml
│ │ ├── auth_proxy_role_binding.yaml
│ │ ├── auth_proxy_role.yaml
│ │ ├── auth_proxy_service.yaml
│ │ ├── kustomization.yaml
│ │ ├── leader_election_role_binding.yaml
│ │ ├── leader_election_role.yaml
│ │ ├── role_binding.yaml
│ │ ├── role.yaml
│ │ ├── service_account.yaml
│ │ ├── welcome_editor_role.yaml
│ │ └── welcome_viewer_role.yaml
│ └── samples
│ └── webapp_v1_welcome.yaml
├── controllers
│ ├── suite_test.go
│ └── welcome_controller.go
├── Dockerfile
├── go.mod
├── go.sum
├── hack
│ └── boilerplate.go.txt
├── main.go
├── Makefile
├── PROJECT
└── README.md
14 directories, 40 files
[[email protected] demo]# cat config/crd/bases/webapp.demo.welcome.domain_welcomes.yaml
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.9.0
creationTimestamp: null
name: welcomes.webapp.demo.welcome.domain
spec:
group: webapp.demo.welcome.domain
names:
kind: Welcome
listKind: WelcomeList
plural: welcomes
singular: welcome
scope: Namespaced
versions:
- name: v1
schema:
openAPIV3Schema:
description: Welcome is the Schema for the welcomes API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: WelcomeSpec defines the desired state of Welcome
properties:
name:
description: Foo is an example field of Welcome. Edit welcome_types.go
to remove/update Foo string `json:"foo,omitempty"`
type: string
type: object
status:
description: WelcomeStatus defines the observed state of Welcome
type: object
type: object
served: true
storage: true
subresources:
status: {}- establish Welcome Type resources
[[email protected] demo]# kubectl create -f config/crd/bases/
customresourcedefinition.apiextensions.k8s.io/welcomes.webapp.demo.welcome.domain created
[[email protected] demo]# kubectl create -f config/samples/webapp_v1_welcome.yaml
welcome.webapp.demo.welcome.domain/welcome-sample created- Use `kubectl get crd` Command to view custom objects
[[email protected] demo]# kubectl get crd | grep welcome
welcomes.webapp.demo.welcome.domain 2022-06-25T09:10:37Z- adopt kubectl get welcome The command can see the created welcome object
[[email protected] demo]# kubectl get welcome
NAME AGE
welcome-sample 2m20shere CRD Will not complete any work , It's just ETCD Created a record in , We need to run Controller To help us complete the work and finally achieve welcome Defined states .
[[email protected] demo]# make run
/root/demo/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/root/demo/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
api/v1/welcome_types.go
go vet ./...
go run ./main.go
1.6561485987622015e+09 INFO controller-runtime.metrics Metrics server is starting to listen {"addr": ":8080"}
1.6561485987624757e+09 INFO setup starting manager
1.6561485987638762e+09 INFO Starting server {"path": "/metrics", "kind": "metrics", "addr": "[::]:8080"}
1.656148598763948e+09 INFO Starting server {"kind": "health probe", "addr": "[::]:8081"}
1.656148598764167e+09 INFO Starting EventSource {"controller": "welcome", "controllerGroup": "webapp.demo.welcome.domain", "controllerKind": "Welcome", "source": "kind source: *v1.Welcome"}
1.6561485987641926e+09 INFO Starting Controller {"controller": "welcome", "controllerGroup": "webapp.demo.welcome.domain", "controllerKind": "Welcome"}
1.6561485988653958e+09 INFO Starting workers {"controller": "welcome", "controllerGroup": "webapp.demo.welcome.domain", "controllerKind": "Welcome", "worker count": 1}
create deployment success!
create service success!Start the controller locally in the above way , Convenient for debugging and verification
- View the created deployment、service
[[email protected] demo]# kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
welcome-sample 1/1 1 1 34s
[[email protected] demo]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4d2h
welcome-sample LoadBalancer 10.106.36.129 <pending> 8080:30037/TCP 39s
![leetcode：1567. Length of the longest subarray whose product is a positive number [dp[i] indicates the maximum length ending with I]](/img/a4/c5c31de7a0a3b34a188bfec0b5d184.png)