CRI basic principles
Early kubernetes use docker as the default runtime, and later joined rkt, each time adding a new run, k8s should modify the interface. container runtime to join, and to achieve significant functional differences in different containers. Standard is CRI (Container RunTime Interface). k8s no longer cares about the bottom of the container runtime, kubelet only perceived CRI server, CRI and CRI server simply follow the standard implementation
CRI interfaces are defined in k8s.io/cri-api/pkg/apis/services.go, look at the specific definition of what interfaces
RuntimeService Interface
RuntimeVersioner, responsible for returning the runtime name, version, and the version of the API, an interface function Version
- Version(apiVersion string) (*runtimeapi.VersionResponse, error)
ContainerManager, of the container user interface, simple
- CreateContainer(podSandboxID string, config *runtimeapi.ContainerConfig, sandboxConfig *runtimeapi.PodSandboxConfig) (string, error)
- StartContainer(containerID string) error
- StopContainer(containerID string, timeout int64) error
- RemoveContainer(containerID string) error
- ListContainers(filter *runtimeapi.ContainerFilter) ([]*runtimeapi.Container, error)
- ContainerStatus(containerID string) (*runtimeapi.ContainerStatus, error)
- UpdateContainerResources(containerID string, resources *runtimeapi.LinuxContainerResources) error
- ExecSync(containerID string, cmd []string, timeout time.Duration) (stdout []byte, stderr []byte, err error)
- Exec (runtimeapi.ExecRequest *) (* runtimeapi.ExecResponse, error)
- Attach(req *runtimeapi.AttachRequest) (*runtimeapi.AttachResponse, error)
- ReopenContainerLog(ContainerID string) error
PodSandboxManager, responsible for operating the sandbox (Pause container or virtual machine) is thread-safe, including as follows:
- RunPodSandbox(config *runtimeapi.PodSandboxConfig, runtimeHandler string) (string, error)
- StopPodSandbox(podSandboxID string) error
- RemovePodSandbox(podSandboxID string) error
- PodSandboxStatus(podSandboxID string) (*runtimeapi.PodSandboxStatus, error)
- ListPodSandbox(filter *runtimeapi.PodSandboxFilter) ([]*runtimeapi.PodSandbox, error)
- PortForward(*runtimeapi.PortForwardRequest) (*runtimeapi.PortForwardResponse, error)
Status query interface
// RuntimeService interface should be implemented by a container runtime. // The methods should be thread-safe. type RuntimeService interface { RuntimeVersioner ContainerManager PodSandboxManager ContainerStatsManager // UpdateRuntimeConfig updates runtime configuration if specified UpdateRuntimeConfig(runtimeConfig *runtimeapi.RuntimeConfig) error // Status returns the status of the runtime. Status() (*runtimeapi.RuntimeStatus, error) }
ImageManagerService Interface
It defines the image of interfaces, including query, download, delete, etc.
// ImageManagerService interface should be implemented by a container image
// manager.
// The methods should be thread-safe.
type ImageManagerService interface {
// ListImages lists the existing images.
ListImages(filter *runtimeapi.ImageFilter) ([]*runtimeapi.Image, error)
// ImageStatus returns the status of the image.
ImageStatus(image *runtimeapi.ImageSpec) (*runtimeapi.Image, error)
// PullImage pulls an image with the authentication config.
PullImage(image *runtimeapi.ImageSpec, auth *runtimeapi.AuthConfig, podSandboxConfig *runtimeapi.PodSandboxConfig) (string, error)
// RemoveImage removes the image.
RemoveImage(image *runtimeapi.ImageSpec) error
// ImageFsInfo returns information of the filesystem that is used to store images.
ImageFsInfo() ([]*runtimeapi.FilesystemUsage, error)
}
RunKubelet
--> createAndInitKubelet
--> NewMainKubelet
-> getRuntimeAndImageServices (Chapter 1 explains)
--> NewKubeGenericRuntimeManager
1. getRuntimeAndImageServices function
NewRemoteRuntimeService remote runtime instantiates service, the interface to achieve the above RuntimeService
Examples of NewRemoteImageService remote mirroring service, the interface to achieve the above ImageManagerService
func getRuntimeAndImageServices(remoteRuntimeEndpoint string, remoteImageEndpoint string, runtimeRequestTimeout metav1.Duration) (internalapi.RuntimeService, internalapi.ImageManagerService, error) {
rs, err := remote.NewRemoteRuntimeService(remoteRuntimeEndpoint, runtimeRequestTimeout.Duration)
if err != nil {
return nil, nil, err
}
is, err := remote.NewRemoteImageService(remoteImageEndpoint, runtimeRequestTimeout.Duration)
if err != nil {
return nil, nil, err
}
return rs, is, err
}1.1 NewRemoteRuntimeService function
Implementation is to establish connection with the remote endpoint GRPC instantiated RemoteRuntimeService, implements RuntimeService interface, path
pkg / kubelet / remote / remote_runtime.go, the implementation of the interface GRPC calling client sends a request GRPC
// NewRemoteRuntimeService creates a new internalapi.RuntimeService.
func NewRemoteRuntimeService(endpoint string, connectionTimeout time.Duration) (internalapi.RuntimeService, error) {
klog.V(3).Infof("Connecting to runtime service %s", endpoint)
addr, dailer, err := util.GetAddressAndDialer(endpoint)
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), connectionTimeout)
defer cancel()
conn, err := grpc.DialContext(ctx, addr, grpc.WithInsecure(), grpc.WithDialer(dailer), grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(maxMsgSize)))
if err != nil {
klog.Errorf("Connect remote runtime %s failed: %v", addr, err)
return nil, err
}
return &RemoteRuntimeService{
timeout: connectionTimeout,
runtimeClient: runtimeapi.NewRuntimeServiceClient(conn),
logReduction: logreduction.NewLogReduction(identicalErrorDelay),
}, nil
}
1.1.1 RunPodSandbox create a sandbox
Sending a request to the remote server GRPC RunPodSandboxRequest
// RunPodSandbox creates and starts a pod-level sandbox. Runtimes should ensure
// the sandbox is in ready state.
func (r *RemoteRuntimeService) RunPodSandbox(config *runtimeapi.PodSandboxConfig, runtimeHandler string) (string, error) {
// Use 2 times longer timeout for sandbox operation (4 mins by default)
// TODO: Make the pod sandbox timeout configurable.
ctx, cancel := getContextWithTimeout(r.timeout * 2)
defer cancel()
resp, err := r.runtimeClient.RunPodSandbox(ctx, &runtimeapi.RunPodSandboxRequest{
Config: config,
RuntimeHandler: runtimeHandler,
})
if err != nil {
klog.Errorf("RunPodSandbox from runtime service failed: %v", err)
return "", err
}
if resp.PodSandboxId == "" {
errorMessage := fmt.Sprintf("PodSandboxId is not set for sandbox %q", config.GetMetadata())
klog.Errorf("RunPodSandbox failed: %s", errorMessage)
return "", errors.New(errorMessage)
}
return resp.PodSandboxId, nil
}1.2 NewRemoteImageService function
GRPC implementation is to establish connection with remote, instantiation RemoteImageService, to achieve the above interfaces ImageManagerService
Achieved path pkg / kubelet / remote / remote_image.go, the implementation of the interface GRPC calling client sends a request GRPC
// NewRemoteImageService creates a new internalapi.ImageManagerService.
func NewRemoteImageService(endpoint string, connectionTimeout time.Duration) (internalapi.ImageManagerService, error) {
klog.V(3).Infof("Connecting to image service %s", endpoint)
addr, dailer, err := util.GetAddressAndDialer(endpoint)
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), connectionTimeout)
defer cancel()
conn, err := grpc.DialContext(ctx, addr, grpc.WithInsecure(), grpc.WithDialer(dailer), grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(maxMsgSize)))
if err != nil {
klog.Errorf("Connect remote image service %s failed: %v", addr, err)
return nil, err
}
return &RemoteImageService{
timeout: connectionTimeout,
imageClient: runtimeapi.NewImageServiceClient(conn),
}, nil
}
2. NewKubeGenericRuntimeManager function
Examples of kubeGenericRuntimeManager, implements KubeGenericRuntime interface defined in pkg / kubelet / kuberuntime / kuberuntime_manager.go, the interface comprising, a container run, and command interfaces
// NewKubeGenericRuntimeManager creates a new kubeGenericRuntimeManager
func NewKubeGenericRuntimeManager(
recorder record.EventRecorder,
livenessManager proberesults.Manager,
seccompProfileRoot string,
containerRefManager *kubecontainer.RefManager,
machineInfo *cadvisorapi.MachineInfo,
podStateProvider podStateProvider,
osInterface kubecontainer.OSInterface,
runtimeHelper kubecontainer.RuntimeHelper,
httpClient types.HttpGetter,
imageBackOff *flowcontrol.Backoff,
serializeImagePulls bool,
imagePullQPS float32,
imagePullBurst int,
cpuCFSQuota bool,
cpuCFSQuotaPeriod metav1.Duration,
runtimeService internalapi.RuntimeService,
imageService internalapi.ImageManagerService,
internalLifecycle cm.InternalContainerLifecycle,
legacyLogProvider LegacyLogProvider,
runtimeClassManager *runtimeclass.Manager,
) (KubeGenericRuntime, error)Assignment runtime
klet.containerRuntime = runtime klet.streamingRuntime = runtime klet.runner = runtime