概述
本实验向您展示如何使用 Oracle Database Kubernetes Operator在 Kubernetes 集群中部署和运行 Oracle Database。
有关 Operator 提供的其他部署选项的更多详细信息，请参阅 Oracle Database Operator for Kubernetes 文档（例如运行自治数据库或在 CDB 上运行本地 PDB）
此实验申请地址在这里。
实验帮助在这里。
在本实验中，我们将使用两种类型的持久存储：
- 一个动态块卷，一旦数据库被删除，它将被自动删除，
- 静态 NFS 文件系统，允许在 kubernetes 集群的节点之间进行自动故障转移。
目标
- 在 Oracle Cloud Infrastructure 上设置 Oracle Kubernetes Engine 实例
- 安装数据库 Kubernetes Operator
- 使用动态块卷在 Kubernetes 上配置和启动数据库实例
- 连接到数据库并删除实例
- 创建 NFS 共享文件系统
- 使用文件系统配置和启动具有多个 Pod 的数据库
- 通过停止最初运行数据库的节点来验证节点故障转移
- 销毁环境
预计研讨会持续时间：1 小时
Lab 1: Set up an Oracle Kubernetes Engine instance on OCI
Introduction
本实验将引导您完成在 Oracle 云基础设施上创建托管 Kubernetes 环境的步骤。
Oracle Cloud Infrastructure Container Engine for Kubernetes 是一种完全托管、可扩展且高度可用的服务，可用于将容器应用程序部署到云中。 当您的开发团队想要可靠地构建、部署和管理云原生应用程序时，请使用 Container Engine for Kubernetes（有时缩写为 OKE）。 您指定应用程序所需的计算资源，OKE 在现有 OCI 租约中的 Oracle 云基础设施上配置这些资源。
在本实验中，您将：
- 创建 OKE（Oracle Kubernetes Engine）实例。
- 打开 OCI Cloud Shell 并配置 kubectl 以与 Kubernetes 集群交互。
Task 1: Create an OKE cluster
快速创建（Quick Create）功能使用默认设置，根据需要使用新的网络资源创建快速集群。 这种方法是创建新集群的最快方法。 如果您接受所有默认值，只需单击几下即可创建一个新集群。 自动为集群创建新的网络资源，以及一个节点池和三个工作节点。本实验将使用Quick Create方式。
以下需显式设置，其余均使用默认值：
- Kubernetes API Endpoint: Public Endpoint
- Node Type: Managed
- Kubernetes Worker Nodes: Public Workers
以下为部分截图：
总共耗时12分钟（这个时间比较稳定），其中创建cluster用3分钟，创建node pool用3分钟。最终状态如下：
注意，这里创建了3个子网，一个用于Kubenetes API，一个用于负载均衡，还有一个用于3个K8S node，图上没有显示，是以oke-nodesubnet开头的。
为何专门指出，是因为在后续NFS的实验中，Mount Target锁使用的子网需要与K8S node的子网一致，否则K8S node无法访问到NFS。这个错误导致我浪费了至少6个小时的时间
Task 2: Configure kubectl (Kubernetes Cluster CLI)
我没有Cloud Shell Access的权限，所以用了Local Access的方法。过程详见Setting Up Local Access to Clusters
先创建一个OCI实例，安装OCI CLI，过程略。
然后配置OCI CLI，过程略：
oci setup config
验证OCI CLI配置成功：
oci iam region list
通过OCI CLI创建kubeconfig：
mkdir -p $HOME/.kube
CLUSTER_ID=ocid1.cluster.oc1.ap-tokyo-1.aaaaaaaay65nbbpqwgvt23p6grixlvavj2rnowlzckm6x7lyrcftl6arjcha
oci ce cluster create-kubeconfig --cluster-id $CLUSTER_ID --file $HOME/.kube/config --region ap-tokyo-1 --token-version 2.0.0 --kube-endpoint PUBLIC_ENDPOINT
# suggest add below to .bash_profile
export KUBECONFIG=$HOME/.kube/config
安装kubectl，详见这里。
确认kubectl可以访问到cluster
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
10.0.10.170 Ready node 96m v1.25.4
10.0.10.21 Ready node 96m v1.25.4
10.0.10.9 Ready node 96m v1.25.4
Lab 2: Install the Oracle Database Kubernetes Operator
Introduction
Operator 是一个特定于应用程序的控制器，它扩展了 Kubernetes 以创建、配置和管理复杂应用程序的实例。 Oracle 数据库 Kubernetes Operator管理 kubernetes 集群上数据库部署的管理和操作。
Task 1: Prepare to pull the DB Docker image
要从 Oracle Container Registry 访问包含 Oracle 21c 企业版数据库的预配置 docker 镜像，您需要登录并接受所需的开发人员许可协议。这个操作只需做一次。
登录Oracle Container Registry ，中间一系列操作省略。
最终获得数据库企业版的Pull Command for Latest，但我们现在无需运行他：
docker pull
container-registry.oracle.com/database/enterprise:latest
Task 2: Install the operator using the Cloud Shell
kubectl apply -f https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml
kubectl apply -f https://raw.githubusercontent.com/oracle/oracle-database-operator/main/oracle-database-operator.yaml
获取安装状态：
$ kubectl get pod -n oracle-database-operator-system -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
oracle-database-operator-controller-manager-5747ccc95-fqsnp 1/1 Running 0 13s 10.244.1.4 10.0.10.130 <none> <none>
oracle-database-operator-controller-manager-5747ccc95-h2qqr 1/1 Running 0 13s 10.244.0.4 10.0.10.43 <none> <none>
oracle-database-operator-controller-manager-5747ccc95-t82n2 1/1 Running 0 13s 10.244.0.133 10.0.10.95 <none> <none>
确保其状态就绪，例如READY为1/1，STATUS为Running。
否则由于Oracle Database Kubernetes Operator未就绪，后续可能出现错误。例如：
$ kubectl apply -f https://objectstorage.us-ashburn-1.oraclecloud.com/p/LNAcA6wNFvhkvHGPcWIbKlyGkicSOVCIgWLIu6t7W2BQfwq2NSLCsXpTL9wVzjuP/n/c4u04/b/livelabsfiles/o/developer-library/singleinstancedatabase-fss.yaml
Error from server (InternalError): error when creating "https://objectstorage.us-ashburn-1.oraclecloud.com/p/LNAcA6wNFvhkvHGPcWIbKlyGkicSOVCIgWLIu6t7W2BQfwq2NSLCsXpTL9wVzjuP/n/c4u04/b/livelabsfiles/o/developer-library/singleinstancedatabase-fss.yaml": Internal error occurred: failed calling webhook "msingleinstancedatabase.kb.io": failed to call webhook: Post "https://oracle-database-operator-webhook-service.oracle-database-operator-system.svc:443/mutate-database-oracle-com-v1alpha1-singleinstancedatabase?timeout=10s": EOF
Lab 3: Deploy a SingleInstance Database with a dynamically allocated Block Volume
Introduction
动态持久性
在本实验中，我们将使用 Dynamic Persistence Provisioning，这是一种通过指定存储类自动配置的持久卷。 由于我们在 Oracle OCI 上运行，我们将使用 oci-bv 存储类。 此存储类有助于动态配置 OCI 块卷。 此类支持的访问模式是 ReadWriteOnce。 对于其他云提供商，您可以类似地使用他们的动态配置存储类。
我们还将动态配置卷的回收策略指定为删除。 在这种情况下，当相应的数据库部署被删除时，卷也会被删除。
目标：
创建在 Kubernetes 上运行的数据库，使用块卷作为持久性存储
Task 1: Store passwords in Kubernetes Secrets
创建数据库时，我们需要一些密码：
- 用于从 Oracle 容器存储库中提取 DB docker 容器的 Oracle 帐户密码
- 我们将要创建的数据库的管理员密码
首先要安装docker，参见这里。
使用Oracle网站用户名和口令登录：
docker login container-registry.oracle.com
登录成功后，会产生.docker目录：
$ ls -l .docker
total 4
-rw-------. 1 opc opc 125 Apr 5 02:01 config.json
创建secret，在用户HOME目录执行，即.docker目录的父目录:
cd ~
kubectl create secret generic oracle-container-registry-secret --from-file=.dockerconfigjson=.docker/config.json --type=kubernetes.io/dockerconfigjson
创建数据库管理员的口令：
kubectl create secret generic admin-secret --from-literal=oracle_pwd=Your-DB-Password
将 Your-DB-Password 替换为您选择的密码，长度应为 12 个字符，包含字母、数字和大写字母。
Task 2: Create the DB Config file for the Operator
要由 Operator 开始创建数据库，我们必须创建一个描述所需数据库设置的配置文件。 对于本实验，我们将使用文件 singleinstancedatabase-create.yaml，其中包含准备用于本实验这一部分的配置。
此文件内容如下，不需要修改：
#
# Copyright (c) 2021, Oracle and/or its affiliates.
# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl.
#
apiVersion: database.oracle.com/v1alpha1
kind: SingleInstanceDatabase
metadata:
name: sidb-test1
namespace: default
spec:
## Use only alphanumeric characters for sid
sid: ORCL1
## A source database ref to clone from, leave empty to create a fresh database
cloneFrom: ""
## NA if cloning from a SourceDB (cloneFrom is set)
edition: enterprise
## Should refer to SourceDB secret if cloning from a SourceDB (cloneFrom is set)
## Secret containing SIDB password mapped to secretKey
## This secret will be deleted after creation of the database unless keepSecret is set to true
adminPassword:
secretName: admin-secret
# secretKey:
keepSecret: true
## NA if cloning from a SourceDB (cloneFrom is set)
charset: AL32UTF8
## NA if cloning from a SourceDB (cloneFrom is set)
pdbName: orclpdb1
## Enable/Disable Flashback
flashBack: false
## Enable/Disable ArchiveLog
archiveLog: false
## Enable/Disable ForceLogging
forceLog: false
## NA if cloning from a SourceDB (cloneFrom is set)
## Specify both sgaSize and pgaSize (in MB) or dont specify both
## Specify Non-Zero value to use
initParams:
cpuCount: 0
processes: 0
sgaTarget: 0
pgaAggregateTarget: 0
## Database image details
## Database can be patched by updating the RU version/image
## Major version changes are not supported
image:
pullFrom: container-registry.oracle.com/database/enterprise:latest
pullSecrets: oracle-container-registry-secret
## size : Minimum size of pvc | class : PVC storage Class
## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany
## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes.
persistence:
size: 100Gi
storageClass: "oci"
accessMode: "ReadWriteOnce"
## Type of service . Applicable on cloud enviroments only
## if loadBalService : false, service type = "NodePort". else "LoadBalancer"
loadBalancer: true
serviceAnnotations:
service.beta.kubernetes.io/oci-load-balancer-shape: "flexible"
service.beta.kubernetes.io/oci-load-balancer-shape-flex-min: "10"
service.beta.kubernetes.io/oci-load-balancer-shape-flex-max: "10"
## Deploy only on nodes having required labels. Format label_name : label_value
## Leave empty if there is no such requirement.
## Uncomment to use
# nodeSelector:
# failure-domain.beta.kubernetes.io/zone: bVCG:PHX-AD-1
# pool: sidb
## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode
replicas: 1
在文件的顶层部分，参数 kind 指的是要创建的数据库类型，在这种情况下，我们将在 Kubernetes 集群上运行的容器中启动一个数据库，称为单实例数据库。 其他可能的选择是在 OCI 上使用自治数据库、使用外部容器数据库等。 参数名称定义了我们将要创建的 oracle 数据库名称，以及用于通过各种 kubectl 命令引用数据库的名称。
secretName 定义包含数据库密码的秘密名称。 您可以使用参数 keepSecret 指定在创建后删除此机密以增强安全性。
Image部分指定从何处拉取数据库容器Image。 在这种情况下，我们使用默认Image，您可以构建自定义图像并引用该Image。 参数 pullSecrets 指的是我们存储容器存储库凭据的Secret名称，在本例中为 Oracle 容器存储库。
persistence 部分定义了要使用的持久存储类型。 在这种情况下，我们将使用类 oci 来使用 OCI 块卷。 参数 accessMode 指定这是一个块卷，只能安装在集群的单个节点上。 在下一个实验中，我们将使用可以同时安装在多个节点上的 NFS 卷。
最后，参数 replicas 指定了我们想要启动和运行的 pod 数量。 由于这是一个基于块的卷，只能安装在我们使用的集群的单个节点上 。
Task 3: Launch and track the DB creation
启动数据库的创建是通过在集群上应用配置文件的单个命令完成的。 接下来，操作员将启动必要的操作来启动数据库，这在新环境中大约需要 15 分钟——例如，必须从容器存储库中复制Image。
在本节中，我们将解释一些命令，这些命令允许您跟踪此创建过程中发生的事情以及如何调试您可能遇到的问题。
应用配置文件以启动数据库创建：
kubectl apply -f https://objectstorage.us-ashburn-1.oraclecloud.com/p/LNAcA6wNFvhkvHGPcWIbKlyGkicSOVCIgWLIu6t7W2BQfwq2NSLCsXpTL9wVzjuP/n/c4u04/b/livelabsfiles/o/developer-library/singleinstancedatabase-create.yaml
验证实例定义已提交给Operator：
$ kubectl get singleinstancedatabase sidb-test1
NAME EDITION STATUS VERSION CONNECT STR TCPS CONNECT STR OEM EXPRESS URL
sidb-test1 Enterprise Pending Unavailable Unavailable Unavailable Unavailable
我们可以使用以下命令查看更多详细信息：
$ kubectl describe singleinstancedatabase sidb-test1
Name: sidb-test1
Namespace: default
Labels: <none>
Annotations: <none>
API Version: database.oracle.com/v1alpha1
Kind: SingleInstanceDatabase
Metadata:
Creation Timestamp: 2023-04-04T09:41:16Z
Finalizers:
database.oracle.com/singleinstancedatabasefinalizer
Generation: 1
Managed Fields:
API Version: database.oracle.com/v1alpha1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:kubectl.kubernetes.io/last-applied-configuration:
f:spec:
.:
f:adminPassword:
.:
f:keepSecret:
f:secretKey:
f:secretName:
f:archiveLog:
f:charset:
f:cloneFrom:
f:edition:
f:flashBack:
f:forceLog:
f:image:
.:
f:pullFrom:
f:pullSecrets:
f:initParams:
.:
f:cpuCount:
f:pgaAggregateTarget:
f:processes:
f:sgaTarget:
f:loadBalancer:
f:pdbName:
f:persistence:
.:
f:accessMode:
f:size:
f:storageClass:
f:replicas:
f:serviceAnnotations:
.:
f:service.beta.kubernetes.io/oci-load-balancer-shape:
f:service.beta.kubernetes.io/oci-load-balancer-shape-flex-max:
f:service.beta.kubernetes.io/oci-load-balancer-shape-flex-min:
f:sid:
Manager: kubectl-client-side-apply
Operation: Update
Time: 2023-04-04T09:41:16Z
API Version: database.oracle.com/v1alpha1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:finalizers:
.:
v:"database.oracle.com/singleinstancedatabasefinalizer":
Manager: manager
Operation: Update
Time: 2023-04-04T09:41:16Z
API Version: database.oracle.com/v1alpha1
Fields Type: FieldsV1
fieldsV1:
f:status:
.:
f:charset:
f:cloneFrom:
f:clusterConnectString:
f:conditions:
.:
k:{
"type":"ReconcileQueued"}:
.:
f:lastTransitionTime:
f:message:
f:observedGeneration:
f:reason:
f:status:
f:type:
f:connectString:
f:datafilesCreated:
f:datafilesPatched:
f:edition:
f:initParams:
f:isTcpsEnabled:
f:oemExpressUrl:
f:pdbConnectString:
f:pdbName:
f:persistence:
.:
f:accessMode:
f:size:
f:storageClass:
f:releaseUpdate:
f:replicas:
f:role:
f:sid:
f:status:
f:tcpsConnectString:
f:tcpsPdbConnectString:
Manager: manager
Operation: Update
Subresource: status
Time: 2023-04-04T09:42:02Z
Resource Version: 38177
UID: af85922d-00cf-4dfc-8edb-f4700ce005fa
Spec:
Admin Password:
Keep Secret: true
Secret Key: oracle_pwd
Secret Name: admin-secret
Charset: AL32UTF8
Edition: enterprise
Image:
Pull From: container-registry.oracle.com/database/enterprise:latest
Pull Secrets: oracle-container-registry-secret
Init Params:
Load Balancer: true
Pdb Name: orclpdb1
Persistence:
Access Mode: ReadWriteOnce
Size: 100Gi
Storage Class: oci
Replicas: 1
Service Annotations:
service.beta.kubernetes.io/oci-load-balancer-shape: flexible
service.beta.kubernetes.io/oci-load-balancer-shape-flex-max: 10
service.beta.kubernetes.io/oci-load-balancer-shape-flex-min: 10
Sid: ORCL1
Status:
Charset: AL32UTF8
Clone From: Unavailable
Cluster Connect String: sidb-test1-ext.default:1521/ORCL1
Conditions:
Last Transition Time: 2023-04-04T09:42:02Z
Message: no reconcile errors
Observed Generation: 1
Reason: LastReconcileCycleQueued
Status: True
Type: ReconcileQueued
Connect String: 131.186.59.114:1521/ORCL1
Datafiles Created: false
Datafiles Patched: false
Edition: Enterprise
Init Params:
Is Tcps Enabled: false
Oem Express URL: https://131.186.59.114:5500/em
Pdb Connect String: 131.186.59.114:1521/ORCLPDB1
Pdb Name: orclpdb1
Persistence:
Access Mode: ReadWriteOnce
Size: 100Gi
Storage Class: oci
Release Update: Unavailable
Replicas: 1
Role: Unavailable
Sid: ORCL1
Status: Pending
Tcps Connect String: Unavailable
Tcps Pdb Connect String: Unavailable
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Database Pending 4s (x5 over 49s) SingleInstanceDatabase waiting for a pod to get to running state
我们看到Operator正在等待 pod 可用。
我们可以检查将启动数据库的 pod 的状态，并遵循创建过程的不同步骤：
$ kubectl get pod
NAME READY STATUS RESTARTS AGE
sidb-test1-u9wze 0/1 Init:0/2 0 2m57s
要获得有关创建的更多详细信息，您可以发出以下命令，将 pod 的确切名称替换为您环境中的名称：
$ kubectl describe pod sidb-test1-u9wze
Name: sidb-test1-u9wze
Namespace: default
Priority: 0
Service Account: default
Node: 10.0.10.21/10.0.10.21
Start Time: Tue, 04 Apr 2023 09:41:35 +0000
Labels: app=sidb-test1
version=
Annotations: <none>
Status: Pending
IP:
IPs: <none>
Controlled By: SingleInstanceDatabase/sidb-test1
Init Containers:
init-permissions:
Container ID:
Image: container-registry.oracle.com/database/enterprise:latest
Image ID:
Port: <none>
Host Port: <none>
Command:
/bin/sh
-c
chown 54321:54321 /opt/oracle/oradata || true
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/opt/oracle/oradata from datamount (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-9dnst (ro)
init-wallet:
Container ID:
Image: container-registry.oracle.com/database/enterprise:latest
Image ID:
Port: <none>
Host Port: <none>
Command:
/bin/sh
Args:
-c
if [ ! -f $ORACLE_BASE/oradata/.${ORACLE_SID}${CHECKPOINT_FILE_EXTN} ] || [ ! -f ${ORACLE_BASE}/oradata/dbconfig/$ORACLE_SID/.docker_enterprise ]; then while [ ! -f ${WALLET_DIR}/ewallet.p12 ] || pgrep -f $WALLET_CLI > /dev/null; do sleep 0.5; done; fi
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment:
ORACLE_SID: ORCL1
WALLET_CLI: mkstore
WALLET_DIR: /opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet
Mounts:
/opt/oracle/oradata from datamount (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-9dnst (ro)
Containers:
sidb-test1:
Container ID:
Image: container-registry.oracle.com/database/enterprise:latest
Image ID:
Ports: 1521/TCP, 5500/TCP
Host Ports: 0/TCP, 0/TCP
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Readiness: exec [/bin/sh -c if [ -f $ORACLE_BASE/checkDBLockStatus.sh ]; then $ORACLE_BASE/checkDBLockStatus.sh ; else $ORACLE_BASE/checkDBStatus.sh; fi ] delay=20s timeout=20s period=60s #success=1 #failure=3
Environment:
SVC_HOST: sidb-test1
SVC_PORT: 1521
CREATE_PDB: true
ORACLE_SID: ORCL1
WALLET_DIR: /opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet
ORACLE_PDB: orclpdb1
ORACLE_CHARACTERSET: AL32UTF8
ORACLE_EDITION: enterprise
INIT_SGA_SIZE:
INIT_PGA_SIZE:
SKIP_DATAPATCH: true
Mounts:
/opt/oracle/oradata from datamount (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-9dnst (ro)
Conditions:
Type Status
Initialized False
Ready False
ContainersReady False
PodScheduled True
Volumes:
datamount:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: sidb-test1
ReadOnly: false
oracle-pwd-vol:
Type: Secret (a volume populated by a Secret)
SecretName: admin-secret
Optional: true
kube-api-access-9dnst:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 4m8s (x2 over 4m23s) default-scheduler 0/3 nodes are available: 3 pod has unbound immediate PersistentVolumeClaims. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling.
Normal Scheduled 4m5s default-scheduler Successfully assigned default/sidb-test1-u9wze to 10.0.10.21
Normal SuccessfulAttachVolume 3m49s attachdetach-controller AttachVolume.Attach succeeded for volume "ocid1.volume.oc1.ap-tokyo-1.abxhiljr3e6n22weyra4v7fcrdqyflbmj3b5cgzboetmgfpofhuhu5lmcajq"
Normal Pulling 3m37s kubelet Pulling image "container-registry.oracle.com/database/enterprise:latest"
Normal Pulled 6s kubelet Successfully pulled image "container-registry.oracle.com/database/enterprise:latest" in 3m31.220010488s
Normal Created 1s kubelet Created container init-permissions
此时在OCI Console中的Block Storage和Load Balancer中可以看到为其创建的100GB块存储和负载均衡器。
现在 pod 应该正在运行，但还没有准备好：重新发出以下命令：
$ kubectl get pod
NAME READY STATUS RESTARTS AGE
sidb-test1-u9wze 0/1 Running 0 6m1s
反复运行以下语句：
$ kubectl describe singleinstancedatabase sidb-test1
直到状态变为Healthy：
$ kubectl describe singleinstancedatabase sidb-te st1
Name: sidb-test1
Namespace: default
Labels: <none>
Annotations: <none>
API Version: database.oracle.com/v1alpha1
Kind: SingleInstanceDatabase
Metadata:
Creation Timestamp: 2023-04-04T09:41:16Z
Finalizers:
database.oracle.com/singleinstancedatabasefinalizer
Generation: 1
Managed Fields:
API Version: database.oracle.com/v1alpha1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:kubectl.kubernetes.io/last-applied-configuration:
f:spec:
.:
f:adminPassword:
.:
f:keepSecret:
f:secretKey:
f:secretName:
f:archiveLog:
f:charset:
f:cloneFrom:
f:edition:
f:flashBack:
f:forceLog:
f:image:
.:
f:pullFrom:
f:pullSecrets:
f:initParams:
.:
f:cpuCount:
f:pgaAggregateTarget:
f:processes:
f:sgaTarget:
f:loadBalancer:
f:pdbName:
f:persistence:
.:
f:accessMode:
f:size:
f:storageClass:
f:replicas:
f:serviceAnnotations:
.:
f:service.beta.kubernetes.io/oci-load-balancer-shape:
f:service.beta.kubernetes.io/oci-load-balancer-shape-flex-max:
f:service.beta.kubernetes.io/oci-load-balancer-shape-flex-min:
f:sid:
Manager: kubectl-client-side-apply
Operation: Update
Time: 2023-04-04T09:41:16Z
API Version: database.oracle.com/v1alpha1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:finalizers:
.:
v:"database.oracle.com/singleinstancedatabasefinalizer":
Manager: manager
Operation: Update
Time: 2023-04-04T09:41:16Z
API Version: database.oracle.com/v1alpha1
Fields Type: FieldsV1
fieldsV1:
f:status:
.:
f:archiveLog:
f:charset:
f:cloneFrom:
f:clusterConnectString:
f:conditions:
.:
k:{
"type":"ReconcileBlocked"}:
.:
f:lastTransitionTime:
f:message:
f:observedGeneration:
f:reason:
f:status:
f:type:
k:{
"type":"ReconcileComplete"}:
.:
f:lastTransitionTime:
f:message:
f:observedGeneration:
f:reason:
f:status:
f:type:
k:{
"type":"ReconcileQueued"}:
.:
f:lastTransitionTime:
f:message:
f:observedGeneration:
f:reason:
f:status:
f:type:
f:connectString:
f:datafilesCreated:
f:datafilesPatched:
f:edition:
f:flashBack:
f:forceLog:
f:initParams:
f:isTcpsEnabled:
f:oemExpressUrl:
f:pdbConnectString:
f:pdbName:
f:persistence:
.:
f:accessMode:
f:size:
f:storageClass:
f:releaseUpdate:
f:replicas:
f:role:
f:sid:
f:status:
f:tcpsConnectString:
f:tcpsPdbConnectString:
Manager: manager
Operation: Update
Subresource: status
Time: 2023-04-04T09:55:56Z
Resource Version: 42552
UID: af85922d-00cf-4dfc-8edb-f4700ce005fa
Spec:
Admin Password:
Keep Secret: true
Secret Key: oracle_pwd
Secret Name: admin-secret
Charset: AL32UTF8
Edition: enterprise
Image:
Pull From: container-registry.oracle.com/database/enterprise:latest
Pull Secrets: oracle-container-registry-secret
Init Params:
Load Balancer: true
Pdb Name: orclpdb1
Persistence:
Access Mode: ReadWriteOnce
Size: 100Gi
Storage Class: oci
Replicas: 1
Service Annotations:
service.beta.kubernetes.io/oci-load-balancer-shape: flexible
service.beta.kubernetes.io/oci-load-balancer-shape-flex-max: 10
service.beta.kubernetes.io/oci-load-balancer-shape-flex-min: 10
Sid: ORCL1
Status:
Archive Log: false
Charset: AL32UTF8
Clone From: Unavailable
Cluster Connect String: sidb-test1-ext.default:1521/ORCL1
Conditions:
Last Transition Time: 2023-04-04T09:54:41Z
Message: no pod is ready currently
Observed Generation: 1
Reason: LastReconcileCycleQueued
Status: True
Type: ReconcileQueued
Last Transition Time: 2023-04-04T09:54:54Z
Message: processing datapatch execution
Observed Generation: 1
Reason: LastReconcileCycleBlocked
Status: True
Type: ReconcileBlocked
Last Transition Time: 2023-04-04T09:55:56Z
Message: no reconcile errors
Observed Generation: 1
Reason: LastReconcileCycleCompleted
Status: True
Type: ReconcileComplete
Connect String: 131.186.59.114:1521/ORCL1
Datafiles Created: true
Datafiles Patched: true
Edition: Enterprise
Flash Back: false
Force Log: false
Init Params:
Is Tcps Enabled: false
Oem Express URL: https://131.186.59.114:5500/em
Pdb Connect String: 131.186.59.114:1521/ORCLPDB1
Pdb Name: orclpdb1
Persistence:
Access Mode: ReadWriteOnce
Size: 100Gi
Storage Class: oci
Release Update: 21.3.0.0.0
Replicas: 1
Role: PRIMARY
Sid: ORCL1
Status: Healthy
Tcps Connect String: Unavailable
Tcps Pdb Connect String: Unavailable
Events: <none>
[opc@instance-kubectl-client ~]$ kubectl describe singleinstancedatabase sidb-test1
Name: sidb-test1
Namespace: default
Labels: <none>
Annotations: <none>
API Version: database.oracle.com/v1alpha1
Kind: SingleInstanceDatabase
Metadata:
Creation Timestamp: 2023-04-04T09:41:16Z
Finalizers:
database.oracle.com/singleinstancedatabasefinalizer
Generation: 1
Managed Fields:
API Version: database.oracle.com/v1alpha1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.:
f:kubectl.kubernetes.io/last-applied-configuration:
f:spec:
.:
f:adminPassword:
.:
f:keepSecret:
f:secretKey:
f:secretName:
f:archiveLog:
f:charset:
f:cloneFrom:
f:edition:
f:flashBack:
f:forceLog:
f:image:
.:
f:pullFrom:
f:pullSecrets:
f:initParams:
.:
f:cpuCount:
f:pgaAggregateTarget:
f:processes:
f:sgaTarget:
f:loadBalancer:
f:pdbName:
f:persistence:
.:
f:accessMode:
f:size:
f:storageClass:
f:replicas:
f:serviceAnnotations:
.:
f:service.beta.kubernetes.io/oci-load-balancer-shape:
f:service.beta.kubernetes.io/oci-load-balancer-shape-flex-max:
f:service.beta.kubernetes.io/oci-load-balancer-shape-flex-min:
f:sid:
Manager: kubectl-client-side-apply
Operation: Update
Time: 2023-04-04T09:41:16Z
API Version: database.oracle.com/v1alpha1
Fields Type: FieldsV1
fieldsV1:
f:metadata:
f:finalizers:
.:
v:"database.oracle.com/singleinstancedatabasefinalizer":
Manager: manager
Operation: Update
Time: 2023-04-04T09:41:16Z
API Version: database.oracle.com/v1alpha1
Fields Type: FieldsV1
fieldsV1:
f:status:
.:
f:archiveLog:
f:charset:
f:cloneFrom:
f:clusterConnectString:
f:conditions:
.:
k:{
"type":"ReconcileBlocked"}:
.:
f:lastTransitionTime:
f:message:
f:observedGeneration:
f:reason:
f:status:
f:type:
k:{
"type":"ReconcileComplete"}:
.:
f:lastTransitionTime:
f:message:
f:observedGeneration:
f:reason:
f:status:
f:type:
k:{
"type":"ReconcileQueued"}:
.:
f:lastTransitionTime:
f:message:
f:observedGeneration:
f:reason:
f:status:
f:type:
f:connectString:
f:datafilesCreated:
f:datafilesPatched:
f:edition:
f:flashBack:
f:forceLog:
f:initParams:
f:isTcpsEnabled:
f:oemExpressUrl:
f:pdbConnectString:
f:pdbName:
f:persistence:
.:
f:accessMode:
f:size:
f:storageClass:
f:releaseUpdate:
f:replicas:
f:role:
f:sid:
f:status:
f:tcpsConnectString:
f:tcpsPdbConnectString:
Manager: manager
Operation: Update
Subresource: status
Time: 2023-04-04T09:55:56Z
Resource Version: 42552
UID: af85922d-00cf-4dfc-8edb-f4700ce005fa
Spec:
Admin Password:
Keep Secret: true
Secret Key: oracle_pwd
Secret Name: admin-secret
Charset: AL32UTF8
Edition: enterprise
Image:
Pull From: container-registry.oracle.com/database/enterprise:latest
Pull Secrets: oracle-container-registry-secret
Init Params:
Load Balancer: true
Pdb Name: orclpdb1
Persistence:
Access Mode: ReadWriteOnce
Size: 100Gi
Storage Class: oci
Replicas: 1
Service Annotations:
service.beta.kubernetes.io/oci-load-balancer-shape: flexible
service.beta.kubernetes.io/oci-load-balancer-shape-flex-max: 10
service.beta.kubernetes.io/oci-load-balancer-shape-flex-min: 10
Sid: ORCL1
Status:
Archive Log: false
Charset: AL32UTF8
Clone From: Unavailable
Cluster Connect String: sidb-test1-ext.default:1521/ORCL1
Conditions:
Last Transition Time: 2023-04-04T09:54:41Z
Message: no pod is ready currently
Observed Generation: 1
Reason: LastReconcileCycleQueued
Status: True
Type: ReconcileQueued
Last Transition Time: 2023-04-04T09:54:54Z
Message: processing datapatch execution
Observed Generation: 1
Reason: LastReconcileCycleBlocked
Status: True
Type: ReconcileBlocked
Last Transition Time: 2023-04-04T09:55:56Z
Message: no reconcile errors
Observed Generation: 1
Reason: LastReconcileCycleCompleted
Status: True
Type: ReconcileComplete
Connect String: 131.186.59.114:1521/ORCL1
Datafiles Created: true
Datafiles Patched: true
Edition: Enterprise
Flash Back: false
Force Log: false
Init Params:
Is Tcps Enabled: false
Oem Express URL: https://131.186.59.114:5500/em
Pdb Connect String: 131.186.59.114:1521/ORCLPDB1
Pdb Name: orclpdb1
Persistence:
Access Mode: ReadWriteOnce
Size: 100Gi
Storage Class: oci
Release Update: 21.3.0.0.0
Replicas: 1
Role: PRIMARY
Sid: ORCL1
Status: Healthy
Tcps Connect String: Unavailable
Tcps Pdb Connect String: Unavailable
Events: <none>
或用以下命令确认：
$ kubectl get singleinstancedatabase sidb-test1
NAME EDITION STATUS VERSION CONNECT STR TCPS CONNECT STR OEM EXPRESS URL
sidb-test1 Enterprise Healthy 21.3.0.0.0 131.186.59.114:1521/ORCL1 Unavailable https://131.186.59.114:5500/em
Task 4: Connect to the Database
好的，看起来我们的数据库确实已启动并正在运行！ 让我们尝试连接到企业管理器……
单击上面输出中的链接，使用你的 IP 地址，在我的例子中是 https://131.186.59.114:5500/em。
输入用户名sys及其口令，输入PDB名称orclpdb1，然后连上了。
获取数据库连接串：
$ kubectl get singleinstancedatabase sidb-test1 -o "jsonpath={.status.pdbConnectString}" && echo -e "\n"
131.186.59.114:1521/ORCLPDB1
使用easy connect方式连接（这里需要预先安装sqlplus），或者使用任一带sqlplus的客户端也可以：
$ sqlplus sys@131.186.59.114:1521/ORCLPDB1 as sysdba
SQL*Plus: Release 19.0.0.0.0 - Production on Tue Apr 4 12:33:52 2023
Version 19.17.0.0.0
Copyright (c) 1982, 2022, Oracle. All rights reserved.
Connected to:
Oracle Database 21c Enterprise Edition Release 21.0.0.0.0 - Production
Version 21.3.0.0.0
SYS@131.186.59.114:1521/ORCLPDB1> show parameter sga
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
allow_group_access_to_sga boolean FALSE
lock_sga boolean FALSE
pre_page_sga boolean TRUE
sga_max_size big integer 1536M
sga_min_size big integer 0
sga_target big integer 0
SYS@131.186.59.114:1521/ORCLPDB1> show parameter pga
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
pga_aggregate_limit big integer 2G
pga_aggregate_target big integer 512M
如果您想要释放此数据库占用的资源（Kubernetes集群中的Pod、块卷和负载平衡器），您可以发出以下命令删除数据库并自动删除相关资源：
kubectl delete singleinstancedatabase.database.oracle.com sidb-test1
确认删除：
$ kubectl get singleinstancedatabase sidb-test1
Error from server (NotFound): singleinstancedatabases.database.oracle.com "sidb-test1" not found
Lab 4: Deploy a SingleInstance Database with a static NFS filesystem
Introduction
静态持久性
在本实验中，我们将为我们的数据库使用静态持久性选项之一，因此我们将手动创建一个持久性卷并在我们的数据库配置文件中引用它。
您可以为此使用块卷，但结果与第一个实验中的一样：该卷只能安装在单个节点上，因此没有故障转移选项。
因此，我们将使用一个 NFS 卷，它可以同时安装在所有 3 个节点上，并允许我们演示当一个节点出现故障时会发生什么。
目标：创建在 Kubernetes 上运行的数据库，使用 NFS 卷作为持久性存储
Task 1: Prepare your NFS persistent volume
创建Mount Point和Export，过程略。唯一需要注意的是，选择网络时需要选择OKE的网络，以及K8S node所在的子网。
请记下安装点的 IP 地址，您将在稍后的设置中需要此信息。 在本例中，地址是 ：10.0.10.120。
记下Export Path：稍后您将在数据库配置文件中需要此信息： /FileSystem-20230405-0739-26
NFS卷创建后，此时的空间使用率为0。
拷贝文件系统的OCID，后续会用到。本例为ocid1.filesystem.oc1.ap_tokyo_1.aaaaaaaaaaafmqvpnzzhillqojxwiotboawxi33lpfxs2mjnmfsc2mia
最后设置网络访问策略，以允许对NFS的访问。过程略。
接下来我们需要创建一个指向我们刚刚创建的配置的 kubernetes 持久卷。
创建文件pv.yaml，内容如下：
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-vol-1
spec:
storageClassName: oci-fss
capacity:
storage: 224Gi
volumeMode: Filesystem
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
csi:
driver: fss.csi.oraclecloud.com
volumeHandle: "ocid1.filesystem.oc1.ap_tokyo_1.aaaaaaaaaaafmr5wnzzhillqojxwiotboawxi33lpfxs2mjnmfsc2mia:10.0.10.120:/FileSystem-20230405-0739-26"
volumeHandle的格式为："<OCID of the file system>:<Mount Target IP Address>:/<Export Path>"，需要替换为相应的值。
现在使用 kubectl 应用配置：
kubectl apply -f pv.yaml
Task 2: Creating the DB Config file for the Operator
要由 Operator 开始创建数据库，我们必须创建一个描述所需数据库设置的配置文件。 在本实验中，我们将使用文件 singleinstancedatabase_fss.yaml，其中包含准备好用于本部分实验的配置。
此文件无需修改，其内容为：
#
# Copyright (c) 2021, Oracle and/or its affiliates.
# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl.
#
apiVersion: database.oracle.com/v1alpha1
kind: SingleInstanceDatabase
metadata:
name: sidb-test2
namespace: default
spec:
## Use only alphanumeric characters for sid
sid: ORCL1
## A source database ref to clone from, leave empty to create a fresh database
cloneFrom: ""
## NA if cloning from a SourceDB (cloneFrom is set)
edition: enterprise
## Should refer to SourceDB secret if cloning from a SourceDB (cloneFrom is set)
## Secret containing SIDB password mapped to secretKey
## This secret will be deleted after creation of the database unless keepSecret is set to true
adminPassword:
secretName: admin-secret
# secretKey:
keepSecret: true
## NA if cloning from a SourceDB (cloneFrom is set)
charset: AL32UTF8
## NA if cloning from a SourceDB (cloneFrom is set)
pdbName: orclpdb1
## Enable/Disable Flashback
flashBack: false
## Enable/Disable ArchiveLog
archiveLog: false
## Enable/Disable ForceLogging
forceLog: false
## NA if cloning from a SourceDB (cloneFrom is set)
## Specify both sgaSize and pgaSize (in MB) or dont specify both
## Specify Non-Zero value to use
initParams:
cpuCount: 0
processes: 0
sgaTarget: 0
pgaAggregateTarget: 0
## Database image details
## Database can be patched by updating the RU version/image
## Major version changes are not supported
image:
pullFrom: container-registry.oracle.com/database/enterprise:latest
pullSecrets: oracle-container-registry-secret
## size : Minimum size of pvc | class : PVC storage Class
## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany
## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes.
persistence:
size: 224Gi
storageClass: "oci-fss"
accessMode: "ReadWriteMany"
volumeName: "nfs-vol-1"
## Type of service . Applicable on cloud enviroments only
## if loadBalService : false, service type = "NodePort". else "LoadBalancer"
loadBalancer: true
serviceAnnotations:
service.beta.kubernetes.io/oci-load-balancer-shape: "flexible"
service.beta.kubernetes.io/oci-load-balancer-shape-flex-min: "10"
service.beta.kubernetes.io/oci-load-balancer-shape-flex-max: "10"
## Deploy only on nodes having required labels. Format label_name : label_value
## Leave empty if there is no such requirement.
## Uncomment to use
# nodeSelector:
# failure-domain.beta.kubernetes.io/zone: bVCG:PHX-AD-1
# pool: sidb
## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode
replicas: 3
在文件的顶层部分，定义配置名称的参数名称现在是 sidb-test2。
persistence 部分定义了要使用的持久存储类型。 在这种情况下，我们将使用类 oci-fss。 参数 accessMode 现在定义为 ReadWriteMany，因为我们将运行多个都可以访问该卷的 pod。 只有一个 pod 将保存活动数据库，其他的处于备用状态。 参数 volumeName 是指向我们之前定义的持久卷 nfs-vol-1 的链接。
最后，参数 replicas 指定了我们想要启动和运行的 pod 数量。 我们将在集群的每个节点上运行一个 pod，因此我们将此参数设置为 3。
Task 3: Launching and tracking the DB creation
应用配置文件以启动数据库创建：
kubectl apply -f https://objectstorage.us-ashburn-1.oraclecloud.com/p/LNAcA6wNFvhkvHGPcWIbKlyGkicSOVCIgWLIu6t7W2BQfwq2NSLCsXpTL9wVzjuP/n/c4u04/b/livelabsfiles/o/developer-library/singleinstancedatabase-fss.yaml
您可以使用下面的一组命令来验证在上一个实验中创建数据库的过程：
kubectl get singleinstancedatabase sidb-test2
kubectl describe singleinstancedatabase sidb-test2
kubectl get pod
kubectl describe pod sidb-test2-<your_id>
请注意，您将看到 3 个 pod，选择 1 以验证 pod 的正确启动。
一些可能对调试任何问题有用的额外命令：
kubectl logs sidb-test2-<your_id>
kubectl get pod -n oracle-database-operator-system
kubectl logs -n oracle-database-operator-system oracle-database-operator-controller-manager-<your-id>
例如：
$ kubectl get pod -n oracle-database-operator-system
NAME READY STATUS RESTARTS AGE
oracle-database-operator-controller-manager-5747ccc95-5bs4b 1/1 Running 0 4h7m
oracle-database-operator-controller-manager-5747ccc95-wh6vv 1/1 Running 0 4h7m
oracle-database-operator-controller-manager-5747ccc95-wnfj9 1/1 Running 0 4h7m
下面这个命令的尾部，SuccessfulAttachVolume显示可以识别到NFS卷：
$ kubectl describe pod sidb-test1-u9wze
Error from server (NotFound): pods "sidb-test1-u9wze" not found
[opc@instance-20230405-1333-kubectl ~]$ kubectl describe pod sidb-test1-7c7pn
Name: sidb-test1-7c7pn
Namespace: default
Priority: 0
Service Account: default
Node: 10.0.10.43/10.0.10.43
Start Time: Wed, 05 Apr 2023 06:09:10 +0000
Labels: app=sidb-test1
version=
Annotations: <none>
Status: Pending
IP:
IPs: <none>
Controlled By: SingleInstanceDatabase/sidb-test1
Init Containers:
init-permissions:
Container ID:
Image: container-registry.oracle.com/database/enterprise:latest
Image ID:
Port: <none>
Host Port: <none>
Command:
/bin/sh
-c
chown 54321:54321 /opt/oracle/oradata || true
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/opt/oracle/oradata from datamount (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-kc7jd (ro)
init-wallet:
Container ID:
Image: container-registry.oracle.com/database/enterprise:latest
Image ID:
Port: <none>
Host Port: <none>
Command:
/bin/sh
Args:
-c
if [ ! -f $ORACLE_BASE/oradata/.${ORACLE_SID}${CHECKPOINT_FILE_EXTN} ] || [ ! -f ${ORACLE_BASE}/oradata/dbconfig/$ORACLE_SID/.docker_enterprise ]; then while [ ! -f ${WALLET_DIR}/ewallet.p12 ] || pgrep -f $WALLET_CLI > /dev/null; do sleep 0.5; done; fi
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment:
ORACLE_SID: ORCL1
WALLET_CLI: mkstore
WALLET_DIR: /opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet
Mounts:
/opt/oracle/oradata from datamount (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-kc7jd (ro)
Containers:
sidb-test1:
Container ID:
Image: container-registry.oracle.com/database/enterprise:latest
Image ID:
Ports: 1521/TCP, 5500/TCP
Host Ports: 0/TCP, 0/TCP
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Readiness: exec [/bin/sh -c if [ -f $ORACLE_BASE/checkDBLockStatus.sh ]; then $ORACLE_BASE/checkDBLockStatus.sh ; else $ORACLE_BASE/checkDBStatus.sh; fi ] delay=20s timeout=20s period=60s #success=1 #failure=3
Environment:
SVC_HOST: sidb-test1
SVC_PORT: 1521
CREATE_PDB: true
ORACLE_SID: ORCL1
WALLET_DIR: /opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet
ORACLE_PDB: orclpdb1
ORACLE_CHARACTERSET: AL32UTF8
ORACLE_EDITION: enterprise
INIT_SGA_SIZE:
INIT_PGA_SIZE:
SKIP_DATAPATCH: true
Mounts:
/opt/oracle/oradata from datamount (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-kc7jd (ro)
Conditions:
Type Status
Initialized False
Ready False
ContainersReady False
PodScheduled True
Volumes:
datamount:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: sidb-test1
ReadOnly: false
oracle-pwd-vol:
Type: Secret (a volume populated by a Secret)
SecretName: admin-secret
Optional: true
kube-api-access-kc7jd:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 47s (x2 over 62s) default-scheduler 0/3 nodes are available: 3 pod has unbound immediate PersistentVolumeClaims. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling.
Normal Scheduled 45s default-scheduler Successfully assigned default/sidb-test1-7c7pn to 10.0.10.43
Normal SuccessfulAttachVolume 28s attachdetach-controller AttachVolume.Attach succeeded for volume "ocid1.volume.oc1.ap-tokyo-1.abxhiljrljffpdycnmwlwssndvw73kuym4u4dx4nlv3surth33yd7lw3dzxa"
Normal Pulling 16s kubelet Pulling image "container-registry.oracle.com/database/enterprise:latest"
数据库启动并运行后，您可以返回到 OCI 控制台，导航到文件系统菜单并验证卷的利用率是否增加，通常是 4 GiB（这么看来，每一个POD都安装了数据库软件，而共享的NFS上只是数据文件）。
获取连接串：
$ kubectl get singleinstancedatabase sidb-test2 -o "jsonpath={.status.pdbConnectString}" && echo -e "\n"
150.230.105.36:1521/ORCLPDB1
使用sqlplus连接：
sqlplus sys/[email protected]:1521/ORCLPDB1 as sysdba
Lab 5: Failover to another node and redeploy the DB
Introduction
在本实验中，我们将使用具有我们在上一个实验中设置的 NFS 卷持久性的数据库，并验证故障转移是否发生在不同的集群节点上。
- 首先我们将创建一个表并插入一条记录
- 接下来，我们将手动停止运行数据库的 pod，并检查它是否出现在另一个节点上
- 我们将重新连接到数据库以验证我们的数据仍然存在
目标：演示数据库故障转移
Task 1: Insert some records in the database
获取连接串：
kubectl get singleinstancedatabase sidb-test2 -o "jsonpath={.status.pdbConnectString}" && echo -e "\n"
连接数据库：
sqlplus sys/[email protected]:1521/ORCLPDB1 as sysdba
插入测试数据：
create table mytest (my_id int, mytext varchar(255) ) ;
insert into mytest values (10,'Hello World');
select * from mytest;
检查活动 pod 的 IP 地址：
kubectl get pods -o wide
输出如下：
$ kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
sidb-test2-55zoj 0/1 Running 4 (5m53s ago) 27m 10.244.0.134 10.0.10.73 <none> <none>
sidb-test2-n4upz 1/1 Running 1 (21m ago) 31m 10.244.0.5 10.0.10.213 <none> <none>
sidb-test2-vht46 0/1 Running 5 (3m15s ago) 27m 10.244.1.5 10.0.10.253 <none> <none>
您可以看到 sidb-test2 数据库的第二个 pod 处于就绪状态(因为其READY状态为1/1)，这是数据库处于活动状态的 pod。 记下对应的节点地址（上例中为10.244.0.5）。
您应该会看到集群的 3 个实例，其私有 IP 地址与上一个命令的输出相对应。
- 单击具有活动 pod 的实例
- 单击停止按钮停止实例，然后单击停止实例按钮
- 在Cloud Shell中，可以输入命令kubectl get nodes来检测kubernetes集群检测到节点停止的时刻
- 现在重新发出命令以可视化 pod
kubectl get pods -o wide并注意到另一个 pod 已激活，在集群的另一个节点上运行
验证数据库仍然可用
- 通过sqlplus重新连接数据库
- 使用 select * from mytest 验证数据是否可用；
恭喜，您测试了跨集群节点的数据库故障转移！
如果你想释放这个数据库占用的资源（你的 Kubernetes 集群中的三个 pod 和一个负载均衡器），你可以发出以下命令来删除数据库：
kubectl delete singleinstancedatabase.database.oracle.com sidb-test2
您使用的 NFS 卷是单独创建的，并且在删除实例后仍然存在，因此您应该通过 OCI 控制台手动删除文件系统和挂载点。
您已完成本实验，您可以访问单实例数据库与 Oracle Database Operator for Kubernetes 页面上的文档以获取更多选项。
错误
$ sqlplus sys@150.230.105.36:1521/ORCL1 as sysdba
SQL*Plus: Release 21.0.0.0.0 - Production on Thu Apr 6 01:37:29 2023
Version 21.9.0.0.0
Copyright (c) 1982, 2022, Oracle. All rights reserved.
ERROR:
ORA-12547: TNS:lost contact
Enter user-name: