将补丁解压到独立目录，确保grid和oracle都有权限识别读取,两个节点都需要确认权限：
例如：
mkdir /patch
chown -R grid:oinstall /patch
chmod -R 777 /patch
su - grid
opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /patch/31326369/31326369/31305339/31281355
opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /patch/31326369/31326369/31305339/31305087
opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /patch/31326369/31326369/31305339/31304218
opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /patch/31326369/31326369/31305339/31335188
su - oracle
opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /patch/31326369/31326369/31305339/31281355
opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /patch/31326369/31326369/31305339/31305087
以上预先检查很有必要。
root用户下分布在节点1和节点2执行
节点1执行成功:
/oracle/app/19c/grid/OPatch/opatchauto apply /patch/31326369/31326369/31305339
日志如下:
[root@rac1 31326369]# /oracle/app/19c/grid/OPatch/opatchauto apply /patch/31326369/31326369/31305339
OPatchauto session is initiated at Sun Jul 26 21:37:57 2020
System initialization log file is /oracle/app/19c/grid/cfgtoollogs/opatchautodb/systemconfig2020-07-26_09-37-59PM.log.
Session log file is /oracle/app/19c/grid/cfgtoollogs/opatchauto/opatchauto2020-07-26_09-38-19PM.log
The id for this session is R57Y
Executing OPatch prereq operations to verify patch applicability on home /oracle/app/oracle/product/19c/dbhome_1
Executing OPatch prereq operations to verify patch applicability on home /oracle/app/19c/grid
Patch applicability verified successfully on home /oracle/app/oracle/product/19c/dbhome_1
Patch applicability verified successfully on home /oracle/app/19c/grid
Verifying SQL patch applicability on home /oracle/app/oracle/product/19c/dbhome_1
SQL patch applicability verified successfully on home /oracle/app/oracle/product/19c/dbhome_1
Preparing to bring down database service on home /oracle/app/oracle/product/19c/dbhome_1
Successfully prepared home /oracle/app/oracle/product/19c/dbhome_1 to bring down database service
Bringing down CRS service on home /oracle/app/19c/grid
CRS service brought down successfully on home /oracle/app/19c/grid
Performing prepatch operation on home /oracle/app/oracle/product/19c/dbhome_1
Perpatch operation completed successfully on home /oracle/app/oracle/product/19c/dbhome_1
Start applying binary patch on home /oracle/app/oracle/product/19c/dbhome_1
Binary patch applied successfully on home /oracle/app/oracle/product/19c/dbhome_1
Performing postpatch operation on home /oracle/app/oracle/product/19c/dbhome_1
Postpatch operation completed successfully on home /oracle/app/oracle/product/19c/dbhome_1
Start applying binary patch on home /oracle/app/19c/grid
Binary patch applied successfully on home /oracle/app/19c/grid
Starting CRS service on home /oracle/app/19c/grid
CRS service started successfully on home /oracle/app/19c/grid
Preparing home /oracle/app/oracle/product/19c/dbhome_1 after database service restarted
No step execution required.........
Trying to apply SQL patch on home /oracle/app/oracle/product/19c/dbhome_1
SQL patch applied successfully on home /oracle/app/oracle/product/19c/dbhome_1
OPatchAuto successful.
--------------------------------Summary--------------------------------
Patching is completed successfully. Please find the summary as follows:
Host:rac1
RAC Home:/oracle/app/oracle/product/19c/dbhome_1
Version:19.0.0.0.0
Summary:
==Following patches were SKIPPED:
Patch: /patch/31326369/31326369/31305339/31304218
Reason: This patch is not applicable to this specified target type - "rac_database"
Patch: /patch/31326369/31326369/31305339/31335188
Reason: This patch is not applicable to this specified target type - "rac_database"
==Following patches were SUCCESSFULLY applied:
Patch: /patch/31326369/31326369/31305339/31281355
Log: /oracle/app/oracle/product/19c/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-26_21-42-23PM_1.log
Patch: /patch/31326369/31326369/31305339/31305087
Log: /oracle/app/oracle/product/19c/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-26_21-42-23PM_1.log
Host:rac1
CRS Home:/oracle/app/19c/grid
Version:19.0.0.0.0
Summary:
==Following patches were SUCCESSFULLY applied:
Patch: /patch/31326369/31326369/31305339/31281355
Log: /oracle/app/19c/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-26_21-45-01PM_1.log
Patch: /patch/31326369/31326369/31305339/31304218
Log: /oracle/app/19c/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-26_21-45-01PM_1.log
Patch: /patch/31326369/31326369/31305339/31305087
Log: /oracle/app/19c/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-26_21-45-01PM_1.log
Patch: /patch/31326369/31326369/31305339/31335188
Log: /oracle/app/19c/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-26_21-45-01PM_1.log
OPatchauto session completed at Sun Jul 26 21:55:19 2020
Time taken to complete the session 17 minutes, 22 seconds
[grid@rac1 ~]$ opatch lspatches
31335188;TOMCAT RELEASE UPDATE 19.0.0.0.0 (31335188)
31305087;OCW RELEASE UPDATE 19.8.0.0.0 (31305087)
31304218;ACFS RELEASE UPDATE 19.8.0.0.0 (31304218)
31281355;Database Release Update : 19.8.0.0.200714 (31281355)
OPatch succeeded.
[root@rac1 31326369]# su - oracle
Last login: Sun Jul 26 21:54:33 CST 2020
[oracle@rac1 ~]$ opatch lspatches
31305087;OCW RELEASE UPDATE 19.8.0.0.0 (31305087)
31281355;Database Release Update : 19.8.0.0.200714 (31281355)
OPatch succeeded.
节点2执行报错及处理：
[root@rac2 /]# /oracle/app/19c/grid/OPatch/opatchauto apply /patch/31326369/31326369/31305339
OPatchauto session is initiated at Sun Jul 26 21:59:27 2020
System initialization log file is /oracle/app/19c/grid/cfgtoollogs/opatchautodb/systemconfig2020-07-26_09-59-29PM.log.
Session log file is /oracle/app/19c/grid/cfgtoollogs/opatchauto/opatchauto2020-07-26_09-59-52PM.log
The id for this session is DEC6
Executing OPatch prereq operations to verify patch applicability on home /oracle/app/19c/grid
Executing OPatch prereq operations to verify patch applicability on home /oracle/app/oracle/product/19c/dbhome_1
Patch applicability verified successfully on home /oracle/app/oracle/product/19c/dbhome_1
Patch applicability verified successfully on home /oracle/app/19c/grid
Verifying SQL patch applicability on home /oracle/app/oracle/product/19c/dbhome_1
SQL patch applicability verified successfully on home /oracle/app/oracle/product/19c/dbhome_1
Preparing to bring down database service on home /oracle/app/oracle/product/19c/dbhome_1
Successfully prepared home /oracle/app/oracle/product/19c/dbhome_1 to bring down database service
Bringing down CRS service on home /oracle/app/19c/grid
CRS service brought down successfully on home /oracle/app/19c/grid
Performing prepatch operation on home /oracle/app/oracle/product/19c/dbhome_1
Perpatch operation completed successfully on home /oracle/app/oracle/product/19c/dbhome_1
Start applying binary patch on home /oracle/app/oracle/product/19c/dbhome_1
Binary patch applied successfully on home /oracle/app/oracle/product/19c/dbhome_1
Performing postpatch operation on home /oracle/app/oracle/product/19c/dbhome_1
Postpatch operation completed successfully on home /oracle/app/oracle/product/19c/dbhome_1
Start applying binary patch on home /oracle/app/19c/grid
Failed while applying binary patches on home /oracle/app/19c/grid
Execution of [OPatchAutoBinaryAction] patch action failed, check log for more details. Failures:
Patch Target : rac2->/oracle/app/19c/grid Type[crs]
Details: [
---------------------------Patching Failed---------------------------------
Command execution failed during patching in home: /oracle/app/19c/grid, host: rac2.
Command failed: /oracle/app/19c/grid/OPatch/opatchauto apply /patch/31326369/31326369/31305339 -oh /oracle/app/19c/grid -target_type cluster -binary -invPtrLoc /oracle/app/19c/grid/oraInst.loc -jre /oracle/app/19c/grid/OPatch/jre -persistresult /oracle/app/19c/grid/opatchautocfg/db/sessioninfo/sessionresult_rac2_crs_1.ser -analyzedresult /oracle/app/19c/grid/opatchautocfg/db/sessioninfo/sessionresult_analyze_rac2_crs_1.ser
Command failure output:
==Following patches FAILED in apply:
Patch: /patch/31326369/31326369/31305339/31281355
Log: /oracle/app/19c/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-26_22-06-23PM_1.log
Reason: Failed during Patching: oracle.opatch.opatchsdk.OPatchException: ApplySession failed in system modification phase... 'ApplySession::apply failed: java.io.IOException: oracle.sysman.oui.patch.PatchException: java.io.FileNotFoundException: /oracle/app/oraInventory/ContentsXML/oui-patch.xml (Permission denied)'
After fixing the cause of failure Run opatchauto resume
]
OPATCHAUTO-68061: The orchestration engine failed.
OPATCHAUTO-68061: The orchestration engine failed with return code 1
OPATCHAUTO-68061: Check the log for more details.
OPatchAuto failed.
OPatchauto session completed at Sun Jul 26 22:09:40 2020
Time taken to complete the session 10 minutes, 13 seconds
opatchauto failed with error code 42
[root@rac2 /]# ll /oracle/app/oraInventory/ContentsXML/oui-patch.xml
-rw-r--r-- 1 oracle oinstall 174 Jul 26 22:06 /oracle/app/oraInventory/ContentsXML/oui-patch.xml
[root@rac2 /]# chown -R grid:oinstall /oracle/app/oraInventory/ContentsXML/oui-patch.xml
[root@rac2 /]# ll /oracle/app/oraInventory/ContentsXML/oui-patch.xml
-rw-r--r-- 1 grid oinstall 174 Jul 26 22:06 /oracle/app/oraInventory/ContentsXML/oui-patch.xml
[root@rac2 /]# /oracle/app/19c/grid/OPatch/opatchauto apply /patch/31326369/31326369/31305339 -oh /oracle/app/19c/grid -target_type cluster -binary -invPtrLoc /oracle/app/19c/grid/oraInst.loc -jre /oracle/app/19c/grid/OPatch/jre -persistresult /oracle/app/19c/grid/opatchautocfg/db/sessioninfo/sessionresult_rac2_crs_1.ser -analyzedresult /oracle/app/19c/grid/opatchautocfg/db/sessioninfo/sessionresult_analyze_rac2_crs_1.ser
Oracle Home : /oracle/app/19c/grid
OPatchAuto binary patching Tool
Copyright (c)2014, Oracle Corporation. All rights reserved.
OPatchauto Version : 13.9.5.0.0
Running from : /oracle/app/19c/grid
opatchauto log file: /oracle/app/19c/grid/cfgtoollogs/opatchauto/opatchauto_2020-07-26_22-29-59_binary.log
Target type : cluster
Patch selected: /patch/31326369/31326369/31305339
Analyze session result created from file :: /oracle/app/19c/grid/opatchautocfg/db/sessioninfo/sessionresult_analyze_rac2_crs_1.ser
Patch /patch/31326369/31326369/31305339/31281355 failed due to analyze failure.
opatchauto FAILED on some patches.
[root@rac2 /]# ll /oracle/app/oraInventory
total 12
drwxrwx--- 3 grid oinstall 35 Jul 21 20:47 backup
drwxrwx--- 2 grid oinstall 107 Jul 26 22:09 ContentsXML
drwxrwx--- 2 grid oinstall 46 Jul 26 22:07 locks
drwxrwx--- 2 grid oinstall 4096 Jul 26 22:06 logs
-rw-rw---- 1 grid oinstall 59 Jul 21 20:32 oraInst.loc
-rwxrwx--- 1 grid oinstall 1644 Jul 21 20:32 orainstRoot.sh
[root@rac2 /]# /oracle/app/19c/grid/OPatch/opatchauto apply /patch/31326369/31326369/31305339
OPatchauto session is initiated at Sun Jul 26 22:30:49 2020
System initialization log file is /oracle/app/19c/grid/cfgtoollogs/opatchautodb/systemconfig2020-07-26_10-30-56PM.log.
Clusterware is either not running or not configured. You have the following 2 options:
1. Configure and start the Clusterware on this node and re-run the tool
2. Run the tool with '-oh <GI_HOME>' to first patch the Grid Home, then invoke tool with '-database <oracle database name>' or '-oh <RAC_HOME>' to patch the RAC home
OPATCHAUTO-72029: CLusterware home not configured.
OPATCHAUTO-72029: Clusterware is either not running or not configured or cluster is software only GI
OPATCHAUTO-72029: If only Grid Infrastructure software is installed, please run opatchauto with '-oh' option . Alternatively configure and start the Clusterware before running opatchauto.
OPatchauto session completed at Sun Jul 26 22:31:01 2020
Time taken to complete the session 0 minute, 12 seconds
Topology creation failed.
[root@rac2 /]# pwd
/
[root@rac2 /]# su - grid
Last login: Sun Jul 26 22:06:22 CST 2020
[grid@rac2 ~]$ which crsctl
/oracle/app/19c/grid/bin/crsctl
[grid@rac2 ~]$ exit
logout
[root@rac2 /]# ps -ef|grep d.bin
root 95233 3221 0 22:31 pts/0 00:00:00 grep --color=auto d.bin
[root@rac2 /]# /oracle/app/19c/grid/bin/crsctl start crs
CRS-6706: Oracle Clusterware Release patch level ('4172947236') does not match Software patch level ('724960844'). Oracle Clusterware cannot be started.
CRS-4000: Command Start failed, or completed with errors.
由于打补丁过程失败，所以本地集群crs服务无法启动，自动打补丁需要指定grid家目录操作：
[root@rac2 /]# /oracle/app/19c/grid/OPatch/opatchauto apply /patch/31326369/31326369/31305339 -oh /oracle/app/19c/grid
OPatchauto session is initiated at Sun Jul 26 22:32:59 2020
System initialization log file is /oracle/app/19c/grid/cfgtoollogs/opatchautodb/systemconfig2020-07-26_10-33-02PM.log.
Session log file is /oracle/app/19c/grid/cfgtoollogs/opatchauto/opatchauto2020-07-26_10-33-04PM.log
The id for this session is VLHE
Executing OPatch prereq operations to verify patch applicability on home /oracle/app/19c/grid
Patch applicability verification failed on home /oracle/app/19c/grid
Execution of [OPatchAutoBinaryAction] patch action failed, check log for more details. Failures:
Patch Target : rac2->/oracle/app/19c/grid Type[crs]
Details: [
---------------------------Patching Failed---------------------------------
Command execution failed during patching in home: /oracle/app/19c/grid, host: rac2.
Command failed: /oracle/app/19c/grid/OPatch/opatchauto apply /patch/31326369/31326369/31305339 -oh /oracle/app/19c/grid -target_type cluster -binary -invPtrLoc /oracle/app/19c/grid/oraInst.loc -jre /oracle/app/19c/grid/OPatch/jre -persistresult /oracle/app/19c/grid/opatchautocfg/db/sessioninfo/sessionresult_analyze_rac2_crs_1.ser -analyze -online -prepare_home
Command failure output:
==Following patches FAILED in analysis for apply:
Patch: /patch/31326369/31326369/31305339/31305087
Log:
Reason: Failed during listing in Analysis: java.lang.Exception: oracle.opatch.opatchsdk.OPatchException: Unable to create patchObject
Possible causes are:
ORACLE_HOME/inventory/oneoffs/31281355 is corrupted. PatchObject constructor: Input file "/oracle/app/19c/grid/inventory/oneoffs/31281355/etc/config/actions" or "/oracle/app/19c/grid/inventory/oneoffs/31281355/etc/config/inventory" does not exist.
Patch: /patch/31326369/31326369/31305339/31304218
Log:
Reason: Failed during listing in Analysis: java.lang.Exception: oracle.opatch.opatchsdk.OPatchException: Unable to create patchObject
Possible causes are:
ORACLE_HOME/inventory/oneoffs/31281355 is corrupted. PatchObject constructor: Input file "/oracle/app/19c/grid/inventory/oneoffs/31281355/etc/config/actions" or "/oracle/app/19c/grid/inventory/oneoffs/31281355/etc/config/inventory" does not exist.
Patch: /patch/31326369/31326369/31305339/31335188
Log:
Reason: Failed during listing in Analysis: java.lang.Exception: oracle.opatch.opatchsdk.OPatchException: Unable to create patchObject
Possible causes are:
ORACLE_HOME/inventory/oneoffs/31281355 is corrupted. PatchObject constructor: Input file "/oracle/app/19c/grid/inventory/oneoffs/31281355/etc/config/actions" or "/oracle/app/19c/grid/inventory/oneoffs/31281355/etc/config/inventory" does not exist.
Patch: /patch/31326369/31326369/31305339/31281355
Log:
Reason: Failed during listing in Analysis: java.lang.Exception: oracle.opatch.opatchsdk.OPatchException: Unable to create patchObject
Possible causes are:
ORACLE_HOME/inventory/oneoffs/31281355 is corrupted. PatchObject constructor: Input file "/oracle/app/19c/grid/inventory/oneoffs/31281355/etc/config/actions" or "/oracle/app/19c/grid/inventory/oneoffs/31281355/etc/config/inventory" does not exist.
After fixing the cause of failure Run opatchauto resume
]
OPATCHAUTO-68061: The orchestration engine failed.
OPATCHAUTO-68061: The orchestration engine failed with return code 1
OPATCHAUTO-68061: Check the log for more details.
OPatchAuto failed.
OPatchauto session completed at Sun Jul 26 22:33:17 2020
Time taken to complete the session 0 minute, 18 seconds
opatchauto failed with error code 42
上述错误处理方法是从节点1拷贝目录到节点2即可：
[grid@rac1 ~]$ ll /oracle/app/19c/grid/inventory/oneoffs
total 0
drwxr-x--- 4 grid oinstall 29 Apr 18 2019 29401763
drwxr-x--- 4 grid oinstall 29 Apr 18 2019 29517242
drwxr-x--- 4 grid oinstall 29 Apr 18 2019 29517247
drwxr-x--- 4 grid oinstall 29 Apr 18 2019 29585399
drwxr-xr-x 4 grid oinstall 29 Jul 26 21:48 31281355
drwxr-xr-x 4 grid oinstall 29 Jul 26 21:48 31304218
drwxr-xr-x 4 grid oinstall 29 Jul 26 21:49 31305087
drwxr-xr-x 4 grid oinstall 29 Jul 26 21:50 31335188
[grid@rac1 ~]$ cd /oracle/app/19c/grid/inventory/oneoffs
[grid@rac1 oneoffs]$ scp -rp 31* rac2:/oracle/app/19c/grid/inventory/oneoffs/
inventory.xml 100% 276KB 68.2MB/s 00:00
actions.xml 100% 2289KB 100.0MB/s 00:00
inventory.xml 100% 18KB 17.6KB/s 00:01
actions.xml 100% 54KB 17.0MB/s 00:00
inventory.xml 100% 71KB 36.9MB/s 00:00
actions.xml 100% 227KB 77.7MB/s 00:00
inventory.xml 100% 1887 2.0MB/s 00:00
actions.xml 100% 2651 2.2MB/s 00:00
然后节点2重新执行打补丁操作：
[root@rac2 /]# ll /oracle/app/19c/grid/inventory/oneoffs
total 0
drwxr-x--- 4 grid oinstall 29 Jul 21 20:31 29401763
drwxr-x--- 4 grid oinstall 29 Jul 21 20:31 29517242
drwxr-x--- 4 grid oinstall 29 Jul 21 20:31 29517247
drwxr-x--- 4 grid oinstall 29 Jul 21 20:31 29585399
[root@rac2 /]#
[root@rac2 /]# ll /oracle/app/19c/grid/inventory/oneoffs
total 0
drwxr-x--- 4 grid oinstall 29 Jul 21 20:31 29401763
drwxr-x--- 4 grid oinstall 29 Jul 21 20:31 29517242
drwxr-x--- 4 grid oinstall 29 Jul 21 20:31 29517247
drwxr-x--- 4 grid oinstall 29 Jul 21 20:31 29585399
drwxr-xr-x 4 grid oinstall 29 Jul 26 21:48 31281355
drwxr-xr-x 4 grid oinstall 29 Jul 26 21:48 31304218
drwxr-xr-x 4 grid oinstall 29 Jul 26 21:49 31305087
drwxr-xr-x 4 grid oinstall 29 Jul 26 21:50 31335188
[root@rac2 /]# /oracle/app/19c/grid/OPatch/opatchauto apply /patch/31326369/31326369/31305339 -oh /oracle/app/19c/grid
OPatchauto session is initiated at Sun Jul 26 22:37:50 2020
System initialization log file is /oracle/app/19c/grid/cfgtoollogs/opatchautodb/systemconfig2020-07-26_10-37-52PM.log.
Session log file is /oracle/app/19c/grid/cfgtoollogs/opatchauto/opatchauto2020-07-26_10-37-55PM.log
The id for this session is 9PFE
Executing OPatch prereq operations to verify patch applicability on home /oracle/app/19c/grid
Patch applicability verified successfully on home /oracle/app/19c/grid
Checking shared status of home.....
Bringing down CRS service on home /oracle/app/19c/grid
CRS service brought down successfully on home /oracle/app/19c/grid
Start applying binary patch on home /oracle/app/19c/grid
Binary patch applied successfully on home /oracle/app/19c/grid
Checking shared status of home.....
Starting CRS service on home /oracle/app/19c/grid
CRS service started successfully on home /oracle/app/19c/grid
OPatchAuto successful.
--------------------------------Summary--------------------------------
Patching is completed successfully. Please find the summary as follows:
Host:rac2
CRS Home:/oracle/app/19c/grid
Version:19.0.0.0.0
Summary:
==Following patches were SKIPPED:
Patch: /patch/31326369/31326369/31305339/31281355
Reason: This patch is already been applied, so not going to apply again.
==Following patches were SUCCESSFULLY applied:
Patch: /patch/31326369/31326369/31305339/31304218
Log: /oracle/app/19c/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-26_22-38-18PM_1.log
Patch: /patch/31326369/31326369/31305339/31305087
Log: /oracle/app/19c/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-26_22-38-18PM_1.log
Patch: /patch/31326369/31326369/31305339/31335188
Log: /oracle/app/19c/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-26_22-38-18PM_1.log
OPatchauto session completed at Sun Jul 26 22:43:34 2020
Time taken to complete the session 5 minutes, 44 seconds
[root@rac2 /]#
[grid@rac2 ~]$ opatch lspatches
31335188;TOMCAT RELEASE UPDATE 19.0.0.0.0 (31335188)
31305087;OCW RELEASE UPDATE 19.8.0.0.0 (31305087)
31304218;ACFS RELEASE UPDATE 19.8.0.0.0 (31304218)
31281355;Database Release Update : 19.8.0.0.200714 (31281355)
OPatch succeeded.
[grid@rac2 ~]$ exit
logout
[root@rac2 /]# su - oracle
Last login: Sun Jul 26 22:06:21 CST 2020
[oracle@rac2 ~]$ opatch lspatches
31305087;OCW RELEASE UPDATE 19.8.0.0.0 (31305087)
31281355;Database Release Update : 19.8.0.0.200714 (31281355)
OPatch succeeded.
[root@rac2 /]# /oracle/app/19c/grid/bin/crsctl stop cluster -all
CRS-4639: Could not contact Oracle High Availability Services
CRS-4000: Command Stop failed, or completed with errors.
[root@rac2 /]# /oracle/app/19c/grid/bin/crsctl start crs
CRS-4123: Oracle High Availability Services has been started.