12c RAC打补丁

在某项目安装了最新的12.2.0.1的GI、RAC，之后打了2018年10月最新的补丁，特记录关键日志于此。

■Download the latest patch set for GI and Database
下载最新的补丁包
p28714316_122010_Linux-x86-64.zip

■Read README.html and apply patch
解压缩根据README文件操作
The GI Oct 2018 Release Update 12.2.0.1.181016 includes updates for both the Clusterware home and Database home that can be applied in a rolling fashion.
You must use the OPatch utility version 12.2.0.1.12 or later to apply this patch for all platforms.

■Download OPatch Utility and install it
需要下载最新的opatch程序来执行打补丁，将opatch程序安装到所有节点的GI home、database home
When patching the GI Home, a shared location on ACFS only needs to be unmounted on the node where the GI Home is being patched.
The new opatch utility should be updated in all the Oracle RAC database homes and the GI home that are being patched.
To update Opatch, use the following instructions:
1、Download the OPatch utility to a temporary directory.
2、For each Oracle RAC database home and the GI home that are being patched, run the following commands as the home owner to extract the OPatch utility.
$ unzip <OPATCH-ZIP> -d <ORACLE_HOME>
$ <ORACLE_HOME>/OPatch/opatch version
||
unzip /home/soft/p6880880_122010_Linux-x86-64.zip -d $ORACLE_HOME
$ORACLE_HOME/OPatch/opatch version

$ORACLE_HOME/OPatch/opatch lsinventory -detail -oh $ORACLE_HOME
There are no Interim patches installed in this Oracle Home.

■CheckConflictAgainstOHWithDetail
检查是否有冲突
For Grid Infrastructure Home, as home user:
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir ./28714316/28662603
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir ./28714316/28662626
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir ./28714316/28662640
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir ./28714316/26839277
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir ./28714316/28566910

For Database home, as home user:
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir ./28714316/28662603
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir ./28714316/28662626

■Run OPatch SystemSpace Check
检查空间是否满足
For Grid Infrastructure Home, as home user:
Create file /tmp/patch_list_gihome.txt with the following content:
vi /tmp/patch_list_gihome.txt
/home/soft/patch/28714316/28662603
/home/soft/patch/28714316/28662626
/home/soft/patch/28714316/28662640
/home/soft/patch/28714316/26839277
/home/soft/patch/28714316/28566910
Run the opatch command to check if enough free space is available in the Grid Infrastructure Home:
$ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /tmp/patch_list_gihome.txt

For Database home, as home user:
Create file /tmp/patch_list_dbhome.txt with the following content:
vi /tmp/patch_list_dbhome.txt
/home/soft/patch/28714316/28662603
/home/soft/patch/28714316/28662626
Run opatch command to check if enough free space is available in the Database Home:
$ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /tmp/patch_list_dbhome.txt

■One-off Patch Conflict Detection and Resolution
【一次性补丁冲突检查，此步骤可掠过】
/u01/app/12.2.0/grid/OPatch/opatchauto apply /home/soft/patch/28714316 -analyze

■opatchauto
使用自动方式打补丁
The utility must be executed by an operating system (OS) user with root privileges, and it must be executed on each node in the cluster if the GI home or Oracle RAC database home is in non-shared storage. The utility should not be run in parallel on the cluster nodes.【需用root用户在所有节点的GI home、database home执行，不能并行执行】

export PATH=$PATH:/u01/app/12.2.0/grid/OPatch
To patch the GI home and all Oracle RAC database homes of the same version:
opatchauto apply /home/soft/patch/28714316

报错，查看log：
2018-11-20 19:24:52,787 INFO  [57] oracle.dbsysmodel.driver.sdk.util.OsysUtility$ReaderThread - Can't locate Cwd.pm:   /root/perl5/lib/perl5/Cwd.pm: (null) at /u01/app/oracle/product/12.2.0/db_1/OPatch/auto/database/bin/RemoteHostExecutor.pl line 2.
可见是perl安装包的问题，yum没有现成的包可用，搜索可见oracle相关目录有可用的perl软件，如下

[root@cb2qdb1:~]find / -name "Cwd.pm"
/usr/lib64/perl5/vendor_perl/Cwd.pm
/u01/app/12.2.0/grid/perl/lib/5.22.0/x86_64-linux-thread-multi/Cwd.pm
/u01/app/oracle/product/12.2.0/db_1/perl/lib/5.22.0/x86_64-linux-thread-multi/Cwd.pm

建立软连接到/root/perl5/lib/perl5【或者拷贝到root下】
ln -s /u01/app/12.2.0/grid/perl/lib/5.22.0/x86_64-linux-thread-multi /root/perl5/lib/perl5

export PERL5LIB=/u01/app/12.2.0/grid/perl/lib/5.22.0【注意：这个环境变量PERL5LIB是必须的，否则也会报错，此处浪费半小时】
[root@cb2qdb1:~/perl5/lib]opatchauto apply /home/soft/patch/28714316

OPatchauto session is initiated at Tue Nov 20 19:42:57 2018

System initialization log file is /u01/app/12.2.0/grid/cfgtoollogs/opatchautodb/systemconfig2018-11-20_07-42-59PM.log.

Session log file is /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/opatchauto2018-11-20_07-43-38PM.log
The id for this session is V8PS

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/12.2.0/db_1

Executing OPatch prereq operations to verify patch applicability on home /u01/app/12.2.0/grid
Patch applicability verified successfully on home /u01/app/oracle/product/12.2.0/db_1

Patch applicability verified successfully on home /u01/app/12.2.0/grid

Verifying SQL patch applicability on home /u01/app/oracle/product/12.2.0/db_1
SQL patch applicability verified successfully on home /u01/app/oracle/product/12.2.0/db_1

Preparing to bring down database service on home /u01/app/oracle/product/12.2.0/db_1
Successfully prepared home /u01/app/oracle/product/12.2.0/db_1 to bring down database service

Bringing down CRS service on home /u01/app/12.2.0/grid
Prepatch operation log file location: /u01/app/grid/crsdata/cb2qdb1/crsconfig/crspatch_cb2qdb1_2018-11-20_07-44-53PM.log
CRS service brought down successfully on home /u01/app/12.2.0/grid

Performing prepatch operation on home /u01/app/oracle/product/12.2.0/db_1
Perpatch operation completed successfully on home /u01/app/oracle/product/12.2.0/db_1

Start applying binary patch on home /u01/app/oracle/product/12.2.0/db_1
Binary patch applied successfully on home /u01/app/oracle/product/12.2.0/db_1

Performing postpatch operation on home /u01/app/oracle/product/12.2.0/db_1
Postpatch operation completed successfully on home /u01/app/oracle/product/12.2.0/db_1

Start applying binary patch on home /u01/app/12.2.0/grid
Binary patch applied successfully on home /u01/app/12.2.0/grid

Starting CRS service on home /u01/app/12.2.0/grid
Postpatch operation log file location: /u01/app/grid/crsdata/cb2qdb1/crsconfig/crspatch_cb2qdb1_2018-11-20_07-51-03PM.log
CRS service started successfully on home /u01/app/12.2.0/grid

Preparing home /u01/app/oracle/product/12.2.0/db_1 after database service restarted
No step execution required.........

Trying to apply SQL patch on home /u01/app/oracle/product/12.2.0/db_1
SQL patch applied successfully on home /u01/app/oracle/product/12.2.0/db_1

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:cb2qdb1
RAC Home:/u01/app/oracle/product/12.2.0/db_1
Version:12.2.0.1.0
Summary:

==Following patches were SKIPPED:

Patch: /home/soft/patch/28714316/28662640
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /home/soft/patch/28714316/26839277
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /home/soft/patch/28714316/28566910
Reason: This patch is not applicable to this specified target type - "rac_database"

==Following patches were SUCCESSFULLY applied:

Patch: /home/soft/patch/28714316/28662603
Log: /u01/app/oracle/product/12.2.0/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2018-11-20_19-46-55PM_1.log

Patch: /home/soft/patch/28714316/28662626
Log: /u01/app/oracle/product/12.2.0/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2018-11-20_19-46-55PM_1.log

Host:cb2qdb1
CRS Home:/u01/app/12.2.0/grid
Version:12.2.0.1.0
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /home/soft/patch/28714316/26839277
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-11-20_19-48-17PM_1.log

Patch: /home/soft/patch/28714316/28566910
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-11-20_19-48-17PM_1.log

Patch: /home/soft/patch/28714316/28662603
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-11-20_19-48-17PM_1.log

Patch: /home/soft/patch/28714316/28662626
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-11-20_19-48-17PM_1.log

Patch: /home/soft/patch/28714316/28662640
Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2018-11-20_19-48-17PM_1.log

OPatchauto session completed at Tue Nov 20 20:01:18 2018
Time taken to complete the session 18 minutes, 21 seconds

在第二个节点执行如下步骤：

[root@cb2qdb2:~]mkdir /root/perl5/lib
ln -s /u01/app/12.2.0/grid/perl/lib/5.22.0/x86_64-linux-thread-multi /root/perl5/lib/perl5
export PATH=$PATH:/u01/app/12.2.0/grid/OPatch
export PERL5LIB=/u01/app/12.2.0/grid/perl/lib/5.22.0
opatchauto apply /home/soft/patch/28714316
一切顺利，然而，到最后一步时，网络断了
Preparing home /u01/app/oracle/product/12.2.0/db_1 after database service restarted
No step execution required.........

Trying to apply SQL patch on home /u01/app/oracle/product/12.2.0/db_1
The network connection was aborted by the local system.

回滚吧，奥特曼：
export PATH=$PATH:/u01/app/12.2.0/grid/OPatch
export PERL5LIB=/u01/app/12.2.0/grid/perl/lib/5.22.0
opatchauto rollback /home/soft/patch/28714316

之后再次重打补丁：
opatchauto apply /home/soft/patch/28714316
再次断网，再次回滚，再次重打，成功。

■Patch Post-Installation Instructions

After installing the patch, perform the following actions:
1、Apply conflict resolution patches as explained in Applying Conflict Resolution Patches.
2、If you are not using opatchauto, then load modified SQL files into the database, as explained in Loading Modified SQL Files into the Database.

■Verify
$ORACLE_HOME/OPatch/opatch lsinventory -detail -oh $ORACLE_HOME
select * from dba_registry_history;
SQL> select * from dba_registry_history;
ACTION_TIME
--------------------------------------------
ACTION                         NAMESPACE
------------------------------ -------------
VERSION                                ID
------------------------------ ----------
COMMENTS
--------------------------------------------

BOOTSTRAP                      DATAPATCH
12.2.0.1
RDBMS_12.2.0.1.0DBOCT2018RU_LINUX.X64_181002