Solaris 10安装oracle10G-RAC笔记(含升级过程)

◆主要的参考文档:
[Solaris 10][Oracle 10g with RAC and ASM][Ver1[1].0].doc
此文档可以在网上搜到。

◆本次安装环境如下:
Sun E6900机器两台,操作系统版本solaris 10,盘阵是emc,主机通过光纤交换机连接到emc的盘阵。
集群软件采用oracle cluster,ocr和仲裁盘采用裸磁盘,存储管理方式采用ASM。

◆对操作系统安装的要求:
两台机器的操作系统10版本打补丁到最新,如下pkg需要安装好:
pkginfo -i SUNWarc SUNWbtool SUNWhea SUNWlibC SUNWlibm SUNWlibms SUNWsprot SUNWtoo SUNWi1of SUNWxwfnt
A机的ce1网卡作为公网网卡,且对应到主机名pmdb1,对应IP是10.154.38.134;
A机的ce3网卡作为私网网卡,对应IP是192.168.0.134;
B机的ce1网卡作为公网网卡,且对应到主机名pmdb2,对应公网IP 10.154.38.137;
B机的ce3网卡作为私网网卡,对应IP是192.168.0.137;
/oracle本地文件系统:至少50G;根目录至少50G;swap、/tmp按照标准配置即可

◆/etc/hosts如下:
127.0.0.1 localhost
10.154.38.134   pmdb1
10.154.38.135   pmdb1-vip
192.168.0.134   pmdb1-priv
10.154.38.137   pmdb2
10.154.38.138   pmdb2-vip
192.168.0.137   pmdb2-priv

◆用户环境建立:
groupadd -g 600 oinstall
groupadd -g 700 dba
useradd -u 700 -g oinstall -G dba -d /oracle oracle
passwd oracle
安装rac时还会用到nobody用户,检查nobody用户是否存在,如果用户不存在,则需要添加用户;

◆系统核心参数修改:
projadd -U oracle -K "project.max-shm-memory=(priv,4096MB,deny)" user.oracle
more /etc/project
projmod -s -K "project.max-sem-ids=(priv,256,deny)" user.oracle
可用oralce用户查看具体的配置:prctl -i project user.oracle

◆/etc/system如下:
oracle官方建议把shmmax的参数也加到/etc/system文件下(可见Doc id : 399895.1 from metalink)
set noexec_user_stack=1
set shmsys:shminfo_shmmax=4294967295

◆oracle用户.rhosts如下:
pmdb1 oracle
pmdb2 oracle
pmdb1-priv oracle
pmdb2-priv oracle

◆oracle用户.profile如下:
umask 022
ORACLE_SID=pmdb1
ORACLE_BASE=/oracle
ORACLE_HOME=$ORACLE_BASE/product/10.2/db
ORA_CRS_HOME=$ORACLE_BASE/product/10.2/crs
NLS_LANG=AMERICAN_AMERICA.ZHS16GBK
ORA_NLS33=$ORACLE_HOME/ocommon/nls/admin/data
PATH=$PATH:$ORACLE_HOME/bin:$ORA_CRS_HOME/bin
export ORACLE_SID ORACLE_BASE ORACLE_HOME ORA_CRS_HOME NLS_LANG ORA_NLS33 PATH

DISPLAY=10.154.34.245:0.0;export DISPLAY

◆建立相关路径:
mkdir -p /oracle/product/10.2/db
mkdir -p /oracle/product/10.2/crs

◆磁盘属性修改:
chown -R root:oinstall /dev/rdsk/emcpower0?;chmod -R 640 /dev/rdsk/emcpower0?
chown -R oracle:dba /dev/rdsk/emcpower1?;chmod -R 644 /dev/rdsk/emcpower1?

◆solaris下emc的磁盘使用规则:
emcpower0a->s0
emcpower0b->s1
emcpower0c->s2
emcpower0d->s3
emcpower0e->s4
emcpower0f->s5
emcpower0g->s6
emcpower0h->s7

◆磁盘使用列表:
/dev/rdsk/下可以看到c0*、c*、emc*是三条lun的路径,务必统一使用emcpower路径。
emcpower0g用作ocr;
emcpower1g用作voting disk;
emcpower2g-emcpower9g已经修改为从1磁道开始使用;
chown -R oracle:dba /dev/rdsk/emcpower[2-9]?;chmod -R 660 /dev/rdsk/emcpower[2-9]?
所以看到,一个关键的问题是,sun下面,asm磁盘必须跳过开始的0磁道,否则无法使用。这个问题,困扰了我几个小时,才找到答案。

[client(2557)]CRS-1006:The OCR location /dev/rdsk/emcpower0g is inaccessible. Details in /oracle/product/10.2/crs/log/pmdb1/client/ocrconfig_2557.log
c0t5006048AD52EA647d87-93已经label;
出错:
[  OCRRAW][1]proprinit:problem reading the bootblock or superbloc 22
删除,修改ocr磁盘的开始磁道为1或2,重装;(format使用的插曲:修改了patition后需要重新label一次,以把信息注入磁盘)

root@pmdb1 # /oracle/product/10.2/crs/root.sh
WARNING: directory '/oracle/product/10.2' is not owned by root
WARNING: directory '/oracle/product' is not owned by root
WARNING: directory '/oracle' is not owned by root
Checking to see if Oracle CRS stack is already configured

Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory '/oracle/product/10.2' is not owned by root
WARNING: directory '/oracle/product' is not owned by root
WARNING: directory '/oracle' is not owned by root
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: pmdb1 pmdb1-priv pmdb1
node 2: pmdb2 pmdb2-priv pmdb2
clscfg: Arguments check out successfully.

NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Oracle Cluster Registry for cluster has already been initialized
Startup will be queued to init within 30 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
Failure at final check of Oracle CRS stack.
10

无法使用ocr、voting disk的错误消除了,但是这次的错误如下:
root@pmdb1 # more alertpmdb1.log
2010-04-17 15:56:09.668
[cssd(5807)]CRS-1604:CSSD voting file is offline: /dev/rdsk/emcpower1g. Details in /oracle/product/10.2/crs/log/pmdb1/cssd/ocssd.log.
--->
[    CSSD]2010-04-17 15:56:09.695 [1] >ERROR:   clssnmvInit: Insufficient number of voting devices available (0 of 1)
[    CSSD]2010-04-17 15:56:09.695 [1] >ERROR:   clssnmInitNMInfo: Failed to initialize voting device
[    CSSD]2010-04-17 15:56:09.695 [1] >ERROR:   clssscmain: clssnmNMInitialize failed

另外一个节点的错误一样。

这次修改了两个关键地方就可以了,一个是磁盘属性严格修改为:
chown -R root:oinstall /dev/rdsk/emcpower0?;chmod -R 640 /dev/rdsk/emcpower0?
chown -R oracle:dba /dev/rdsk/emcpower1?;chmod -R 644 /dev/rdsk/emcpower1?
另外一个是完全清除ocr和voting disk的内容:
dd if=/dev/zero of=/dev/rdsk/emcpower0g bs=8192 count=12800
dd if=/dev/zero of=/dev/rdsk/emcpower1g bs=8192 count=12800

在第二个节点执行root.sh的时候,最后一步出错,在第一个节点手动执行vipca,按照提示修改即可。至此,ok:
$ crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora.pmdb1.gsd  application    ONLINE    ONLINE    pmdb1
ora.pmdb1.ons  application    ONLINE    ONLINE    pmdb1
ora.pmdb1.vip  application    ONLINE    ONLINE    pmdb1
ora.pmdb2.gsd  application    ONLINE    ONLINE    pmdb2
ora.pmdb2.ons  application    ONLINE    ONLINE    pmdb2
ora.pmdb2.vip  application    ONLINE    ONLINE    pmdb2

◆接下来安装oracle database软件;

◆下载最新的双机、数据库补丁;

◆安装补丁:
unzip p6810189_10204_Solaris-64.zip
/oracle/product/10.2/db/bin/crsctl check crs
/oracle/product/10.2/db/bin/crsctl stop crs

执行升级时,务必先选择ORA_CRS_HOME路径,首先对clusterware升级,然后再选择ORACLE_HOME对database软件升级。
升级clusterware的最后,提示如下:
To complete the installation of this patchset, you must perform the following tasks on each node:
1. Log in as the root user.
2. As the root user, perform the following tasks:

a. Shutdown the CRS daemons by issuing the following command:
/oracle/product/10.2/crs/bin/crsctl stop crs
b. Run the shell script located at:
/oracle/product/10.2/crs/install/root102.sh
This script will automatically start the CRS daemons on the
patched node upon completion.
3. After completing this procedure, proceed to the next node and repeat.

升级完db后的提示:
The iSQL*Plus URL is:
http://pmdb1:5560/isqlplus

The iSQL*Plus DBA URL is:
http://pmdb1:5560/isqlplus/dba

◆执行netca创建监听;

◆执行dbca创建asm;
创建时发现没有发现相应的裸磁盘,查看属性,没问题,最后查明,指定asm_diskstring为/dev/rdsk/emcpower*即可。
在这里,控诉下oracle,这样的弱智问题,会让初次设置的人,困扰很久,在这里,鼓捣了几个小时,才找到答案,汗!
周末的连轴转,呵呵,午饭都没吃,唉,算是把这个平台的RAC安装弄通了。

◆创建数据库,同时添加TAF连接,另外还要配置local_listener:
alter system set local_listener="(ADDRESS = (PROTOCOL = TCP)(HOST = 10.154.38.135)(PORT = 1521))" scope=both sid='pmdb1';
alter system set local_listener="(ADDRESS = (PROTOCOL = TCP)(HOST = 10.154.38.138)(PORT = 1521))" scope=both sid='pmdb2';

◆最终结果:
$ crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora....RVER.cs application    ONLINE    ONLINE    pmdb2
ora....db1.srv application    ONLINE    ONLINE    pmdb1
ora....db2.srv application    ONLINE    ONLINE    pmdb2
ora.pmdb.db    application    ONLINE    ONLINE    pmdb2
ora....b1.inst application    ONLINE    ONLINE    pmdb1
ora....b2.inst application    ONLINE    ONLINE    pmdb2
ora....SM1.asm application    ONLINE    ONLINE    pmdb1
ora....B1.lsnr application    ONLINE    ONLINE    pmdb1
ora.pmdb1.gsd  application    ONLINE    ONLINE    pmdb1
ora.pmdb1.ons  application    ONLINE    ONLINE    pmdb1
ora.pmdb1.vip  application    ONLINE    ONLINE    pmdb1
ora....SM2.asm application    ONLINE    ONLINE    pmdb2
ora....B2.lsnr application    ONLINE    ONLINE    pmdb2
ora.pmdb2.gsd  application    ONLINE    ONLINE    pmdb2
ora.pmdb2.ons  application    ONLINE    ONLINE    pmdb2
ora.pmdb2.vip  application    ONLINE    ONLINE    pmdb2

发表评论

电子邮件地址不会被公开。 必填项已用*标注