正式生产库,配置双节点的RAC + 单实例的 DATAGUARD

正式生产库,配置DATAGUARD


RAC+单实例DATAGUARD  配置


RAC 两节点:
192.1.0.101    rac1    
192.1.0.102    rac2

存储:ASM
DB_UNIQUE_NAME= racdb
实例:racdb1; racdb2;


DG:
192.1.0.103   dg1    
存储 :ASM

数据库名: racdb
DB_UNIQUE_NAME=stbdb;




一、系统准备

    1.1 DG 磁盘分区:
    
    本地磁盘,磁盘分区ASM DISK 对照表

    磁盘分区    磁盘大小    RAID    ASM NAME    ASM GROUPDISKNAME
    -----------------------------------------------------------------
    /dev/sdb    
    /dev/sdb1    300        GB    RAID5    RECOVERY1    +RECOVERY
    /dev/sdb2    300        GB    RAID5    RECOVERY2    +RECOVERY
    /dev/sdb3    4023.1  GB    RAID5    RECOVERY3    +RECOVERY

    /dev/sdc1    899.5    MB    RAID0    DATA1        +DATA    
    /dev/sdd1    899.5    GB    RAID0    DATA2        +DATA    

    /dev/sde1    899.5    GB    RAID0    DATA3        +DATA    
    /dev/sdf1    899.5    GB    RAID0    DATA4        +DATA    
    /dev/sdg1    899.5    GB    RAID0    DATA5        +DATA

    /dev/sdb 为多个小盘合并成为一个RAID5的磁盘,因考虑到多个小盘(8*146G)多分区不好管理,
    同时这几个多个小盘已使用了2年左右,合并一起做RAID5也可以做个容错处理。

    

    1.2    做一个SSH对待访问


    #在dg1上执行建立私匙,以方便拷贝文件

    # su - oracle
    $ mkdir ~/.ssh
    $ ssh-keygen -t rsa
    $ ssh-keygen -t dsa

    #将dg1的加入
    $ ssh dg1 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
    $ ssh dg1 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
    $ scp ~/.ssh/authorized_keys dg1:~/.ssh/authorized_keys

    (遇到一个问题:
        [oracle@rac1 .ssh]$ ssh dg1 date
        Warning: the RSA host key for 'dg1' differs from the key for the IP address '192.1.0.103'
        Offending key for IP in /home/oracle/.ssh/known_hosts:15
        Matching host key in /home/oracle/.ssh/known_hosts:17
        Are you sure you want to continue connecting (yes/no)? yes
        2013年 10月 12日 星期六 11:39:08 CST
        解决方法:
        查看known_hosts 内容,发现'192.1.0.103' 以前指定了其它的服务器,直接删除对应的其它服务器行。
        问题解决。
    )
    还有一个有趣的问题,在RAC1中配置了ssh 对等访问,系统会自动更新到其它的RAC节点中,比如我这里有RAC2;

    在单机上先安装一个GRID(ASM),再安装ORACLE.
    1.3 安装GRID
        【】为独立服务器配置ORACLE GRID INFRASTRUCTURE

        安装此项选择,还要选择一个ASM磁盘,是开始时没想到的。后来另外分出两个分区做ASM特征盘,
        有点象RAC中的RAC表决盘(或OCR磁盘)。


安装完成GRID后,查看状态如下:


[grid@dg1 ~]$ crs_stat -t
Name           Type           Target    State     Host        
------------------------------------------------------------
ora.CRS.dg     ora....up.type ONLINE    ONLINE    dg1         
ora.DATA.dg    ora....up.type ONLINE    ONLINE    dg1   
      
ora....ER.lsnr ora....er.type ONLINE    ONLINE    dg1         
ora....VERY.dg ora....up.type ONLINE    ONLINE    dg1         
ora.asm        ora.asm.type   ONLINE    ONLINE    dg1         
ora.cssd       ora.cssd.type  ONLINE    ONLINE    dg1         
ora.diskmon    ora....on.type OFFLINE   OFFLINE               
ora.evmd       ora.evm.type   ONLINE    ONLINE    dg1         
ora.ons        ora.ons.type   OFFLINE   OFFLINE               
[grid@dg1 ~]$

(红色部分为添加的ASM 磁盘组)



    1.4 安装ORACLE

    1.3,1.4 两个安装参考:
    http://blog.csdn.net/miyatang/article/details/6659839

    

先前测试的配置日志(http://blog.csdn.net/miyatang/article/details/9979579)

二、数据库配置

    
    2.1 监听/ASM 配置
    2.1.1 listener.ora

    [oracle@localhost admin]$ cat listener.ora
    # listener.ora Network Configuration File: /opt/app/oracle/product/11.2.0/network/admin/listener.ora
    # Generated by Oracle configuration tools.

    LISTENER =
      (DESCRIPTION_LIST =
        (DESCRIPTION =
          (ADDRESS = (PROTOCOL = TCP)(HOST = 192.1.0.103)(PORT = 1521))
        )
      )

    2.1.2 tnsnames.ora
    [oracle@localhost admin]$ cat tnsnames.ora
    # tnsnames.ora Network Configuration File: /opt/app/oracle/product/11.2.0/network/admin/tnsnames.ora
    # Generated by Oracle configuration tools.

    RACDB1 =
      (DESCRIPTION =
        (ADDRESS_LIST =
          (ADDRESS = (PROTOCOL = TCP)(HOST = 192.1.0.101)(PORT = 1521))
          (ADDRESS = (PROTOCOL = TCP)(HOST = 192.1.0.111)(PORT = 1521))
        )
        (CONNECT_DATA =
          (SERVICE_NAME = racdb)
          (INSTANCE_NAME = racdb1)
        )
      )

    RACDB =
      (DESCRIPTION =
        (ADDRESS = (PROTOCOL = TCP)(HOST = rac-scan)(PORT = 1521))
        (CONNECT_DATA =
          (SERVER = DEDICATED)
          (SERVICE_NAME = racdb)
        )
      )

    RACDB2 =
      (DESCRIPTION =
        (ADDRESS_LIST =
          (ADDRESS = (PROTOCOL = TCP)(HOST = 192.1.0.102)(PORT = 1521))
          (ADDRESS = (PROTOCOL = TCP)(HOST = 192.1.0.112)(PORT = 1521))
        )
        (CONNECT_DATA =
          (SERVICE_NAME = racdb)
          (INSTANCE_NAME = racdb2)
        )
      )


    stbdb =
      (DESCRIPTION =
        (ADDRESS = (PROTOCOL = TCP)(HOST = 192.1.0.103)(PORT = 1521))
        (CONNECT_DATA =
          (SERVER = DEDICATED)
          (SERVICE_NAME = stbdb)
        )
      )

    把stbdb 部分插入到两个节点的 tnsnames.ora 文件中。
    
    2.1.3 grid 用户下,使用asmca 建立 +DATA,+RECOVERY 两个ASM 磁盘组。
        测试查看ASM 磁盘组:
        [grid@localhost grid]$ asmcmd
        Connected to an idle instance.
        ASMCMD> ls
        ASMCMD-8102: no connection to ASM; command requires ASM to run
        
        修改grid 用户下的.bash_profile

            ORACLE_SID=+ASM; export ORACLE_SID         (原为:ORACLE_SID=+ASM1; 此内容为RAC中拷贝而来有两个实例ASM1,ASM2.)
            PATH=$ORACLE_HOME/bin:$PATH; export PATH
            

    修改后问题解决:

        [grid@localhost grid]$ asmcmd
        ASMCMD> ls
        CRS/
        DATA/
        RECOVERY/
        ASMCMD> exit

    2.2.配置参数:

        2.2.1、打开Forced Logging模式

        SQL> alter database force logging;
        Database altered.
        
        2.2.2
        SQL> alter system set log_archive_config='DG_CONFIG=(stbdb,racdb)';
        SQL> ALTER SYSTEM SET LOG_ARCHIVE_MAX_PROCESSES=30;
        SQL> ALTER SYSTEM SET REMOTE_LOGIN_PASSWORDFILE=EXCLUSIVE SCOPE=SPFILE;
        SQL> ALTER SYSTEM SET FAL_SERVER=stbdb;
        SQL> alter system set FAL_CLIENT=racdb;
        SQL> alter system set standby_file_management=auto;


        ALTER SYSTEM SET LOG_ARCHIVE_DEST_2='SERVICE=stbdb LGWR SYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=stbdb' scope=both;
        ALTER SYSTEM SET LOG_ARCHIVE_DEST_STATE_2=DEFER;
        --先设置成不活动的,后面再修改成:  ENABLE;
        --ALTER SYSTEM SET LOG_ARCHIVE_DEST_STATE_2 = ENABLE scope=both;


        #添加STANDBY LOGFILE:
        

        select group#,THREAD#,l.MEMBERS,l.STATUS from v$log l

            GROUP#    THREAD#    MEMBERS    STATUS
        1    1    1    2    INACTIVE
        2    2    1    2    CURRENT
        3    3    2    2    CURRENT
        4    4    2    2    ACTIVE
        5    5    1    2    ACTIVE
        6    6    2    2    ACTIVE




         alter database add STANDBY logfile thread 1 group 7 ;
         alter database add STANDBY logfile thread 1 group 8 ;
         alter database add STANDBY logfile thread 1 group 9 ;

         alter database add STANDBY logfile thread 2 group 10 ;
         alter database add STANDBY logfile thread 2 group 11 ;
         alter database add STANDBY logfile thread 2 group 12 ;


    #2.2 生成pfile,并修改。

        SQL> create pfile='/tmp/initstbdb.ora' from spfile;


        1.修改
            *.audit_file_dest='/opt/app/oracle/admin/racdb/adump'
          修改后:
            *.audit_file_dest='/opt/app/oracle/admin/stbdb/adump'

            建立目录 :
            mkdir -p /opt/app/oracle/admin/stbdb/adump
            chown oracle:oinstall  /opt/app/oracle/admin/stbdb/adump


        2.添加以下内容:

        *.db_file_name_convert='+DATA/racdb/datafile','+DATA/stbdb/datafile','+DATA/racdb/tempfile','+DATA/stbdb/tempfile'
        *.LOG_FILE_NAME_CONVERT='+DATA/racdb/onlinelog','+DATA/stbdb/onlinelog','+RECOVERY/racdb/onlinelog' ,'+RECOVERY/stbdb/onlinelog'


        修改前的 initstbdb.ora

            racdb1.__db_cache_size=13623099392
            racdb2.__db_cache_size=13623099392
            racdb2.__java_pool_size=67108864
            racdb1.__java_pool_size=67108864
            racdb2.__large_pool_size=67108864
            racdb1.__large_pool_size=67108864
            racdb1.__oracle_base='/opt/app/oracle'#ORACLE_BASE set from environment
            racdb2.__oracle_base='/opt/app/oracle'#ORACLE_BASE set from environment
            racdb2.__pga_aggregate_target=9462349824
            racdb1.__pga_aggregate_target=9462349824
            racdb2.__sga_target=17582522368
            racdb1.__sga_target=17582522368
            racdb2.__shared_io_pool_size=0
            racdb1.__shared_io_pool_size=0
            racdb1.__shared_pool_size=3556769792
            racdb2.__shared_pool_size=3690987520
            racdb2.__streams_pool_size=0
            racdb1.__streams_pool_size=67108864
            *.audit_file_dest='/opt/app/oracle/admin/racdb/adump'
            *.audit_trail='NONE'
            *.cluster_database=true
            *.compatible='11.2.0.0.0'
            *.control_files='+DATA/racdb/controlfile/current.267.812303153','+RECOVERY/racdb/controlfile/current.3283.812303153'
            *.db_block_size=8192
            *.db_create_file_dest='+DATA'
            *.db_domain=''
            *.db_flashback_retention_target=1440
            *.db_name='racdb'
            *.db_recovery_file_dest='+RECOVERY'
            *.db_recovery_file_dest_size=3221225472000
            *.diagnostic_dest='/opt/app/oracle'
            *.dispatchers='(PROTOCOL=TCP) (SERVICE=racdbXDB)'
            *.fal_client='RACDB'
            *.fal_server='DGDB'
            racdb2.instance_number=2
            racdb1.instance_number=1
            *.log_archive_config='DG_CONFIG=(dgdb,racdb)'
            *.log_archive_dest_2='SERVICE=dgdb LGWR ASYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=dgdb'
            *.log_archive_dest_state_2='DEFER'
            *.log_archive_format='%t_%s_%r.dbf'
            *.log_archive_max_processes=30
            racdb1.memory_max_target=29360128000
            *.memory_target=27031240704
            *.nls_language='SIMPLIFIED CHINESE'
            *.nls_territory='CHINA'
            *.open_cursors=5000
            *.processes=8000
            *.remote_listener='rac-scan:1521'
            *.remote_login_passwordfile='EXCLUSIVE'
            *.resource_manager_plan=''
            *.session_cached_cursors=100
            *.sessions=5000
            *.standby_file_management='MANUAL'
            racdb2.thread=2
            racdb1.thread=1
            racdb2.undo_tablespace='UNDOTBS2'
            racdb1.undo_tablespace='UNDO_TBS01'
            [oracle@localhost dbs]$

        修改后的initstbdb.ora

            [oracle@localhost dbs]$ cat initstbdb.ora
            stbdb.__db_cache_size=13623099392
            stbdb.__java_pool_size=67108864
            stbdb.__large_pool_size=67108864
            stbdb.__oracle_base='/opt/app/oracle'#ORACLE_BASE set from environment
            stbdb.__pga_aggregate_target=9462349824
            stbdb.__sga_target=17582522368
            stbdb.__shared_io_pool_size=0
            stbdb.__shared_pool_size=3556769792
            stbdb.__streams_pool_size=67108864
            *.audit_file_dest='/opt/app/oracle/admin/stbdb/adump'
            *.audit_trail='NONE'
            *.compatible='11.2.0.0.0'
            *.control_files='+DATA/racdb/controlfile/current.267.812303153','+RECOVERY/racdb/controlfile/current.3283.812303153'
            *.db_block_size=8192
            *.db_create_file_dest='+DATA'
            *.db_domain=''
            *.db_flashback_retention_target=1440
            *.db_name='racdb'
            db_unique_name='stbdb'
            *.db_recovery_file_dest='+RECOVERY'
            *.db_recovery_file_dest_size=900000000000
            *.diagnostic_dest='/opt/app/oracle'
            *.dispatchers='(PROTOCOL=TCP) (SERVICE=stbdb)'
            *.fal_client='STBDB'
            *.fal_server='RACDB1,RACDB2'
            stbdb.instance_number=1
            *.db_file_name_convert='+DATA/racdb/datafile','+DATA/stbdb/datafile','+DATA/racdb/tempfile','+DATA/stbdb/tempfile'
            *.LOG_FILE_NAME_CONVERT='+DATA/racdb/onlinelog','+DATA/stbdb/onlinelog','+RECOVERY/racdb/onlinelog' ,'+RECOVERY/stbdb/onlinelog'
            *.log_archive_config='DG_CONFIG=(dgdb,racdb)'
            *.log_archive_dest_2='SERVICE=dgdb LGWR ASYNC VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) DB_UNIQUE_NAME=dgdb'
            *.log_archive_dest_state_2='DEFER'
            *.log_archive_format='%t_%s_%r.dbf'
            *.log_archive_max_processes=30
            stbdb.memory_max_target=29360128000
            *.memory_target=27031240704
            *.nls_language='SIMPLIFIED CHINESE'
            *.nls_territory='CHINA'
            *.open_cursors=5000
            *.processes=8000
            *.remote_listener='stbdb:1521'
            *.remote_login_passwordfile='EXCLUSIVE'
            *.resource_manager_plan=''
            *.session_cached_cursors=100
            *.sessions=5000
            *.standby_file_management='AUTO'
            thread=1
            undo_tablespace='UNDOTBS1'





    #2.3创建控制文件 :
        切换两次日志及做两次归档
        SQL> alter system switch logfile;

        System altered.

        SQL> alter system switch logfile;

        System altered.

        SQL> alter system archive log current;

        System altered.

        SQL> alter system archive log current;

        System altered.


        SQL> alter database create standby controlfile as '/tmp/control01.ctl';

        Database altered.

    #2.4创建密码文件
        [oracle@ract1 ~]$ orapwd file=/tmp/orapwstbdb.ora password=sa force=y ignorecase=y





    #2.5把文件拷贝到 dg1 服务器:
        
        scp /tmp/orapwstbdb.ora dg1:/opt/app/oracle/product/11.2.0/dbs/orapwstbdb.ora
        scp /tmp/inittdb.ora dg1:/tmp/inittdb.ora

    #2.6 在STANDBY 端
        SQL> create spfile from pfile='/tmp/inittdb.ora';
        File created.
        SQL> startup nomount;

    #2.7复制数据库

    (执行命令要在PRIMARY 中的某一个节点中进行,不然会出错)

 rman target / auxiliary sys/oracle@stbdb nocatalog
        [oracle@dg1 dbs]$ rman target sys/oracle@racdb1 auxiliary sys/oracle@stbdb nocatalog

        rman>duplicate target database for standby from active database nofilenamecheck;




三、DATAGUARD 开启




    SQL> alter database recover managed standby database disconnect from session;

    Database altered.


    # 开启STANDBY 数据库

    ALTER DATABASE RECOVER MANAGED STANDBY DATABASE;

    #启用STANDBY数据库,进入RECOVER模式:

    ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT FROM SESSION;

    #启动到只读模式:
    alter database open read only;

    #转换到REAL-TIME模式下:
    --alter database recover managed standby database cancel;
    --ALTER DATABASE OPEN;
    ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE DISCONNECT from session;


四、测试验证




    #PRIMARY 端:

SQL>
SQL> select thread#,sequence#,status from v$log;

   THREAD#  SEQUENCE# STATUS
---------- ---------- ----------------
     1    57930 CURRENT
     1    57929 INACTIVE
     2    43251 INACTIVE
     2    43252 ACTIVE
     1    57928 INACTIVE
     2    43253 CURRENT

6 rows selected.
    SQL>
    #STANDBY 端

    SQL> select archived_thread#,archived_seq#,APPLIED_THREAD#,APPLIED_SEQ# from v$archive_dest_status;
ARCHIVED_THREAD# ARCHIVED_SEQ# APPLIED_THREAD# APPLIED_SEQ#
---------------- ------------- --------------- ------------
           0         0             0          0
           0         0             0          0
           0         0             0          0
           0         0             0          0
           0         0             0          0
           0         0             0          0
           0         0             0          0
           0         0             0          0
           0         0             0          0
           2     43252             2          43252




一、安装前准备 磁盘规划 使用iscsi共享磁盘做成raw设备 Vote_OCR /dev/sdb1 /dev/raw/raw1 Vote_OCR /dev/sdb2 /dev/raw/raw2 Vote_OCR /dev/sdb3 /dev/raw/raw3 Vote_OCR /dev/sdb5 /dev/raw/raw4 Vote_OCR /dev/sdb6 /dev/raw/raw5 DATA / dev/sdb7 /dev/raw/raw6 DATA dev/sdb8 /dev/raw/raw7 DATA dev/sdb9 /dev/raw/raw8 FLR dev/sdb10 /dev/raw/raw9 FLR dev/sdb11 /dev/raw/raw10 以下操作如果没有特殊说明在两个节点都做相同操作 网络及主机名规划 #public 192.168.10.10 node1 192.168.10.20 node2 #vip 192.168.10.100 node1vip 192.168.10.200 node2vip #private 192.168.20.10 node1priv 192.168.20.20 node2priv #scan 192.168.10.101 scanip 修改/etc/hosts文件添加以上内容 # vi /etc/hosts 192.168.10.10 node1 192.168.10.20 node2 #vip 192.168.10.100 node1vip 192.168.10.200 node2vip #private 192.168.20.10 node1priv 192.168.20.20 node2priv #scan 192.168.10.101 scanip 修改以下文件的最后一行 # vi /etc/sysconfig/network HOSTNAME=node1 ---第二台机器修改为node2 命令行修改 # hostname node1 第二节点修改为 node2 执行 [root@localhost ~]# su - [root@node1 ~]# 我们在安装虚拟机时添加了两块网卡 在这里我们把 eth0作为对外访问使用 eth1作为对对内访问使用 配置IP地址 使用setup命令修改IP 分别配置eth0和eth1(我们只演示node1配置,node2配置成相应的IP) 保存退出 重启使配置生效 3、关闭没必要的服务 chkconfig autofs off chkconfig acpid off chkconfig sendmail off chkconfig cups-config-daemon off chkconfig cpus off chkconfig xfs off chkconfig lm_sensors off chkconfig gpm off chkconfig openibd off chkconfig iiim off chkconfig pcmcia off chkconfig cpuspeed off chkconfig nfslock off chkconfig ip6tables off chkconfig rpcidmapd off chkconfig apmd off chkconfig sendmail off chkconfig arptables_jf off chkconifg microcode_ctl off chkconfig rpcgssd off 上述服务有不存在的会提示,服务中读取信息时出错:没有那个文件或目录。没问题,忽略。 停用NTP服务 /sbin/service ntpd stop chkconfig ntpd off mv /etc/ntp.conf /etc/ntp.conf.bak 重启所有节点 安装yum源规划 在/etc/yum.repos.d/ 目录下添加rhel5.repo文件 [root@node1 yum.repos.d]# pwd /etc/yum.repos.d [root@node1 yum.repos.d]# ls rhel5.repo [root@node1 yum.repos.d]# [root@node1 yum.repos.d]# vi rhel5.repo [Server] name=server baseurl=file:///mnt/Server/ enabled=1 gpgcheck=0 [ClusterStorage] name=server baseurl=file:///mnt/ClusterStorage/ enabled=1 gpgcheck=0 挂载光驱到/mnt目录 [root@node1 yum.repos.d]# mount /dev/hdc /mnt mount: block device /dev/hdc is write-protected, mounting read-only [root@node1 yum.repos.d]# cd /mnt/ [root@node1 mnt]# ls Server ClusterStorage -ld dr-xr-xr-x 3 root root 8192 2010-03-22 ClusterStorage dr-xr-xr-x 3 root root 557056 2010-03-22 Server [root@node1 mnt]# 刷新yum列表 [root@node1 mnt]# yum clean all Loaded plugins: rhnplugin, security Cleaning up Everything [root@node1 mnt]# 4、软件包规划 安装所有依赖包 binutils-* compat-libstdc++-* elfutils-libelf-* elfutils-libelf-devel-* gcc-* gcc-c++-* glibc-* glibc-common-* glibc-devel-* glibc-headers-* ksh-* libaio-* libaio-devel-* libgcc-* libstdc++-* make-* sysstat-* expat-* pdksh-* unixODBC-* 使用YUM安装 # yum install -y binutils-* compat-libstdc++-* elfutils-libelf-* elfutils-libelf-devel-* gcc-* gcc-c++-* glibc-* glibc-common-* glibc-devel-* glibc-headers-* ksh-* libaio-* libaio-devel-* libgcc-* libstdc++-* make-* sysstat-* expat-* pdksh-* unixODBC-* 5、建立用户和目录 用户及目录规划 /usr/sbin/groupadd -g 501 oinstall /usr/sbin/groupadd -g 502 dba /usr/sbin/groupadd -g 504 asmadmin /usr/sbin/groupadd -g 506 asmdba /usr/sbin/groupadd -g 507 asmoper /usr/sbin/useradd -u 501 -g oinstall -G asmadmin,asmdba,asmoper,dba grid /usr/sbin/useradd -u 502 -g oinstall -G dba,asmdba oracle # passwd grid Changing password for user grid. New UNIX password: BAD PASSWORD: it is too short Retype new UNIX password: passwd: all authentication tokens updated successfully. [root@node1 /]# passwd oracle Changing password for user oracle. New UNIX password: BAD PASSWORD: it is based on a dictionary word Retype new UNIX password: passwd: all authentication tokens updated successfully. 创建grid目录结构 mkdir -p /u01/app/oraInventory chown -R grid:oinstall /u01/app/oraInventory chmod -R 775 /u01/app/oraInventory mkdir -p /u01/app/grid chmod -R 775 /u01/app/grid chown -R grid:oinstall /u01/app/grid mkdir -p /u01/app/11.2.0/grid chown -R grid:oinstall /u01/app/11.2.0/grid chmod -R 775 /u01/app/11.2.0/grid 创建oracle目录结构 mkdir -p /u01/app/oracle mkdir /u01/app/oracle/cfgtoollogs chown -R oracle:oinstall /u01/app/oracle chmod -R 775 /u01/app/oracle mkdir -p /u01/app/oracle/product/11.2.0/db_1 chown -R oracle:oinstall /u01/app/oracle/product/11.2.0/db_1 chmod -R 775 /u01/app/oracle/product/11.2.0/db_1 mkdir –p /u01/software chmod -R 775 /u01 iscsi共享存储规划 (分为server端和client端,我们这里是实验环境,为了节省机器,有一台机器既做服务器同时也做客户端<node1+iscsi>,另一台只有客户端<node2>。注:node1这个虚拟机尽量内存稍微大一些),下面我们开始配置。 在node1上准备要使用的磁盘,并做成相应的raw设备 我们在配置虚拟机的时候加了一块20G的盘 sdb 安装和配置 Node1+iscsi端: [root@node1 mnt]# yum install -y *scsi* [root@node1 /]# vi /etc/tgt/targets.conf #添加以下几行 <target iqn.2012-09.com.example:server.target4> #注:红色部分是在我们存储客户端登录的时候需要 backing-store /dev/sdb # Becomes LUN 1 </target> [root@node1 /]# /etc/init.d/tgtd restart [root@node1 /]# chkconfig tgtd on [root@node1 /]#cd /etc/rc.d/rc5.d/ [root@node1 rc5.d]# mv S13iscsi S40iscsi [root@node1 rc5.d]# [root@node1 ~]# /etc/init.d/iscsi start [root@node1 ~] iscsiadm -m discovery -t st -p 192.168.10.10 <-----(存储地址) 192.168.10.10:3260,1 iqn.2012-09.com.example:server.target4 [root@node1 ~]#iscsiadm -m node -T iqn.2012-09.com.example:server.target4 -p 192.168.10.10 -l Logging in to [iface: default, target: iqn.2012-09.com.example:server.target4, portal: 192.168.10.10,3260] Login to [iface: default, target: iqn.2012-09.com.example:server.target4, portal: 192.168.10.10,3260]: successful 查看磁盘信息 [root@node1 /]# fdisk -l Node2端: [root@node2 ~]# yum install *scsi* -y [root@node2 ~]# /etc/init.d/iscsi start [root@node2 ~]# iscsiadm -m discovery -t st -p 192.168.10.10 <-----(存储地址) 192.168.10.10:3260,1 iqn.2012-09.com.example:server.target4 [root@node2 ~]#iscsiadm -m node -T iqn.2012-09.com.example:server.target4 -p 192.168.10.10 -l Logging in to [iface: default, target: iqn.2012-09.com.example:server.target4, portal: 192.168.10.10,3260] Login to [iface: default, target: iqn.2012-09.com.example:server.target4, portal: 192.168.10.10,3260]: successful [root@node2 /]# 查看磁盘信息 # fdisk -l 在其中一个节点上分区 /dev/sdb 分成10个2G大小的分区 Device Boot Start End Blocks Id System /dev/sdb1 1 1908 1953776 83 Linux /dev/sdb2 1909 3816 1953792 83 Linux /dev/sdb3 3817 5724 1953792 83 Linux /dev/sdb4 5725 20480 15110144 5 Extended /dev/sdb5 5725 7632 1953776 83 Linux /dev/sdb6 7633 9540 1953776 83 Linux /dev/sdb7 9541 11448 1953776 83 Linux /dev/sdb8 11449 13356 1953776 83 Linux /dev/sdb9 13357 15264 1953776 83 Linux /dev/sdb10 15265 17172 1953776 83 Linux /dev/sdb11 17173 19080 1953776 83 Linux [root@node2 ~]# fdisk -l /dev/sdb Disk /dev/sdb: 21.4 GB, 21474836480 bytes 64 heads, 32 sectors/track, 20480 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Device Boot Start End Blocks Id System /dev/sdb1 1 1908 1953776 83 Linux /dev/sdb2 1909 3816 1953792 83 Linux /dev/sdb3 3817 5724 1953792 83 Linux /dev/sdb4 5725 20480 15110144 5 Extended /dev/sdb5 5725 7632 1953776 83 Linux /dev/sdb6 7633 9540 1953776 83 Linux /dev/sdb7 9541 11448 1953776 83 Linux /dev/sdb8 11449 13356 1953776 83 Linux /dev/sdb9 13357 15264 1953776 83 Linux /dev/sdb10 15265 17172 1953776 83 Linux /dev/sdb11 17173 19080 1953776 83 Linux [root@node2 ~]# 做完分区之后 在两个节点上分别执行以下命令: # partprobe 配置raw设备(每个节点都操作) #vi /etc/udev/rules.d/60-raw.rules ACTION=="add", KERNEL=="sdb1", RUN+="/bin/raw /dev/raw/raw1 %N" ACTION=="add", KERNEL=="sdb2", RUN+="/bin/raw /dev/raw/raw2 %N" ACTION=="add", KERNEL=="sdb3", RUN+="/bin/raw /dev/raw/raw3 %N" ACTION=="add", KERNEL=="sdb5", RUN+="/bin/raw /dev/raw/raw4 %N" ACTION=="add", KERNEL=="sdb6", RUN+="/bin/raw /dev/raw/raw5 %N" ACTION=="add", KERNEL=="sdb7", RUN+="/bin/raw /dev/raw/raw6 %N" ACTION=="add", KERNEL=="sdb8", RUN+="/bin/raw /dev/raw/raw7 %N" ACTION=="add", KERNEL=="sdb9", RUN+="/bin/raw /dev/raw/raw8 %N" ACTION=="add", KERNEL=="sdb10", RUN+="/bin/raw /dev/raw/raw9 %N" ACTION=="add", KERNEL=="sdb11", RUN+="/bin/raw /dev/raw/raw10 %N" KERNEL=="raw*", OWNER="grid" GROUP="asmadmin", MODE="0660" # start_udev  启动 udev: [确定] [root@node1 ~]# ll /dev/raw/ 总计 0 crw-rw---- 1 grid asmadmin 162, 1 09-29 18:13 raw1 crw-rw---- 1 grid asmadmin 162, 10 09-29 18:13 raw10 crw-rw---- 1 grid asmadmin 162, 2 09-29 18:13 raw2 crw-rw---- 1 grid asmadmin 162, 3 09-29 18:13 raw3 crw-rw---- 1 grid asmadmin 162, 4 09-29 18:13 raw4 crw-rw---- 1 grid asmadmin 162, 5 09-29 18:13 raw5 crw-rw---- 1 grid asmadmin 162, 6 09-29 18:13 raw6 crw-rw---- 1 grid asmadmin 162, 7 09-29 18:13 raw7 crw-rw---- 1 grid asmadmin 162, 8 09-29 18:13 raw8 crw-rw---- 1 grid asmadmin 162, 9 09-29 18:13 raw9 [root@node1 ~]# 各类参数及所需配置设置 配置内核参数 vi /etc/sysctl.conf kernel.shmmni = 4096 kernel.sem = 250 32000 100 128 fs.file-max = 6553600 net.ipv4.ip_local_port_range = 9000 65500 net.core.rmem_default = 262144 net.core.rmem_max = 4194304 net.core.wmem_default = 262144 net.core.wmem_max = 1048576 # sysctl -p net.ipv4.ip_forward = 0 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.default.accept_source_route = 0 kernel.sysrq = 0 kernel.core_uses_pid = 1 net.ipv4.tcp_syncookies = 1 kernel.msgmnb = 65536 kernel.msgmax = 65536 kernel.shmmax = 68719476736 kernel.shmall = 4294967296 kernel.shmmni = 4096 kernel.sem = 250 32000 100 128 net.ipv4.ip_local_port_range = 9000 65500 net.core.rmem_default = 262144 net.core.rmem_max = 4194304 net.core.wmem_default = 262144 net.core.wmem_max = 1048576 fs.file-max = 6553600 fs.file-max = 6815744 fs.aio-max-nr = 1048576 修改limits文件,添加如下内容 vi /etc/security/limits.conf grid soft nproc 2047 grid hard nproc 16384 grid soft nofile 1024 grid hard nofile 65536 oracle soft nproc 2047 oracle hard nproc 16384 oracle soft nofile 1024 oracle hard nofile 65536 配置hangcheck-timer # modprobe hangcheck-timer hangcheck_tick=1 hangcheck_margin=10 hangcheck_reboot=1 修改pam文件 #vi /etc/pam.d/login 添加一行 session required pam_limits.so 修改profile文件 #vi /etc/profile if [ $USER = "oracle" ] || [ $USER = "grid" ]; then if [ $SHELL = "/bin/ksh" ]; then ulimit -p 16384 ulimit -n 65536 else ulimit -u 16384 -n 65536 fi umask 022 fi 配置ssh密钥互认(oracle和grid两个用户) [root@node1 u01]# su - oracle [oracle@node1 ~]$ [oracle@node1 ~]$ ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/home/oracle/.ssh/id_rsa): Created directory '/home/oracle/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/oracle/.ssh/id_rsa. Your public key has been saved in /home/oracle/.ssh/id_rsa.pub. The key fingerprint is: 5d:94:12:23:9c:5b:ae:e8:5e:23:fb:ce:65:bc:a6:23 oracle@node1 [oracle@node1 ~]$ cd .ssh/ [oracle@node1 .ssh]$ mv id_rsa.pub authorized_keys [oracle@node1 .ssh]$ ll 总计 8 -rw-r--r-- 1 oracle oinstall 394 09-27 21:52 authorized_keys -rw------- 1 oracle oinstall 1675 09-27 21:52 id_rsa [oracle@node1 .ssh]$ chmod 600 authorized_keys [oracle@node1 .ssh]$ cd .. [oracle@node1 ~]$ scp -r .ssh/ 192.168.10.20:/home/oracle The authenticity of host '192.168.10.20 (192.168.10.20)' can't be established. RSA key fingerprint is 95:ed:6d:87:61:00:27:ed:38:17:6c:e9:6c:c3:8a:1d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.10.20' (RSA) to the list of known hosts. oracle@192.168.10.20's password: id_rsa 100% 1675 1.6KB/s 00:00 authorized_keys 100% 394 0.4KB/s 00:00 known_hosts 100% 395 0.4KB/s 00:00 [oracle@node1 ~]$ [oracle@node1 ~]$ su - grid 口令: [grid@node1 ~]$ ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/home/grid/.ssh/id_rsa): Created directory '/home/grid/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/grid/.ssh/id_rsa. Your public key has been saved in /home/grid/.ssh/id_rsa.pub. The key fingerprint is: 09:b9:21:44:1e:fc:a1:94:6f:9a:e6:5e:ee:d2:76:e4 grid@node1 [grid@node1 ~]$ cd .ssh/ [grid@node1 .ssh]$ ls id_rsa id_rsa.pub [grid@node1 .ssh]$ mv id_rsa.pub authorized_keys [grid@node1 .ssh]$ chmod 600 authorized_keys [grid@node1 .ssh]$ cd .. [grid@node1 ~]$ scp -r .ssh/ 192.168.10.20:/home/grid The authenticity of host '192.168.10.20 (192.168.10.20)' can't be established. RSA key fingerprint is 95:ed:6d:87:61:00:27:ed:38:17:6c:e9:6c:c3:8a:1d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.10.20' (RSA) to the list of known hosts. grid@192.168.10.20's password: id_rsa 100% 1671 1.6KB/s 00:00 authorized_keys 100% 392 0.4KB/s 00:00 known_hosts 100% 395 0.4KB/s 00:00 [grid@node1 ~]$ 测试: #su - oracle [oracle@node1 ~]$ ssh node2 date The authenticity of host 'node2 (192.168.10.20)' can't be established. RSA key fingerprint is 95:ed:6d:87:61:00:27:ed:38:17:6c:e9:6c:c3:8a:1d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'node2' (RSA) to the list of known hosts. 2012年 09月 27日 星期四 21:57:17 CST [oracle@node1 ~]$ ssh node2priv date The authenticity of host 'node2priv (192.168.20.20)' can't be established. RSA key fingerprint is 95:ed:6d:87:61:00:27:ed:38:17:6c:e9:6c:c3:8a:1d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'node2priv,192.168.20.20' (RSA) to the list of known hosts. 2012年 09月 27日 星期四 21:57:28 CST [oracle@node1 ~]$ ssh node1 date The authenticity of host 'node1 (192.168.10.10)' can't be established. RSA key fingerprint is 95:ed:6d:87:61:00:27:ed:38:17:6c:e9:6c:c3:8a:1d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'node1,192.168.10.10' (RSA) to the list of known hosts. 2012年 09月 27日 星期四 21:57:45 CST [oracle@node1 ~]$ ssh node1priv date The authenticity of host 'node1priv (192.168.20.10)' can't be established. RSA key fingerprint is 95:ed:6d:87:61:00:27:ed:38:17:6c:e9:6c:c3:8a:1d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'node1priv,192.168.20.10' (RSA) to the list of known hosts. 2012年 09月 27日 星期四 21:58:02 CST [oracle@node1 ~]$ [oracle@node1 ~]$ ssh node2 [oracle@node2 ~]$ id uid=502(oracle) gid=501(oinstall) groups=501(oinstall),502(dba),506(asmdba) [oracle@node2 ~]$ ssh node2 date The authenticity of host 'node2 (192.168.10.20)' can't be established. RSA key fingerprint is 95:ed:6d:87:61:00:27:ed:38:17:6c:e9:6c:c3:8a:1d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'node2' (RSA) to the list of known hosts. 2012年 09月 27日 星期四 21:58:41 CST [oracle@node2 ~]$ ssh node2priv date The authenticity of host 'node2priv (192.168.20.20)' can't be established. RSA key fingerprint is 95:ed:6d:87:61:00:27:ed:38:17:6c:e9:6c:c3:8a:1d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'node2priv,192.168.20.20' (RSA) to the list of known hosts. 2012年 09月 27日 星期四 21:58:49 CST [oracle@node2 ~]$ ssh node1 date The authenticity of host 'node1 (192.168.10.10)' can't be established. RSA key fingerprint is 95:ed:6d:87:61:00:27:ed:38:17:6c:e9:6c:c3:8a:1d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'node1,192.168.10.10' (RSA) to the list of known hosts. 2012年 09月 27日 星期四 21:59:01 CST [oracle@node2 ~]$ ssh node1priv date The authenticity of host 'node1priv (192.168.20.10)' can't be established. RSA key fingerprint is 95:ed:6d:87:61:00:27:ed:38:17:6c:e9:6c:c3:8a:1d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'node1priv,192.168.20.10' (RSA) to the list of known hosts. 2012年 09月 27日 星期四 21:59:12 CST [oracle@node2 ~]$ [root@node1 /]# su - grid [grid@node1 ~]$ ssh node1 date The authenticity of host 'node1 (192.168.10.10)' can't be established. RSA key fingerprint is 95:ed:6d:87:61:00:27:ed:38:17:6c:e9:6c:c3:8a:1d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'node1,192.168.10.10' (RSA) to the list of known hosts. 2012年 09月 27日 星期四 21:59:53 CST [grid@node1 ~]$ ssh node1priv date The authenticity of host 'node1priv (192.168.20.10)' can't be established. RSA key fingerprint is 95:ed:6d:87:61:00:27:ed:38:17:6c:e9:6c:c3:8a:1d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'node1priv,192.168.20.10' (RSA) to the list of known hosts. 2012年 09月 27日 星期四 21:59:59 CST [grid@node1 ~]$ ssh node2 date The authenticity of host 'node2 (192.168.10.20)' can't be established. RSA key fingerprint is 95:ed:6d:87:61:00:27:ed:38:17:6c:e9:6c:c3:8a:1d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'node2' (RSA) to the list of known hosts. 2012年 09月 27日 星期四 22:00:09 CST [grid@node1 ~]$ ssh node2priv date The authenticity of host 'node2priv (192.168.20.20)' can't be established. RSA key fingerprint is 95:ed:6d:87:61:00:27:ed:38:17:6c:e9:6c:c3:8a:1d. Are you sure you want to continue connecting (yes/no)? Host key verification failed. [grid@node1 ~]$ ssh node2priv date The authenticity of host 'node2priv (192.168.20.20)' can't be established. RSA key fingerprint is 95:ed:6d:87:61:00:27:ed:38:17:6c:e9:6c:c3:8a:1d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'node2priv,192.168.20.20' (RSA) to the list of known hosts. 2012年 09月 27日 星期四 22:00:16 CST [grid@node1 ~]$ ssh node2 [grid@node2 ~]$ ssh node2 date The authenticity of host 'node2 (192.168.10.20)' can't be established. RSA key fingerprint is 95:ed:6d:87:61:00:27:ed:38:17:6c:e9:6c:c3:8a:1d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'node2' (RSA) to the list of known hosts. 2012年 09月 27日 星期四 22:00:29 CST [grid@node2 ~]$ ssh node2priv date The authenticity of host 'node2priv (192.168.20.20)' can't be established. RSA key fingerprint is 95:ed:6d:87:61:00:27:ed:38:17:6c:e9:6c:c3:8a:1d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'node2priv,192.168.20.20' (RSA) to the list of known hosts. 2012年 09月 27日 星期四 22:00:38 CST [grid@node2 ~]$ ssh node1 date The authenticity of host 'node1 (192.168.10.10)' can't be established. RSA key fingerprint is 95:ed:6d:87:61:00:27:ed:38:17:6c:e9:6c:c3:8a:1d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'node1,192.168.10.10' (RSA) to the list of known hosts. 2012年 09月 27日 星期四 22:00:49 CST [grid@node2 ~]$ ssh node1priv date The authenticity of host 'node1priv (192.168.20.10)' can't be established. RSA key fingerprint is 95:ed:6d:87:61:00:27:ed:38:17:6c:e9:6c:c3:8a:1d. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'node1priv,192.168.20.10' (RSA) to the list of known hosts. 2012年 09月 27日 星期四 22:00:58 CST [grid@node2 ~]$ 最后 在每个节点的 oracle用户 grid用户都执行一个命令 $ ssh node1 date && ssh node1priv date && ssh node2 date && ssh node2priv date $ id uid=501(grid) gid=501(oinstall) groups=501(oinstall),502(dba),504(asmadmin),506(asmdba),507(asmoper) 最后是这个效果: [grid@node2 ~]$ ssh node1 date && ssh node1priv date && ssh node2 date && ssh node2priv date 2012年 09月 27日 星期四 22:04:03 CST 2012年 09月 27日 星期四 22:04:03 CST 2012年 09月 27日 星期四 22:04:00 CST 2012年 09月 27日 星期四 22:04:01 CST [grid@node2 ~]$ su - oracle 口令: [oracle@node2 ~]$ ssh node1 date && ssh node1priv date && ssh node2 date && ssh node2priv date 2012年 09月 27日 星期四 22:04:14 CST 2012年 09月 27日 星期四 22:04:14 CST 2012年 09月 27日 星期四 22:04:11 CST 2012年 09月 27日 星期四 22:04:12 CST [oracle@node2 ~]$ ssh node1 [oracle@node1 ~]$ ssh node1 date && ssh node1priv date && ssh node2 date && ssh node2priv date 2012年 09月 27日 星期四 22:04:22 CST 2012年 09月 27日 星期四 22:04:22 CST 2012年 09月 27日 星期四 22:04:19 CST 2012年 09月 27日 星期四 22:04:20 CST [oracle@node1 ~]$ su - grid 口令: [grid@node1 ~]$ ssh node1 date && ssh node1priv date && ssh node2 date && ssh node2priv date 2012年 09月 27日 星期四 22:04:45 CST 2012年 09月 27日 星期四 22:04:45 CST 2012年 09月 27日 星期四 22:04:42 CST 2012年 09月 27日 星期四 22:04:42 CST [grid@node1 ~]$ 8、配置远程图形界面管理 可以使用xmanager,也可以使用vnc。对于远程链接跨越互联网的建议用vnc,如果在局域网中我们使用xmanager即可。 Xmanager配置说明: 配置文件位置: vi /usr/share/gdm/defaults.conf找到下面内容修改配置选项: [xdmcp] DisplaysPerHost=10 Enable=true //257行 Port=177 //284行 [security] AllowRemoteRoot =true //214行 /etc/inittab文件最后一行添加 x:5:respawn:/usr/sbin/gdm 使用命令 # gdm-restart 重启gdm; 查看gdm监听端口开启成功 netstat -nltpu | grep 177 [root@node1 /]# netstat -nltpu | grep 177 udp 0 0 0.0.0.0:177 0.0.0.0:* 3184/gdm-binary [root@node1 /]# 安装前检测 将软件上传到服务器中的我们之前建好的/u01/software 中并解压 [root@node1 /]# chown –R grid.oinstall /u01/software/ #su -grid #cd /u01/software/grid #./runcluvfy.sh stage -pre crsinst -n node1,node2 -fixup -verbose > check.txt 最后查看check.txt文件,如果有错误,可以尝试使用root身份运行/tmp/CVU_11.2.0.1.0_grid/runfixup.sh。最后我们可能看到关于NTP的错误,我们忽略。 二、图形界面开始安装clusterware [root@node1 ~]# su – grid [grid@node1 ~]$ cd /u01/software/grid [grid@node1 grid]$ ls check1.txt doc response runcluvfy.sh sshsetup welcome.html check.txt install rpm runInstaller stage [grid@node1 grid]$ ./runInstaller 选择语言支持 配置集群名称,第二行的“SCAN名称”要和hosts文件里面的scan那项要对应 添加修改节点信息,一定要和hosts文件中的名字对应。在此步骤可以配置ssh互信。因为我们事先已经配置好,这里就不需要修改了。 网络配置 选择磁盘管理方式,我们选择ASM表决盘管理 创建ASM磁盘组 搜索路径 选择需要的盘 设置密码 选择“是” 不使用 使用默认用户即可 设置grid的base和home目录 设置日志目录 检查环境 这项可以忽略 摘要 开始安装 将两个脚本分别在两个节点上以root身份运行 每个脚本等node1执行完了,在去下一个节点执行 执行完成之后点击确定 等待完成,clusterware安装成功。 三、安装数据软件 #su - oracle $cd /u01/software/database $ cd database/ $ ls doc install response rpm runInstaller sshsetup stage welcome.html $ ./runInstaller 关闭电子邮件接收 选择是 仅安装数据软件 选择RAC数据安装 选择语言支持 企业版安装 选择安装目录 选择操作组 点击完成,直到安装结束。 建(dbca) 以oracle用户身份执行dbca 点击完成直到结束。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值