HowTo:OracleASMonLinux

From Greg Porter's Wiki

Jump to: navigation, search

Contents

Overview

Starting in Oracle database version 10g, you can use Automatic Storage Management (ASM) to store most database related objects in, like tablespaces, datafiles, etc. If you want to make a clustered database using Real Application Clusters (RAC), then you have to use ASM. Getting started with ASM is not too hard.

Select a database version and Linux distribution

Go the Oracle page, pick a database version. I picked 10g, because I'm studying for my 10g upgrade test. For actual use, it's probably best to pick the latest shipping version. Other basic selections are Standard or Enterprise version, and 32 bit or 64 bit. I'm using 64 bit Enterprise for grins. Download the selected version. Burn it to disk or save it it on disk for later.

Once you've decided on a database version, then go find the installation guide for that version. Go to the Oracle documentation site. Find the installation guide for your version. In my case, I'm loading 10g Release 2 on Linux for x86_64, so I went to Oracle® Database Installation Guide 10g Release 2 (10.2) for Linux x86_64. Go to the Software Requirements. Pick one of the listed operating system. For me, loading 10gR2 on Linux x86, I can choose from:

  • Red Hat Enterprise Linux AS/ES 3.0 (Update 4 or later)
  • Red Hat Linux 4.0
  • SUSE Linux Enterprise Server 9.0 with SP 2 or later
  • Asianux 1.0
  • Asianux 2.0

Also look at the release notes, they have last minute additions to the list, in my case I can also choose from:

  • Asianux 3
  • Oracle Enterprise Linux 4/Oracle VM
  • Oracle Enterprise Linux 5/Oracle VM
  • Red Hat Enterprise Linux 5/Oracle VM
  • SUSE Linux Enterprise Server 10
  • SUSE Linux Enterprise Server 11

I'm going to use 64 bit Oracle 10gR2, and put it on something Red Hat-ish. I picked 64 bit CentOS 5 (which is basically RHEL 5).

Install pre-requisites

The most straightforward document that succinctly states the pre-requisites and the install procedures is *NOT* the install guide or release notes. For most platform/database versions, Oracle maintains a fresh Metalink support document that is much better (and much shorter). In order to see it, you have to have an Oracle support account. Of course, as a good Oracle customer, who has properly purchased the software you are using, you have a Customer Support Identifier, and can log into Metalink.

In my case, I'm using 10gR2 on x86_64 RHEL 5. I logged into Metalink, and searched for "install 10g X86_64 RHEL 5". The first hit is the correct document, Requirements For Installing Oracle10gR2 On RHEL 5/OEL 5 (x86_64) [ID 421308.1]. In my case, I followed this step by step, and it was complete and correct. Follow the document, and it'll work.

Install ASM software (ASMLib)

The non-obvious thing that isn't mentioned at all in the install guide are the extra packages required for ASM. The release notes hint at it, they say "Install oracleasm-support package version 2.0.0.1 or higher to use ASMLib on all Linux operating systems." Neither say what those are, nor where to get them.

Googling around a bit reveals that Oracle built something called ASMLib for handling ASM disks. They have a different version of ASMLib for each supported Linux platform, that's why you have to be sure to select a supported Linux version. In my case, I'm using RHEL 5 x86_64, so I went to Oracle ASMLib Downloads for Red Hat Enterprise Linux Server 5 and installed:

  • oracleasm-support-2.1.3-1.el5.x86_64.rpm
  • the driver package that exactly matches the kernel I have
  • oracleasmlib-2.0.4-1.el5.x86_64.rpm


Verify your kernel like so: 

[root@server5 tmp]# uname -r
2.6.18-194.26.1.el5xen

So I installed oracleasm-2.6.18-194.26.1.el5xen-2.0.5-1.el5.x86_64.rpm. 

[root@server5 tmp]# pwd
/tmp
[root@server5 tmp]# wget http://oss.oracle.com/projects/oracleasm-support/dist/files/RPMS/rhel5/amd64/2.1.3
/oracleasm-support-2.1.3-1.el5.x86_64.rpm
--2010-11-26 13:35:20--  http://oss.oracle.com/projects/oracleasm-support/dist/files/RPMS/rhel5/amd64/2.1.3
/oracleasm-support-2.1.3-1.el5.x86_64.rpm
Resolving oss.oracle.com... 141.146.12.120
Connecting to oss.oracle.com|141.146.12.120|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 90444 (88K) [application/x-rpm]
Saving to: `oracleasm-support-2.1.3-1.el5.x86_64.rpm'

100%[======================================>] 90,444      27.7K/s   in 3.2s    

2010-11-26 13:35:24 (27.7 KB/s) - `oracleasm-support-2.1.3-1.el5.x86_64.rpm' saved [90444/90444]

[root@server5 tmp]# rpm -ivh oracleasm-support-2.1.3-1.el5.x86_64.rpm 
warning: oracleasm-support-2.1.3-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing...                ########################################### [100%]
   1:oracleasm-support      ########################################### [100%]
[root@server5 tmp]# 
[root@server5 tmp]# wget http://oss.oracle.com/projects/oracleasm/dist/files/RPMS/rhel5/amd64/2.0.5
/2.6.18-194.26.1.el5/oracleasm-2.6.18-194.26.1.el5xen-2.0.5-1.el5.x86_64.rpm
--2010-11-26 13:35:53--  http://oss.oracle.com/projects/oracleasm/dist/files/RPMS/rhel5/amd64/2.0.5
/2.6.18-194.26.1.el5/oracleasm-2.6.18-194.26.1.el5xen-2.0.5-1.el5.x86_64.rpm
Resolving oss.oracle.com... 141.146.12.120
Connecting to oss.oracle.com|141.146.12.120|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 136398 (133K) [application/x-rpm]
Saving to: `oracleasm-2.6.18-194.26.1.el5xen-2.0.5-1.el5.x86_64.rpm'

100%[======================================>] 136,398     18.9K/s   in 6.7s    

2010-11-26 13:36:03 (19.8 KB/s) - `oracleasm-2.6.18-194.26.1.el5xen-2.0.5-1.el5.x86_64.rpm' saved [136398/136398]

[root@server5 tmp]# rpm -ivh oracleasm-2.6.18-194.26.1.el5xen-2.0.5-1.el5.x86_64.rpm 
warning: oracleasm-2.6.18-194.26.1.el5xen-2.0.5-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing...                ########################################### [100%]
   1:oracleasm-2.6.18-194.26########################################### [100%]

[root@server5 tmp]# wget http://download.oracle.com/otn_software/asmlib/oracleasmlib-2.0.4-1.el5.x86_64.rpm
--2010-11-26 13:36:46--  http://download.oracle.com/otn_software/asmlib/oracleasmlib-2.0.4-1.el5.x86_64.rpm
Resolving download.oracle.com... 68.142.79.70, 68.142.79.69
Connecting to download.oracle.com|68.142.79.70|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 14176 (14K) [text/plain]
Saving to: `oracleasmlib-2.0.4-1.el5.x86_64.rpm'

100%[======================================>] 14,176      11.3K/s   in 1.2s    

2010-11-26 13:36:48 (11.3 KB/s) - `oracleasmlib-2.0.4-1.el5.x86_64.rpm' saved [14176/14176]

[root@server5 tmp]# rpm -ivh oracleasmlib-2.0.4-1.el5.x86_64.rpm 
warning: oracleasmlib-2.0.4-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing...                ########################################### [100%]
   1:oracleasmlib           ########################################### [100%]

Now configure oracleasm for use. 

[root@server5 tmp]# /etc/init.d/oracleasm configure
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting <ENTER> without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface [oracle]: 
Default group to own the driver interface [oinstall]: 
Start Oracle ASM library driver on boot (y/n) [y]: 
Scan for Oracle ASM disks on boot (y/n) [y]: 
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver:                     [  OK  ]

Prepare disks

If you're want to use ASM, you'll need one or more disks to run ASM on. Each one of these needs to have at least one partition, and be "stamped" or marked for ASM use. In my case I have a total of six disks. /dev/xvda is the disk Linux is installed on and there's five more 4 GB disks available, /dev/xvdb, /dev/xvdc, /dev/xvde, /dev/xvdf, and /dev/xvdg. 

[root@server5 tmp]# fdisk -l

Disk /dev/xvda: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

    Device Boot      Start         End      Blocks   Id  System
/dev/xvda1   *           1          13      104391   83  Linux
/dev/xvda2              14        2610    20860402+  8e  Linux LVM

Disk /dev/xvdb: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/xvdb doesn't contain a valid partition table

Disk /dev/xvdc: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/xvdc doesn't contain a valid partition table

Disk /dev/xvde: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/xvde doesn't contain a valid partition table

Disk /dev/xvdf: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/xvdf doesn't contain a valid partition table

Disk /dev/xvdg: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/xvdg doesn't contain a valid partition table

Partition the disks

Partition them with the usual partition tools. You need at least one Linux type 83 partition on each disk. Do this on each disk you want to use for ASM. 

[root@server5 tmp]# fdisk /dev/xvdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-522, default 1): 
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-522, default 522): 
Using default value 522

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

Stamp the disks for ASM use

Once you have some disks to use, then prepare them for ASM use. 

[root@server5 tmp]# /etc/init.d/oracleasm createdisk vol1 /dev/xvdb1
Marking disk "vol1" as an ASM disk:                        [  OK  ]
[root@server5 tmp]# /etc/init.d/oracleasm createdisk vol2 /dev/xvdc1
Marking disk "vol2" as an ASM disk:                        [  OK  ]
[root@server5 tmp]# /etc/init.d/oracleasm createdisk vol3 /dev/xvde1
Marking disk "vol3" as an ASM disk:                        [  OK  ]
[root@server5 tmp]# /etc/init.d/oracleasm createdisk vol4 /dev/xvdf1
Marking disk "vol4" as an ASM disk:                        [  OK  ]
[root@server5 tmp]# /etc/init.d/oracleasm createdisk vol5 /dev/xvdg1
Marking disk "vol5" as an ASM disk:                        [  OK  ]
[root@server5 tmp]# /etc/init.d/oracleasm listdisks
VOL1
VOL2
VOL3
VOL4
VOL5

Install an ASM instance with Oracle Universal Installer (OUI)

Start OUI like the install docs say, but select configure ASM, not create database. Install ASM first. Install ASM first.


You'll see your disks here if you prepared them correctly. You'll see your disks here if you prepared them correctly.


Let OUI complete. OUI'll complete. ASM should be ready to use.

OUI will prompt you to run 2 scripts. Do so. The second one will have a few extra prompts related to ASM that you probably haven't seen before in a regular non-ASM install. 

[root@server5 tmp]# /oracle/oraInventory/orainstRoot.sh
Changing permissions of /oracle/oraInventory to 770.
Changing groupname of /oracle/oraInventory to oinstall.
The execution of the script is complete
[root@server5 tmp]# /oracle/product/10.2.0/db_1/root.sh
Running Oracle10 root.sh script...

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /oracle/product/10.2.0/db_1

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
/etc/oracle does not exist. Creating it now.
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Configuration for local CSS has been initialized

Adding to inittab 
Startup will be queued to init within 90 seconds.
Checking the status of new Oracle init process...
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
        server5
CSS is active on all nodes.
Oracle CSS service is installed and running under init(1M)

Start and use ASM

As the user named oracle, do the usual environmental tweaking to get the user oracle to be able to use oracle executables. The ASM instance is automatically named +ASM, so set ORACLE_SID=+ASM. OUI will leave the ASM instance running, so you can log in and look at it. 

[oracle@server5 10g]$ export ORACLE_BASE=/oracle
[oracle@server5 10g]$ export ORACLE_HOME=/oracle/product/10.2.0/db_1
[oracle@server5 10g]$ export ORACLE_SID=+ASM
[oracle@server5 10g]$ export PATH=$PATH:/oracle/product/10.2.0/db_1/bin
[oracle@server5 10g]$ sqlplus / as sysdba;

SQL*Plus: Release 10.2.0.1.0 - Production on Fri Nov 26 15:20:05 2010

Copyright (c) 1982, 2005, Oracle.  All rights reserved.


Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bit Production
With the Partitioning, OLAP and Data Mining options

SQL> select * from v$instance;

INSTANCE_NUMBER INSTANCE_NAME
--------------- ----------------
HOST_NAME
----------------------------------------------------------------
VERSION           STARTUP_T STATUS       PAR    THREAD# ARCHIVE LOG_SWITCH_WAIT
----------------- --------- ------------ --- ---------- ------- ---------------
LOGINS     SHU DATABASE_STATUS   INSTANCE_ROLE      ACTIVE_ST BLO
---------- --- ----------------- ------------------ --------- ---
              1 +ASM
server5
10.2.0.1.0        26-NOV-10 STARTED      NO           0 STOPPED
ALLOWED    NO  ACTIVE            UNKNOWN            NORMAL    NO
SQL> exit
Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bit Production
With the Partitioning, OLAP and Data Mining options

[oracle@server5 10g]$ ps -ef |grep +ASM
oracle   22854     1  0 15:16 ?        00:00:00 asm_pmon_+ASM
oracle   22856     1  0 15:16 ?        00:00:00 asm_psp0_+ASM
oracle   22858     1  0 15:16 ?        00:00:00 asm_mman_+ASM
oracle   22860     1  0 15:16 ?        00:00:00 asm_dbw0_+ASM
oracle   22862     1  0 15:16 ?        00:00:00 asm_lgwr_+ASM
oracle   22864     1  0 15:16 ?        00:00:00 asm_ckpt_+ASM
oracle   22866     1  0 15:16 ?        00:00:00 asm_smon_+ASM
oracle   22868     1  0 15:16 ?        00:00:00 asm_rbal_+ASM
oracle   22870     1  0 15:16 ?        00:00:00 asm_gmon_+ASM
oracle   23252  5443  0 15:23 pts/1    00:00:00 grep +ASM

So there's an honest-to-gosh database running, even though you didn't install a database. ASM lives inside a minimal database instance, so you'll see the usual Oracle background processes like Process Monitor (PMON) running to support the ASM instance.

Stopping and starting ASM

You use the usual database stop and start procedures to stop and start ASM (it is a database instance after all). 

[oracle@server5 10g]$ sqlplus / as sysdba;

SQL*Plus: Release 10.2.0.1.0 - Production on Fri Nov 26 15:31:58 2010

Copyright (c) 1982, 2005, Oracle.  All rights reserved.


Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bit Production
With the Partitioning, OLAP and Data Mining options

SQL> shutdown;
ASM diskgroups dismounted
ASM instance shutdown
SQL> startup;
ASM instance started

Total System Global Area  130023424 bytes
Fixed Size                  2019032 bytes
Variable Size             102838568 bytes
ASM Cache                  25165824 bytes
ASM diskgroups mounted
SQL> !ps -ef | grep +ASM
oracle   23589     1  0 15:32 ?        00:00:00 asm_pmon_+ASM
oracle   23591     1  0 15:32 ?        00:00:00 asm_psp0_+ASM
oracle   23593     1  0 15:32 ?        00:00:00 asm_mman_+ASM
oracle   23595     1  0 15:32 ?        00:00:00 asm_dbw0_+ASM
oracle   23597     1  0 15:32 ?        00:00:00 asm_lgwr_+ASM
oracle   23599     1  0 15:32 ?        00:00:00 asm_ckpt_+ASM
oracle   23601     1  0 15:32 ?        00:00:00 asm_smon_+ASM
oracle   23603     1  0 15:32 ?        00:00:00 asm_rbal_+ASM
oracle   23605     1  0 15:32 ?        00:00:00 asm_gmon_+ASM
oracle   23607 23583  0 15:32 ?        00:00:00 oracle+ASM (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
oracle   23653 23583  0 15:32 pts/1    00:00:00 /bin/bash -c ps -ef | grep +ASM

Using ASMCMD

If you are using ASM for storage, then you'll be storing files inside of the ASM instance. Since your "stuff" is inside of the ASM instance, you can't see it with normal file system tools like ls. The files in ASM live in ASM, not a file system.

For example, in my DATA disk group in ASM, I'll add a directory named test_dir. 

SQL> alter diskgroup data add directory '+data/test_dir';

Diskgroup altered.

SQL> select * from v$asm_alias;

NAME                                             GROUP_NUMBER FILE_NUMBER
------------------------------------------------ ------------ -----------
FILE_INCARNATION ALIAS_INDEX ALIAS_INCARNATION PARENT_INDEX REFERENCE_INDEX A S
---------------- ----------- ----------------- ------------ --------------- - -
test_dir                                                    1  4294967295
      4294967295           0                 1     16777216        16777269 Y N

Does this file exist in a file system somewhere? Nope. It's in ASM. 

[root@server5 db_1]# find / -name test_dir
[root@server5 db_1]#

Oracle supplied a utility name asmcmd that lets you wander around a bit inside of ASM, and see your "stuff". Here's a few simple examples. 

[oracle@server5 10g]$ asmcmd
ASMCMD> ?
        commands: 
        --------  
        cd
        du
        find
        help
        ls
        lsct
        lsdg
        mkalias
        mkdir
        pwd
        rm
        rmalias
ASMCMD> pwd
+
ASMCMD> ls
DATA/
ASMCMD> cd data
ASMCMD> pwd
+data
ASMCMD> ls
test_dir/
ASMCMD> cd test_dir
ASMCMD> pwd
+data/test_dir

Install database using ASM storage

Now start the Database Configuration Assistant (DBCA), and tell DBCA to use the existing ASM instance to store the new database in.

Select ASM to store the database in. Log into the ASM instance with the password you entered when you made it.


Pick the disk group you made.


Stick with Oracle Managed Files (OMF). OMF will make and name all the required files "the Oracle way".


Use ASM for flash storage as well.


Use Oracle Enterprise Manager (OEM) to manage ASM

Once the database is installed, you'll have access to Oracle Enterprise Manager. Log into OEM.


This looks like "regular" OEM, except there's new links for ASM. If you click on these, you can log in and manage the ASM instance from OEM.


Click on Administration.


Login as sys.


Manage ASM from inside OEM.


Most common tasks can be done from OEM.

Welcome to ASM, now you *HAVE* to use RMAN

If you store your stuff in ASM, you can't get to it with regular file system tools. The typical old school way of backing up a database is to turn the database off and spin the whole mess to tape. You can't do that anymore. A typical recovery is to put back any missing pieces from tape, and then roll the whole thing forward with logs to the point of failure. You can't do that either.

I'm not a big fan of Recovery Manager (RMAN). It seemed too complicated, and I could do a fine job myself. No more.

So what's RMAN good for? Putting stuff into ASM and getting stuff out of ASM.

You can put database pieces in a file system location from tape and tell RMAN to use the file system location as a source place. In effect, putting things into ASM.

You can define a file system location in RMAN, and tell RMAN to back up the database and whatever else to the file system location - in effect, getting stuff out of ASM.

So if you use ASM, you'll have to start using RMAN as well.

To lessen the pain,you could get backup software that's RMAN aware, like Data Protector with Oracle integration.

Retrieved from "http://greg.porter.name/wiki/HowTo:OracleASMonLinux"
Personal tools