asm

I’ve always thought that ASM is a sucking unneeded part of oracle. Is really slow, is another abstraction layer between database a disk block, is not mature for production environment and simply sucks. Yesterday I made an error. Altering a diskgroup to add a new lun to ASM in a RAC environment cause an error because LUN was not visible to ASM on both cluster node

ALTER DISKGROUP ORAFRA ADD DISK '/dev/rhdisk14';
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15075: disk(s) are not visible cluster-wide

Oh my god, LUN rescan was done on both node, what’s the problem? I’ve found a permission problem, On second node LUN was not writable to oinstall group… ok. Changed. And now? Now is interesting.
First, ORAFRA diskgroup was unmounted on second node and as the diskgroup contains on control file second instance was closed in abort!
No panic. I’ve mounted diskgroup and restarted second instance. Curious, what about /dev/rhdisk14 ?
Check in ASM:

SQL> col name format a15
SQL> col path format a20
SQL> set lines 136 pages 400
SQL> select GROUP_NUMBER,DISK_NUMBER,MOUNT_STATUS,HEADER_STATUS,NAME,PATH from v$asm_disk where PATH like '%rhdisk14';

GROUP_NUMBER DISK_NUMBER MOUNT_S HEADER_STATU NAME            PATH
------------ ----------- ------- ------------ --------------- ------------------------------
           0           5 CLOSED  MEMBER                       /dev/rhdisk14

My LUN is MEMBER of nothing! Perhaps an ASM error. As this LUN was added to diskgroup ORAFRA I’ll try to remove it from this diskgroup.

SQL> alter diskgroup ORAFRA drop disk '/dev/rhdisk14' ;
alter diskgroup ORAFRA drop disk '/dev/rhdisk14'
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15054: disk "/DEV/RHDISK14" does not exist in diskgroup "ORAFRA"

Damn, what I’ve to do? Metalink note: 399500.1
This problem can occurs when disk is not visible cluster-wide. For particular type of storage (of course storage I’m using) you have to change a value in LUN configuration: reserve_policy = no_reserve
Ok, this needs a reboot, bad. And to solve my problem? Metalink suggest to check and make LUN really visible and useable to both nodes. Then drop and add again the disk. Is this possible? I think not!