This quick article will test ASM Filter Driver i/o filtering out non-oracle operation. So we will try to overwrite ASM disk with some common linux commands.
Before starting, you need ...
... Grid Infrastructure 12c already installed with ASMFD (if not, check this)
... a root access
Warning ...
Be aware that some commands in this article may DESTROY your data. So make sure you know what you are doing, do it on your own responsibility.
About Oracle ASM Filter Driver
Oracle ASM Filter Driver (Oracle ASMFD) is a kernel module that resides in the I/O path of the Oracle ASM disks. Oracle ASM uses the filter driver to validate write I/O requests to Oracle ASM disks.
docs.oracle.com
Oracle ASMFD simplifies the configuration and management of disk devices by eliminating the need to rebind disk devices used with Oracle ASM each time the system is restarted.
Oracle ASM Filter Driver rejects any I/O requests that are invalid. This action eliminates accidental overwrites of Oracle ASM disks that would cause corruption in the disks and files within the disk group. For example, the Oracle ASM Filter Driver filters out all non-Oracle I/Os which could cause accidental overwrites.
Check if ASM Filtering is enabled
We need to make sure that ASMFD is loaded and filtering is enabled on the disk we gonna try to corrupt (/dev/sda).
# Check if ASMFD is loaded [grid]$ asmcmd afd_state ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host 'oralab01.uxora.com' # List ASM Disk with filtering enabled [grid]$ asmcmd afd_lsdsk --------------------------------------------------------- Label Filtering Path ========================================================= DISK01 ENABLED /dev/sda DISK02 ENABLED /dev/sdb DISK03 ENABLED /dev/sdc DISK04 ENABLED /dev/sdd
Disk manipulation
As root user, we are going to do disk manipulation on /dev/sda which is an asm disk with filtering enabled.
First, we going to read the first bytes of /dev/sda :
# Get header [root]$ od -c -N 128 /dev/sda 0000000 001 202 001 001 \0 \0 \0 \0 \0 \0 \0 200 247 @ 203 220 0000020 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0000040 O R C L D I S K D I S K 0 1 \0 \0 0000060 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0000100 \0 001 \f \0 \0 001 003 D I S K 0 1 \0 \0 0000120 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0000140 \0 \0 \0 \0 \0 \0 \0 \0 D A T A \0 \0 \0 \0 0000160 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0000200
fdisk
# Try partition with fdisk [root]$ fdisk /dev/sda Welcome to fdisk (util-linux 2.23.2). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Device does not contain a recognized partition table Building a new DOS disklabel with disk identifier 0xd2fcbd9d. Command (m for help): n Partition type: p primary (0 primary, 0 extended, 4 free) e extended Select (default p): Using default response p Partition number (1-4, default 1): First sector (2048-50331647, default 2048): Using default value 2048 Last sector, +sectors or +size{K,M,G} (2048-50331647, default 50331647): Using default value 50331647 Partition 1 of type Linux and of size 24 GiB is set Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Error closing file # Check /var/log/message for error messages [root]$ tail -f /var/log/message ... Aug 19 22:05:28 oralab01 kernel: Buffer I/O error on dev sda, logical block 0, lost async page write Aug 19 22:05:28 oralab01 kernel: F 4297671.556/170819200528 fdisk[13113] oracleafd:18:0894:Write IO to ASM managed device: [8] [0] ...
mkfs
# Try to create filesystem with mkfs [root]$ mkfs.xfs -f /dev/sda meta-data=/dev/sda isize=256 agcount=4, agsize=1572864 blks = sectsz=512 attr=2, projid32bit=1 = crc=0 finobt=0, sparse=0 data = bsize=4096 blocks=6291456, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 ftype=0 log =internal log bsize=4096 blocks=3072, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 mkfs.xfs: pwrite64 failed: Input/output error # Check /var/log/message for error messages [root]$ tail -f /var/log/message ... Aug 19 22:24:18 oralab01 kernel: F 4306001.982/170819222418 mkfs.xfs[11228] oracleafd:18:0894:Write IO to ASM managed device: [8] [0] ...
dd
# Try to erase header with dd [root]$ dd if=/dev/zero of=/dev/sda bs=4096 count=1000 1000+0 records in 1000+0 records out 4096000 bytes (4.1 MB) copied, 0.00493748 s, 830 MB/s # Check /var/log/message for error messages [root]$ tail -f /var/log/message ... Aug 19 21:56:46 oralab01 kernel: Buffer I/O error on dev sda, logical block 0, lost async page write Aug 19 21:56:46 oralab01 kernel: Buffer I/O error on dev sda, logical block 1, lost async page write Aug 19 21:56:46 oralab01 kernel: Buffer I/O error on dev sda, logical block 2, lost async page write Aug 19 21:56:46 oralab01 kernel: Buffer I/O error on dev sda, logical block 3, lost async page write Aug 19 21:56:46 oralab01 kernel: Buffer I/O error on dev sda, logical block 4, lost async page write Aug 19 21:56:46 oralab01 kernel: Buffer I/O error on dev sda, logical block 5, lost async page write Aug 19 21:56:46 oralab01 kernel: Buffer I/O error on dev sda, logical block 6, lost async page write Aug 19 21:56:46 oralab01 kernel: Buffer I/O error on dev sda, logical block 7, lost async page write Aug 19 21:56:46 oralab01 kernel: Buffer I/O error on dev sda, logical block 8, lost async page write Aug 19 21:56:46 oralab01 kernel: Buffer I/O error on dev sda, logical block 9, lost async page write ... # Try to erase header with dd direct io [root]$ dd if=/dev/zero of=/dev/sda bs=4096 count=1000 oflag=direct dd: error writing ‘/dev/sda’: Input/output error 1+0 records in 0+0 records out 0 bytes (0 B) copied, 0.000186557 s, 0.0 kB/s # Check /var/log/message for error messages [root]$ tail -f /var/log/message ... Aug 19 22:16:41 oralab01 kernel: F 4298344.799/170819201641 dd[20904] oracleafd:18:0894:Write IO to ASM managed device: [8] [0] ...
io redirection
# Try to erase header with io redirection [root]$ echo HELLOWORLD > /dev/sda # Check /var/log/message for error messages [root]$ tail -f /var/log/message ... Aug 19 21:55:17 oralab01 kernel: Buffer I/O error on dev sda, logical block 0, lost async page write ...
Check ASM Disk
After all theses disks manipulations, let's check /dev/sda disk.
Re-Read header with od
Header still is the same before and after disk manipulations.
# Get header [root]$ od -c -N 128 /dev/sda 0000000 001 202 001 001 \0 \0 \0 \0 \0 \0 \0 200 247 @ 203 220 0000020 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0000040 O R C L D I S K D I S K 0 1 \0 \0 0000060 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0000100 \0 001 \f \0 \0 001 003 D I S K 0 1 \0 \0 0000120 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0000140 \0 \0 \0 \0 \0 \0 \0 \0 D A T A \0 \0 \0 \0 0000160 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0000200
Read header with kfed
Kfed can still read the asm disk header.
Expand/Collapse
# Header with kfed [grid]$ kfed read /dev/sda kfbh.endian: 1 ; 0x000: 0x01 kfbh.hard: 130 ; 0x001: 0x82 kfbh.type: 1 ; 0x002: KFBTYP_DISKHEAD kfbh.datfmt: 1 ; 0x003: 0x01 kfbh.block.blk: 0 ; 0x004: blk=0 kfbh.block.obj: 2147483648 ; 0x008: disk=0 kfbh.check: 2424520871 ; 0x00c: 0x908340a7 kfbh.fcn.base: 0 ; 0x010: 0x00000000 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 kfdhdb.driver.provstr: ORCLDISKDISK01 ; 0x000: length=14 kfdhdb.driver.reserved[0]: 1263749444 ; 0x008: 0x4b534944 kfdhdb.driver.reserved[1]: 12592 ; 0x00c: 0x00003130 kfdhdb.driver.reserved[2]: 0 ; 0x010: 0x00000000 kfdhdb.driver.reserved[3]: 0 ; 0x014: 0x00000000 kfdhdb.driver.reserved[4]: 0 ; 0x018: 0x00000000 kfdhdb.driver.reserved[5]: 0 ; 0x01c: 0x00000000 kfdhdb.compat: 203424000 ; 0x020: 0x0c200100 kfdhdb.dsknum: 0 ; 0x024: 0x0000 kfdhdb.grptyp: 1 ; 0x026: KFDGTP_EXTERNAL kfdhdb.hdrsts: 3 ; 0x027: KFDHDR_MEMBER kfdhdb.dskname: DISK01 ; 0x028: length=6 kfdhdb.grpname: DATA ; 0x048: length=4 kfdhdb.fgname: DISK01 ; 0x068: length=6 kfdhdb.siteguid[0]: 0 ; 0x088: 0x00 kfdhdb.siteguid[1]: 0 ; 0x089: 0x00 kfdhdb.siteguid[2]: 0 ; 0x08a: 0x00 kfdhdb.siteguid[3]: 0 ; 0x08b: 0x00 kfdhdb.siteguid[4]: 0 ; 0x08c: 0x00 kfdhdb.siteguid[5]: 0 ; 0x08d: 0x00 kfdhdb.siteguid[6]: 0 ; 0x08e: 0x00 kfdhdb.siteguid[7]: 0 ; 0x08f: 0x00 kfdhdb.siteguid[8]: 0 ; 0x090: 0x00 kfdhdb.siteguid[9]: 0 ; 0x091: 0x00 kfdhdb.siteguid[10]: 0 ; 0x092: 0x00 kfdhdb.siteguid[11]: 0 ; 0x093: 0x00 kfdhdb.siteguid[12]: 0 ; 0x094: 0x00 kfdhdb.siteguid[13]: 0 ; 0x095: 0x00 kfdhdb.siteguid[14]: 0 ; 0x096: 0x00 kfdhdb.siteguid[15]: 0 ; 0x097: 0x00 kfdhdb.ub1spare[0]: 0 ; 0x098: 0x00 kfdhdb.ub1spare[1]: 0 ; 0x099: 0x00 kfdhdb.ub1spare[2]: 0 ; 0x09a: 0x00 kfdhdb.ub1spare[3]: 0 ; 0x09b: 0x00 kfdhdb.ub1spare[4]: 0 ; 0x09c: 0x00 kfdhdb.ub1spare[5]: 0 ; 0x09d: 0x00 kfdhdb.ub1spare[6]: 0 ; 0x09e: 0x00 kfdhdb.ub1spare[7]: 0 ; 0x09f: 0x00 kfdhdb.ub1spare[8]: 0 ; 0x0a0: 0x00 kfdhdb.ub1spare[9]: 0 ; 0x0a1: 0x00 kfdhdb.ub1spare[10]: 0 ; 0x0a2: 0x00 kfdhdb.ub1spare[11]: 0 ; 0x0a3: 0x00 kfdhdb.ub1spare[12]: 0 ; 0x0a4: 0x00 kfdhdb.ub1spare[13]: 0 ; 0x0a5: 0x00 kfdhdb.ub1spare[14]: 0 ; 0x0a6: 0x00 kfdhdb.ub1spare[15]: 0 ; 0x0a7: 0x00 kfdhdb.crestmp.hi: 33055335 ; 0x0a8: HOUR=0x7 DAYS=0x13 MNTH=0x8 YEAR=0x7e1 kfdhdb.crestmp.lo: 888763392 ; 0x0ac: USEC=0x0 MSEC=0x25d SECS=0xf MINS=0xd kfdhdb.mntstmp.hi: 33055349 ; 0x0b0: HOUR=0x15 DAYS=0x13 MNTH=0x8 YEAR=0x7e1 kfdhdb.mntstmp.lo: 1119166464 ; 0x0b4: USEC=0x0 MSEC=0x148 SECS=0x2b MINS=0x10 kfdhdb.secsize: 512 ; 0x0b8: 0x0200 kfdhdb.blksize: 4096 ; 0x0ba: 0x1000 kfdhdb.ausize: 4194304 ; 0x0bc: 0x00400000 kfdhdb.mfact: 454272 ; 0x0c0: 0x0006ee80 kfdhdb.dsksize: 6144 ; 0x0c4: 0x00001800 kfdhdb.pmcnt: 3 ; 0x0c8: 0x00000003 kfdhdb.fstlocn: 1 ; 0x0cc: 0x00000001 kfdhdb.altlocn: 2 ; 0x0d0: 0x00000002 kfdhdb.f1b1locn: 10 ; 0x0d4: 0x0000000a kfdhdb.redomirrors[0]: 0 ; 0x0d8: 0x0000 kfdhdb.redomirrors[1]: 0 ; 0x0da: 0x0000 kfdhdb.redomirrors[2]: 0 ; 0x0dc: 0x0000 kfdhdb.redomirrors[3]: 0 ; 0x0de: 0x0000 kfdhdb.dbcompat: 168820736 ; 0x0e0: 0x0a100000 kfdhdb.grpstmp.hi: 33055335 ; 0x0e4: HOUR=0x7 DAYS=0x13 MNTH=0x8 YEAR=0x7e1 kfdhdb.grpstmp.lo: 888013824 ; 0x0e8: USEC=0x0 MSEC=0x381 SECS=0xe MINS=0xd kfdhdb.vfstart: 24 ; 0x0ec: 0x00000018 kfdhdb.vfend: 32 ; 0x0f0: 0x00000020 kfdhdb.spfile: 0 ; 0x0f4: 0x00000000 kfdhdb.spfflg: 0 ; 0x0f8: 0x00000000 kfdhdb.flags: 1 ; 0x0fc: 0x00000001 kfdhdb.f1b1fcn.base: 0 ; 0x100: 0x00000000 kfdhdb.f1b1fcn.wrap: 0 ; 0x104: 0x00000000 kfdhdb.ip[0]: 192 ; 0x108: 0xc0 kfdhdb.ip[1]: 168 ; 0x109: 0xa8 kfdhdb.ip[2]: 0 ; 0x10a: 0x00 kfdhdb.ip[3]: 31 ; 0x10b: 0x1f kfdhdb.modstmp: 1503170203 ; 0x10c: 0x59988e9b kfdhdb.checklbl: 0 ; 0x110: 0x00 kfdhdb.verlbl: 0 ; 0x111: 0x00 kfdhdb.ub2spare: 0 ; 0x112: 0x0000 kfdhdb.sitelbl: ; 0x114: length=0 kfdhdb.fglbl: ; 0x124: length=0 kfdhdb.vsnnum: 203424000 ; 0x144: 0x0c200100 kfdhdb.patchvsn: 0 ; 0x148: 0x0000 kfdhdb.operation: 0 ; 0x14a: 0x0000 kfdhdb.xtnd[0]: 0 ; 0x14c: 0x0000 kfdhdb.xtnd[1]: 0 ; 0x14e: 0x0000 kfdhdb.xtnd[2]: 0 ; 0x150: 0x0000 kfdhdb.xtnd[3]: 0 ; 0x152: 0x0000 kfdhdb.xtnd[4]: 0 ; 0x154: 0x0000 kfdhdb.xtnd[5]: 0 ; 0x156: 0x0000 kfdhdb.ub4spare[0]: 0 ; 0x158: 0x00000000 kfdhdb.ub4spare[1]: 0 ; 0x15c: 0x00000000 kfdhdb.ub4spare[2]: 0 ; 0x160: 0x00000000 kfdhdb.ub4spare[3]: 0 ; 0x164: 0x00000000 kfdhdb.ub4spare[4]: 0 ; 0x168: 0x00000000 kfdhdb.ub4spare[5]: 0 ; 0x16c: 0x00000000 kfdhdb.ub4spare[6]: 0 ; 0x170: 0x00000000 kfdhdb.ub4spare[7]: 0 ; 0x174: 0x00000000 kfdhdb.ub4spare[8]: 0 ; 0x178: 0x00000000 kfdhdb.ub4spare[9]: 0 ; 0x17c: 0x00000000 kfdhdb.ub4spare[10]: 0 ; 0x180: 0x00000000 kfdhdb.ub4spare[11]: 0 ; 0x184: 0x00000000 kfdhdb.ub4spare[12]: 0 ; 0x188: 0x00000000 kfdhdb.ub4spare[13]: 0 ; 0x18c: 0x00000000 kfdhdb.ub4spare[14]: 0 ; 0x190: 0x00000000 kfdhdb.ub4spare[15]: 0 ; 0x194: 0x00000000 kfdhdb.ub4spare[16]: 0 ; 0x198: 0x00000000 kfdhdb.ub4spare[17]: 0 ; 0x19c: 0x00000000 kfdhdb.ub4spare[18]: 0 ; 0x1a0: 0x00000000 kfdhdb.ub4spare[19]: 0 ; 0x1a4: 0x00000000 kfdhdb.ub4spare[20]: 0 ; 0x1a8: 0x00000000 kfdhdb.ub4spare[21]: 0 ; 0x1ac: 0x00000000 kfdhdb.ub4spare[22]: 0 ; 0x1b0: 0x00000000 kfdhdb.ub4spare[23]: 0 ; 0x1b4: 0x00000000 kfdhdb.ub4spare[24]: 0 ; 0x1b8: 0x00000000 kfdhdb.ub4spare[25]: 0 ; 0x1bc: 0x00000000 kfdhdb.ub4spare[26]: 0 ; 0x1c0: 0x00000000 kfdhdb.ub4spare[27]: 0 ; 0x1c4: 0x00000000 kfdhdb.ub4spare[28]: 0 ; 0x1c8: 0x00000000 kfdhdb.ub4spare[29]: 0 ; 0x1cc: 0x00000000 kfdhdb.ub4spare[30]: 0 ; 0x1d0: 0x00000000 kfdhdb.acdb.aba.seq: 0 ; 0x1d4: 0x00000000 kfdhdb.acdb.aba.blk: 0 ; 0x1d8: 0x00000000 kfdhdb.acdb.ents: 0 ; 0x1dc: 0x0000 kfdhdb.acdb.ub2spare: 0 ; 0x1de: 0x0000
Check disk with ASM
# Check datagroup with asmcmd [grid]$ asmcmd chkdg data Diskgroup altered. [grid]$ tail -20 /u01/app/grid/diag/asm/+asm/+ASM1/trace/alert_+ASM1.log ... SQL> /* ASMCMD */ALTER DISKGROUP data CHECK NOREPAIR NOTE: starting check of diskgroup DATA GMON querying group 1 at 24 for pid 38, osid 9849 GMON checking disk 0 for group 1 at 25 for pid 38, osid 9849 GMON querying group 1 at 26 for pid 38, osid 9849 GMON checking disk 1 for group 1 at 27 for pid 38, osid 9849 GMON querying group 1 at 28 for pid 38, osid 9849 GMON checking disk 2 for group 1 at 29 for pid 38, osid 9849 GMON querying group 1 at 30 for pid 38, osid 9849 GMON checking disk 3 for group 1 at 31 for pid 38, osid 9849 SUCCESS: check of diskgroup DATA found no errors SUCCESS: /* ASMCMD */ALTER DISKGROUP data CHECK NOREPAIR ...
Conclusion
It does what it says and It works pretty well so far!
This will protect from some wrong disk administration mistakes and avoid the dba a full db backup recovery.
So I think this is a great feature which may save a dba some stressful time.
Hope This Helps,
Michel.
Reference
Administering Oracle ASM Filter Driver (docs.oracle.com)
Oracle’s ASM Filter Driver Revisited (flashdba.com)
Enjoyed this article? Please like it or share it.
Please connect with one of social login below (or fill up name and email)