Comments on: VLDB with ASM? http://www.oracloid.com/2006/05/vldb-with-asm/ The Alex Gorbachev Oracle Blog Sat, 22 Nov 2008 00:15:47 +0000 http://wordpress.org/?v=2.5.1 By: Alex Gorbachev http://www.oracloid.com/2006/05/vldb-with-asm/#comment-15432 Alex Gorbachev Thu, 31 Jan 2008 04:21:03 +0000 http://blog.oracloid.com/2006/05/vldb-with-asm/#comment-15432 Lemu, Sorry I noticed your questions very late. You have probably already resolved it but here is quick explanation for others if they get here via search of other references. the status of HUNG for ASM disk is "awarded" when you requests to drop this disk and ASM doesn't have enough space on other disk to finish rebalancing (copying extents off those disks that are being dropped). It might be not obvious when you use normal or high redundancy. This way you need to consider how your disks are organized in failure groups. That's a nutshell. I hope this comment leads to the right investigation path without overwhelming with details here. Lemu,

Sorry I noticed your questions very late. You have probably already resolved it but here is quick explanation for others if they get here via search of other references.

the status of HUNG for ASM disk is “awarded” when you requests to drop this disk and ASM doesn’t have enough space on other disk to finish rebalancing (copying extents off those disks that are being dropped). It might be not obvious when you use normal or high redundancy. This way you need to consider how your disks are organized in failure groups.

That’s a nutshell. I hope this comment leads to the right investigation path without overwhelming with details here.

]]> By: Lemu Aceron http://www.oracloid.com/2006/05/vldb-with-asm/#comment-14374 Lemu Aceron Mon, 26 Nov 2007 05:35:12 +0000 http://blog.oracloid.com/2006/05/vldb-with-asm/#comment-14374 Hi Alex, Can you shed me some lights on how to bring back an asm disk in HUNG state back to normal state. NAME HEADER_STATU MOUNT_S MODE_ST STATE ------------------------------ ------------ ------- ------- -------- DATA1 MEMBER CACHED ONLINE NORMAL DATA2 MEMBER CACHED ONLINE NORMAL DATA3 MEMBER CACHED ONLINE NORMAL DATA4 MEMBER CACHED ONLINE NORMAL DATA5 MEMBER CACHED ONLINE HUNG DATA6 MEMBER CACHED ONLINE HUNG Hi Alex,

Can you shed me some lights on how to bring back an asm disk in HUNG state back to normal state.

NAME HEADER_STATU MOUNT_S MODE_ST STATE
—————————— ———— ——- ——- ——–
DATA1 MEMBER CACHED ONLINE NORMAL
DATA2 MEMBER CACHED ONLINE NORMAL
DATA3 MEMBER CACHED ONLINE NORMAL
DATA4 MEMBER CACHED ONLINE NORMAL
DATA5 MEMBER CACHED ONLINE HUNG
DATA6 MEMBER CACHED ONLINE HUNG

]]> By: Anton Krasimirov http://www.oracloid.com/2006/05/vldb-with-asm/#comment-13612 Anton Krasimirov Tue, 21 Aug 2007 21:58:42 +0000 http://blog.oracloid.com/2006/05/vldb-with-asm/#comment-13612 Thank you Alex :) ! Thank you Alex :) !

]]> By: Alex Gorbachev http://www.oracloid.com/2006/05/vldb-with-asm/#comment-13602 Alex Gorbachev Tue, 21 Aug 2007 01:59:20 +0000 http://blog.oracloid.com/2006/05/vldb-with-asm/#comment-13602 With 3 FG and normal redundancy the data will be distributed amongst all 3 failure groups so that primary and secondary ASM extents are in different FGs. I.e. if you have normal redundancy diskgroup with 3 FG 100 GB each then you get 150 GB usable space in this DG. If you use 90 GB of this diskgroup (real size of datafiles before mirroring) than each FG will contain 60 GB of data and extent randomly mirrored across 3 FGs. The failure of one FG will cause ASM to re-balance the data so that everything fits on two remaining failure groups - i.e. 90 GB in each FG. This should work just fine. If in the same configuration 120 GB is used, than each failure group contains 80 GB. In case of failure of one FG, ASM won't be able to re-balance the rest of the data on teo remaining FGs - it need to place 120 GB on each failure group whereas each FG contains 100 GB of disks. In order to sustain the failure of a single FG in a normal redundancy diskgroup with <i><b>n</b></i> failure groups, you have to keep at least 100/n percent of space free. 2 FGs - 50% 3 FGs - 33% 4 FGs - 25% 5 FGs - 20% and so on. With 3 FG and normal redundancy the data will be distributed amongst all 3 failure groups so that primary and secondary ASM extents are in different FGs. I.e. if you have normal redundancy diskgroup with 3 FG 100 GB each then you get 150 GB usable space in this DG. If you use 90 GB of this diskgroup (real size of datafiles before mirroring) than each FG will contain 60 GB of data and extent randomly mirrored across 3 FGs. The failure of one FG will cause ASM to re-balance the data so that everything fits on two remaining failure groups - i.e. 90 GB in each FG. This should work just fine.

If in the same configuration 120 GB is used, than each failure group contains 80 GB. In case of failure of one FG, ASM won’t be able to re-balance the rest of the data on teo remaining FGs - it need to place 120 GB on each failure group whereas each FG contains 100 GB of disks.

In order to sustain the failure of a single FG in a normal redundancy diskgroup with n failure groups, you have to keep at least 100/n percent of space free.
2 FGs - 50%
3 FGs - 33%
4 FGs - 25%
5 FGs - 20%
and so on.

]]> By: Anton Krasimirov http://www.oracloid.com/2006/05/vldb-with-asm/#comment-13561 Anton Krasimirov Thu, 16 Aug 2007 17:12:12 +0000 http://blog.oracloid.com/2006/05/vldb-with-asm/#comment-13561 Hello Alex I`m wondering what happens when i have 3 failure groups in normal redundancy . Will the diskgroup be still available if 2 of the failgroups fail ? and .. how is data distributed in this configuration Hello Alex

I`m wondering what happens when i have 3 failure groups in normal redundancy . Will the diskgroup be still available if 2 of the failgroups fail ? and .. how is data distributed in this configuration

]]> By: Alex Gorbachev http://www.oracloid.com/2006/05/vldb-with-asm/#comment-12416 Alex Gorbachev Mon, 18 Jun 2007 16:39:45 +0000 http://blog.oracloid.com/2006/05/vldb-with-asm/#comment-12416 Tim, Peta-Byte with ASM is quite impressive. And with 10.1.0.4 - sounds like a dream. :) You mention different types of storage -- did you mean it's good idea to manage them under ASM? I would disagree with that. Mixing fast and slow storage in one DG is a bad idea as ASM assumes they all have same performance and stripes based on SAME approach. Also, it doesn't look like ASM has convenient provisioning schema. At least, storage admins that I've been working with are not favoring provisioning in ASM. They are missing lots of enterprise features they have in Veritas stack, for example. And they are smart chaps so they can't be completely wrong. Anyway, thanks for sharing your experience! Tim,

Peta-Byte with ASM is quite impressive. And with 10.1.0.4 - sounds like a dream. :)
You mention different types of storage — did you mean it’s good idea to manage them under ASM?
I would disagree with that. Mixing fast and slow storage in one DG is a bad idea as ASM assumes they all have same performance and stripes based on SAME approach.

Also, it doesn’t look like ASM has convenient provisioning schema. At least, storage admins that I’ve been working with are not favoring provisioning in ASM. They are missing lots of enterprise features they have in Veritas stack, for example. And they are smart chaps so they can’t be completely wrong.

Anyway, thanks for sharing your experience!

]]> By: Tim Jacobs http://www.oracloid.com/2006/05/vldb-with-asm/#comment-12413 Tim Jacobs Mon, 18 Jun 2007 16:19:21 +0000 http://blog.oracloid.com/2006/05/vldb-with-asm/#comment-12413 Correct Alex, The SANS have enough reduntancy. I do like the idea of having Oracle level the storage, but I think that would be better when the database is smaller (tera) and there might be different types of storage, NAS and slow and fast storage. Correct Alex, The SANS have enough reduntancy. I do like the idea of having Oracle level the storage,
but I think that would be better when the database is smaller (tera) and there might be different types of
storage, NAS and slow and fast storage.

]]> By: Alex Gorbachev http://www.oracloid.com/2006/05/vldb-with-asm/#comment-12195 Alex Gorbachev Wed, 13 Jun 2007 22:37:36 +0000 http://blog.oracloid.com/2006/05/vldb-with-asm/#comment-12195 Tim, you didn't mirror between the boxes then. Right? Actually, if you have many storage boxes and many failure group - you are not that bad even with ASM mirroring. Tim, you didn’t mirror between the boxes then. Right?
Actually, if you have many storage boxes and many failure group - you are not that bad even with ASM mirroring.

]]> By: Tim Jacobs http://www.oracloid.com/2006/05/vldb-with-asm/#comment-12194 Tim Jacobs Wed, 13 Jun 2007 22:21:57 +0000 http://blog.oracloid.com/2006/05/vldb-with-asm/#comment-12194 I have used ASM with a Peta-Byte of Storage. We used external-reduntancy. No issues. Initial Start was slow due to many many many paths. We used HP's EVAs and EMC storage. This was on 10.1.0.4. ASM worked great. We had problems with the BIG Tablespace not really being that BIG.... But it just gets better with each release. I have used ASM with a Peta-Byte of Storage. We used external-reduntancy. No issues. Initial Start was slow due to many many many paths. We used HP’s EVAs and EMC storage. This was on 10.1.0.4. ASM worked great. We had problems with the BIG Tablespace not really being that BIG…. But it just gets better with each release.

]]> By: Alex Gorbachev http://www.oracloid.com/2006/05/vldb-with-asm/#comment-6373 Alex Gorbachev Mon, 26 Feb 2007 22:26:26 +0000 http://blog.oracloid.com/2006/05/vldb-with-asm/#comment-6373 Shiva, Well, yes until at least 11g that is as rumors say coming with fast resilvering. ;-) Cheers, Alex Shiva,
Well, yes until at least 11g that is as rumors say coming with fast resilvering. ;-)
Cheers,
Alex

]]>