Software or Hardware RAID Part 3: Conclusion - Hardware RAID Where Possible

A while ago I agonised over what an awful lot of people seem to agonise over when building and choosing servers and trying to judge the best options for storage, and came to the conclusion at the time that software RAID was going to be the best way because of the hardware we had available and the need for hardware RAID cards to really be in a PCI-X slot. Yes, you can use PCI slots, but really, you're limited by the I/O bandwidth. Like I said at the time, if you can find the right hardware then hardware RAID is preferable - and that's certainly the conclusion I have come to now. If you don't have the right board, or only have fake RAID available to you, please, just use software RAID. You will have pain, I promise you. You may have to forget hotswapping your disks though, because you really need your disk controller to react in the right way when a disk is disconnected and a new one added otherwise bad things will happen.

Well, the motherboard we were going to use just didn't happen for us for various reasons, and so I specifically looked at boards that would accommodate PCI-X slots. The availability of dual processor, server-based motherboards in the UK seems to be surprisingly spartan, with many dual CPU boards given over to ridiculous dual SLI graphics card workstation requirements. Needless to say, I don't need any of that. I eventually plumped for a Supermicro H8DAE after doing some reading around and reading some horror stories regarding an equivalent Gigabyte motherboard and PCI domains. Having got it all up and running now with CentOS, it's a very nice board with nothing more than I need with dual LAN, 4 PCI-X slots, 2 PCI slots, built-in graphics and IPMI support. Other bits and bobs in the new machine include:

• The hardware RAID card available to us, at a very reasonable price, was an Adaptec 2420SA four port SATA card. It's a very nice card having got it up and running on Linux. The one thing I was worried about were management tools, because the OS will simply not see or know if a RAID array has a problem - it just sees a disk, remember. The Adaptec Storage Manager is perfectly adequate, with e-mail notifications, and I must commend Adaptec on their improved Linux support. aacraid is directly in the kernel now, and I got the whole thing up and running with no yucky driver installs. Yay! We also have a lot more scope for rearranging our storage and adding larger hard drives later on, without any downtime if we play our cards right. Needless to say, this does not mean you shouldn't have a backup system!
  
• Chenbro server case. Solid little thing that is lockable and has a good SATA cage for RAID purposes.
• 2 x AMD 146 Opterons. There is no Pacifica virtualisation support with these, or with the motherboard I have, but quite frankly, so what? I'm rather sceptical about the support and quality for this for the foreseeable furture, and VMware will see us right. We already have some existing VMs anyway.
• Seasonic S12 600W PSU. A top quality PSU that is absolutely silent and totally cool (I'm talking about its temperature there). You couldn't buy any server with a better PSU than this. Spot on. Just be aware of a caveat with PSUs - make sure you get an EPS-SSI compatible PSU if you need it, otherwise some boards like many Tyans will just not power up. An additional CPU power connector isn't necessarily enough.
  
• 2 x Xilence Blade coolers. The ones pictured there seem to have an aluminium block, but mine have a copper one. There's a story behind this :-). When we ordered our motherboard the seller didn't pack the right CPU fitments for a H8DAE and packed the wrong I/O backpanel. They packed those for a H8DAE-2 instead (I got some free SATA cables though ;-)). Our seller refused to send the right stuff out even after it was explained (tossers), and rather than go through the hassle of changing the board I got on to Supermicro support who kindly sent out the right I/O backpanel. I then took the opportunity to get some CPU coolers that were more silent, cooler and better for dust and cleaning than the stock ones I had. I also needed a cooler that had its own motherboard backplate. The flower coolers were too big for the board, so I went for the Xilence coolers. At first I thought they weren't going to fit, so I turned one cooler around one way and the other in the opposite direction. The RAID cage was a little close for comfort to one cooler, but doesn't seem to have made any difference to airflow or cooling having monitored it. Both CPUs maintain an idle temperature of about 40 degrees celsius, with one CPU one or two degrees out, as it always seems to be in dual CPU systems.
  
• 2 x 1 gigabyte sticks of EEC registered DDR. Standard stuff. Just make sure you get EEC registered RAM for these boards.
• 4 x Samsung 200 gigabyte drives. You do get warned by Adaptec that you should use, supposedly, enterprise class disk drives because of the way a normal desktop drive retries on bad sectors. That article does say this can result in data loss, but I'm not entirely sure how, and I'm not sure if this is really relevant or whether it's just a way of selling more expensive hard drives.
  

All in all, a tidy little machine, and but for the system fan which is necessary considering I have two CPUs and four disks in an enclosed RAID cage in there (albeit with its own fan), it would be almost completely silent. I was quite impressed.

Most of my decisions on what operating system to use, and what flavour of Linux to use, centred around what kind of aacraid support it had and whether I could easily get the Adaptec Storage Manager up and running or not. I mean, if the underlying disk storage is flaky and I can't manage the RAID array and get notifications when something goes wrong, what's the point? Everything else will fall to pieces around it. As it turned out, Adaptec has RPMs built for Red Hat and Suse specifically, but it isn't a whole lot of work to get it up and running on any 32 or 64-bit Linux system. The right dependencies are all there, which is the 32 or 64-bit libXP library as far as I can work out.

So I went for a RHEL (Red Hat Enterprise Linux) compatible system, which for us is CentOS 5. We're not running too much on the base system here, as much of our functionality will be on further virtual machines. CentOS has given me the option of using driver disks from Adaptec and other manufacturers if I so wish (hasn't been necessary as it turns out), packages such as Adaptec Storage Manager I can just install without tinkering, a bit more confidence in 64-bit and 32-bit backwards compatible support, better Xen support if I want it and much more predictable updates to the system. There are some things about CentOS, and hence RHEL, that I don't like but I'll get on to them at another time.

Conclusion: Go for hardware RAID every time, but you must find yourself the right controller. If you can't, you don't have the right hardware and you only have, God forbid, fake RAID then go the software RAID route.