Big storage on a budget (Mac Pro)

[ciamara]

I am currently trying to figure out how to get more storage on a reasonable budget. I like the Seagate Barracuda drives. Currently I have a Mac Pro with 4 internal drives:

- 500GB Hitachi 7200 rpm drive for OS

- 500GB Hitachi 7200 rpm drive for Time Machine

- 1TB Seagate Barracuda 7200 rpm drive for data in RAID 1 (Mirror)

- 1TB identical to prior drive (Mirrored)

 

The big data hog, of course, is all the audio (AIFF) files in my music library. I am approaching 500GB and growing fast. I anticipate being at 1-2TB of music data by year-end.

 

A few options:

- firewire drives

- NAS

- add more drives to the Mac Pro (see below)

 

OS X has a great software RAID feature for mirroring or striped RAID arrays. Time Machine is great too. I like the speed of 7200 rpm drives and eSATA, and frankly, as you get to large amounts of data, copying from one to another starts to take a lot of time.

 

I saw this:

http://www.maxupgrades.com/istore/index.cfm?fuseaction=product.display&product_id=158

 

Anyone have any experience with this? Or any other suggestions? (Hopefully the foregoing was clear enough).

 

[The Computer Audiophile]

The MaxUpgrade options look pretty cool.

 

You could just install larger drives into the four existing drive bays.

You could go the MacUpgrade route

You could use an eSATA enclosure that holds several disks

You could get a NAS that holds as much music as you could ever throw at it

 

It all depends on your style. What I mean by that is, do you want to fiddle with the Mac and install aftermarket disk solutions like the MaxUpgrade, do you like hanging external drives off the tower, or do you want tons of disk space off in a back room somewhere out of sight, mind, and earshot?

 

[Andrew S.]

Chris

what are the data transfer speeds like with the NAS you are running - the Thecus? I find the single biggest annoying issue is slow data transfer. Fine once you have it there - total pain to get there. Is ethernet or esata quickest?

 

[ciamara]

Well, the Mac Pro already sits in a machine room away from the listening area, so noise and heat are not concerns. And I don't mind spending a little time tweaking the machine. First and foremost, I'd like to grow stepwise with as little up front cost as possible. The MacUpgrade thing looks cheap enough at $89-139 to take a punt. The second consideration is transfer speed. The 7200rpm drives seem to be fine, though they do hiccup every now and then as they approach 70% capacity. This is tolerable, but I don't want to deal with anything slower.

 

One other consideration ... If you have, say 2TB of AIFF files on 3 different hard drives, does it become difficult to later move the library to another computer? Or is it better to use RAID (striped or otherwise) to make the OS see one disc location?

 

Thank you for your help!

 

[ciamara]

I'm not 100% sure, but I think you get 3 GBit/s for eSATA connections, 1GBit/s for ethernet connections, 800MBit/s for Firewire 800, and 400MBit/s for Firewire 400. It looks like the Thecus offers dual Gigabit connections, so presumably that means 2Gbit/s in either direction? I'm not sure.

 

One other thought ... When you say virtually unlimited space, do you mean simply adding more and more NAS boxes over time? Thank you.

 

[ciamara]

I'm not 100% sure, but I think you get 3 GBit/s for eSATA connections, 1GBit/s for ethernet connections, 800MBit/s for Firewire 800, and 400MBit/s for Firewire 400. It looks like the Thecus offers dual Gigabit connections, so presumably that means 2Gbit/s in either direction? I'm not sure.

 

One other thought ... When you say virtually unlimited space, do you mean simply adding more and more NAS boxes over time? Thank you.

 

[emmodad]

for pure bulk storage, would recommmend you consider drobopro. connection via iSCSI has very fast transfer rates to/from host; you can fall back to FW800 if needed. iSCSI initiator (the SW "driver" for iSCSI on OS X) included in drobo SW. works very well in 10.5.x

 

i have drobopro for bulk library storage (connected iSCSI, and configured for two-drive redundancy (meaning two of the eight drives can fail)); and other drobos / NAS / drives / audio interfaces (connected FW800 and FW400) through the simple "hub" of a 15" PowerBook

 

drobopro unit itself is not "inexpensive," but considering overall cost of "bulk storage system"; mindless expandability (hot swap/add/remove drives, no RAID management required); and data storage redundancy (drobo uses a mix of RAID-style mirroring, parity and striping techniques); it seems to be reasonable money spent up-front in building a storage core easily expandable up to some huge number of TB. I currently have 6x 2TB in it; waiting to populate the other 2 slots (with 2x 2TB) for drive prices to drop in the next round of market maturity over coming months (and other vendors to come in). meanwhile, will be slotting in 2x 1TB that I have lying around from other RAID units I'm selling.

 

one more drobopro benefit: you don't need to populate it with "fast" drives or enterprise-spec'd drives, works admirably with ie lower-priced bulk WD Greenpower drives (dynamic rotation speed control as part of their power saving algorithm) which are "quoted" as 5400-6000rpm. have spoken w/ several of drobo's engineers; their product positioning (RAID-like with simple ease-of-use for users with lots of media, ie photographers, musicians etc) targets use of commodity drives in a high-reliability but simple useage scenario.

 

another bulk-storage option, if you are on OS X and have many disparate drives (and even older ones of varying sizes) sitting around: SoftRaid is generally considered a step up from Apple's integrated RAID support. A simple and stable way to combine many external drives in RAID sets as part of your redundant backup scheme.... i use this with a slew of older drives, provides another manner of redundancy.

 

 

edit -- correct typo FW400n > FW400; some formatting; add iSCSI initiator note and "not inexpensive" paragraph

 

[ciamara]

Thank you very much! I think I'll go ahead and buy 2 x 2TB drives for now to replace the 2 x 500GB drives (I'm planing to reinstall fresh when Snow Leopard arrives anyhow) and I will definitely check out SoftRaid. It sounds like drobopro is the way to go if you have a bunch of different disks lying around. I like being able to grow stepwise like this. Thanks for the help!

 

[ciamara]

So I bought a 2 TB Seagate LP drive for now. I now have:

- 500GB Hitachi 7200rpm for OS

- 2TB Seagate Barracuda LP, which I will use for Time Machine

- 2 x 1TB Seagate Barracuda 7200.11 drives

 

The old 500GB drive is sitting on a shelf with my previous Time Machine backup. As I grow, I think I will probably move towards the drobo or thecus solutions, though I'm still not sure which yet. Speed is not so much of an issue for a music server, even at high res .... It only matters for other things I do, and fortunately, they don't take as much space.

 

I realize this is more of a Mac question, but I am thinking of setting up the 2 x 1TB drives in RAID 0 (striped array). I am aware that if one drive fails, I can't rebuild the set, but I will have the Time Machine backup, so at most will only lose 1 hour of work in case of catastrophe. My question is this ... If, when I install Snow Leopard on the 500GB drive, I choose to erase and install clean (rather than upgrade), will the RAID 0 set be recognized? If not, I will wait until after the OS installation to set up the RAID 0 config.

 

Thanks all for your help!

 

[iamimdoc]

My 2c worth:

 

In general, music is not "mission critical" - ie, a few days without music is very (very) irritating but not the same as downed software needed to run your business, a hospital, etc.

 

Thus, having RAIDS that prevent any down time, IMO, aren't a great benefit to home music lovers. In addition, bad stuff can happen in homes - theft, water damage from broken pipes, fires. etc. Having to rerip your music would be catastrophic due to the time involved.

 

Accordingly, I think the best way to save your music is a working copy and a backup at a friend's house, bank vault, etc - somewhere not in same location as your original.

 

I have found a $50.00 device from IOCELL. One places a drive (or 2) in the gadget and the device can be configured as an NAS AND/OR USB. One has the utility of both the NAS worlds and USB plug and play. Thus for 150.00 or so one can have a 1-2 TB NAS drive. Get a second device and leave it at someone else's house for security.

 

$300-400 all in for 2 TB of files, backed up safely

 

I have 2 of these gadgets and they work flawlessly. There is never a delay with streaming music, no hiccups, etc.

 

 

[ciamara]

I like this idea, though the thought of doing this on a regular basis to keep the two synced up hurts my brain. If there were a way to do this through the internet in real-time it would be great! Remote storage still seems a bit expensive for libraries that are 1TB+. Maybe I'm wrong ... Honestly, I wish Apple would just migrate the whole Time Machine concept to a high speed iDisk that works. iDisk is cool, but 20GB is way too small, not to mention the transfer speeds are way too slow. In time I guess we'll get there....

 

[iamimdoc]

Presumably after the initial setup you aren't _continually_ ripping stuff.

 

If you are, back it up (more) locally (like a $50.00 USB drive) and then move it "off campus", once a month or whatever.

 

I agree, keeping track of all of this is a hassle, particularly if you do it my way as things are not automated. In my scenario, I just put new stuff in a new directory and know this is all I have to back up though.

 

Regards.

 

 

 

[ciamara]

You are right ... it is totally manageable. I'm just a big baby and lazy when it comes to these things! For the next 6-12 months, I think I can live with the 2 x 1TB in striped RAID for all my data (30GB is my entire life / work files; the remainder is audio files) and use the other 2 TB drive for Time Machine backup. As I outgrow that, I think I will look at this NAS / Drobo much more carefully ... and hopefully, prices will come down a bit by then too. Thanks for your help!

 

[Happy]

 

Sanjay:

 

I use 4 internal drives of the MAC Pro. Drive 1 - 500G System, Drives 2/3/4 Apple Concatenated RAID (I tried Soft Raid - wasn't a big fan). This gives high Bandwidth to the disk system - and 1 flat 2.7Tbyte volume - good for itunes.

 

I backup the System disk to an external removable disk once a week. - This is a bit copy to allow hassle free booting should the internal disk fail. (see SuperDuper)

 

I don't use time machine.

 

I dislike drobo although I have one. - Its great until it fails.

 

Be Careful of eSATA cards - cards based on the SIL3132 chip set (all of them except the Sonnett card) cause a kernel panic in Leopard - luckily Snow Leopard throws them out which leaves you with a non functional eSATA card). On a MAC Pro it is possible to use the second eSATA on the motherboard.

 

I have a copy of the music library on secondary machines (this is an ongoing backup using ChronoSync).

 

If you need more space put in 4 drives and use an external firewire drive for the OS.

 

Remember the MTBF of a SATA disk is 2.5 years - they fail cummulatively. You should plan for eventual(inevitable) disk failure.

 

/Paul

 

 

 

[ciamara]

Very helpful indeed. Thank you for this. Any reason you do not like TimeMachine? I once screwed up my system by installing some Apogee hardware, and was able to restore everything within 45 minutes using TimeMachine ... and it was a no brainer. Just curious ...

 

Also, I had no idea you could run OS from a firewire drive. Interesting. Does it slow down the OS, though? I use my machine for a lot of things besides music.

 

P.S. I see you have a Lavry DA10. Nice. How do you like it?

 

[Happy]

 

Time Machine was (is) a huge improvement in backup technologies for consumers. At the very least those who didn't do backups now find it easier. The reason i don't use it is; it can consume lots of disk space, takes time and more importantly if you have a failure of the time machine disks then its can be impossible to recover.(there is nothing worse than going to a backup only to find that the backup has failed - restore from backup is a golden opportunity in the backup/archive world).

 

If you use Time Machine use a RAID 6 backup device - note do not ever use a RAID 5 system; they have been proven to be unreliable (A disk failure with another Latent Sector Error causes complete RAID5 failure).

 

Running the OS from a firewire drive (I think USB also works but I haven't used that) is a tried and tested thing to do. The iniitial boot can be longer; but once the OS is in memory then it doesn't really use the OS disk - the big use of the OS disk is for the swap file - the swap file is better mapped to an internal RAID0 disk.

 

Since my post I have found that in a MAC Pro one can add another disk (or 2) in the spare optical slot - there is a spare SATA port on the motherboard.

 

http://www.transintl.com/macupgrades/index.cfm

 

I started with a Benchmark DAC to my Adams Monitors. I then tried the Lavry - it is excellent. Had it for 3 years before I went to the Alpha DAC.

 

/Paul

 

[ciamara]

Paul, you were right on the Time Machine thing. It is good for work files, backing up your system state regularly, but not so much for audio/media files. Mirroring for a media drive (or striped RAID) seems better to me. On an unrelated note, 2 weeks after I bought the Seagate LP drive, it died. I went back to the retailer and ended up getting two Western Digital Caviar Green 2TB drives. So now I have the following, which I much prefer:

 

- Drive Bay 1: 1TB Seagate Barracuda 7200rpm drive (for OS and work files)

- Drive Bay 2: 1TB Seagate Barracude 7200rpm drive (for TimeMachine -- but excluding media files)

- Drive Bay 3/4: 2TB Western Digital Caviar Green 2TB drives mirrored (for music/media files)

 

 

[Audio_ELF]

Sanjay ...

 

Remember Mirroring is NOT an alternative for a backup.

 

By far the most likely reason for needing to refer to a backup is because of user error or system corruption overwriting data - not the failure of a drive. If you are just using Mirroring, then any overwritten data will be written to both drives, and you will not be able to go back to previous data.

 

I urge you to get an external USB or FireWire drive and do an additional backup using SuperDuper or similar.

 

Eloise

 

[ciamara]

Thank you. My primary concern with my music/media files is drive failure. I'm willing to accept that if I delete something AND empty the trash, I lose the data permanently. Do you still feel this is dangerous? In my prior setup, I used Time Machine to create hourly backups, but this turned out to be a problem, as I am constantly adding and changing media files. Given I have about 1.5 TB of music files (soon to hit 2-3 TB), and I also use this computer for work (I have about 100-150 GB of data files) what would you recommend as the best storage solution/setup? Thanks for your insights.

 

[Audio_ELF]

To my mind, the issue isn't so much risk of deleting and then emptying the trash, it's the risk of (for example) not realising you have everything selected not just tracks from one CD, clicking Info and changing EVERYTHING to the Artist "Kylie Minogue" (for example). Or that Snow Leopard version 10.6.2 has a melt down and deletes your profile including all your music for example.

 

As I say, it's your decision, but for £100 for a 2TB drive and remembering once a week / fortnight / month to do a backup, is it worth the risk? You can also remove the drive from the premises and if the worst should happen, you've got an offsite backup.

 

Eloise

 

[iamimdoc]

I totally agree.

 

Get a USB drive and backup by directory. Put new stuff in new directories and you only have to due an incremental backup. Or backup the whole deal - and wait, wait, wait...

 

Or look at the IOCell (see my August post) device that is NAS type device (but can be unplugged from the network and will work as a USB drive elsewhere). I have 2 and never had a hiccough with > 2TB of data

 

Take it somewhere else offsite (or at least a different location in your house).

 

For < $200.00 you have safety and 2 TB backed up. In the next 6 months it will probably be 3 TB for same price.

 

[ciamara]

Yes, you raise excellent points. I really wish there was a good online backup tool that would synchronize hourly, preserving hourly states ... But I guess we're not there yet. I'll do as you suggest .... Thank you!

 

[Happy]

Just a note: 2 Tbyte SATA drives are out now; 6Tbyte SATA drives will be available by the end of 2010 (They are being tested in labs now).

 

/Paul