instead of SSD server -- cache to OS X RAM Disk for playback?

[emmodad]

long post intended to stimulate discussion on playback from solid-state memory: using portion of built-in computer RAM as a large cache providing solid-state memory playback without the cost of adding large SSD for library storage

 

seems somewhat more cost-effective than many thousands of dollars for some of the "playback-from-SSD" standalone servers which are appearing, or the newer breed of CD players/rippers with several-second buffers.....

 

musings and questions follow, input from folk knowledgeable about iTunes memory management / UNIX / OS X programming would be appreciated.

 

this is quite possibly applicable to other OS's, however my specific interest for this thread is OSX

 

---

 

thought process:

 

- useage mode -- i have a large music library, multiple drobos (and yes, they're backed up to a variety of RAIDs and hard drive sets). i tend to listen to music entire-album-at-a-time or entire-concert-at-a-time. iTunes and other players only cache a small portion of audio after fetch, hence requiring significant disc access during playback. this can cause (depending on people's religious beliefs wrt computer-based audio) power-supply-transmitted noise; time anamolies in data transport and buffering; etc, etc.

 

why not dedicate a portion of the computer's built-in RAM as a CD-sized (or larger) intermediate playback cache from a large music library stored on low-cost external hard drives?

 

---

 

- hard drives (HDs) are cheap storage for large music libraries (mine is currently 5 TB and growing...)

- Many thoughts exist opining that playback sourced from spinning optical drives has issues when compared to playback from solid-state memory, ie SSDs.

- SSDs are expensive, have limited capacity, and generally need a HD bay for location.

- (caveat: small fast "SSDs" up to 100 GB can be based on Compact Flash and other memory cards in expansion slots; i also use a 2xCF Card adapter to create a lower-cost 64GB flash drive in an external FireWire enclosure, it could be used with two 100GB Cards if i could afford them....)

 

but:

 

- typical album is under 1 CD in size; lossless compressed is even smaller. Grateful Dead concerts are however several CDs in size.....

- Mac Mini has 2GB to 4G RAM; PowerBooks and MacBook Pros offer anywhere from 2GB to 8GB RAM

- a portion of this (relatively inexpensive) built-in Mac RAM can be dedicated as a RAM Disk

- (caveat: there are some questions about RAM disk persistence and cacheing of their I/O, hence this post)

- transferring from main music library located on HD to a RAM Disk "located" in computer's internal RAM can be faster than transferring from HD to an SSD located on a bus

 

---

 

- brute force: music files can be copied manually to RAM Disk for subsequent playback out of solid-state memory by the music player SW of choice (I'm listening now to Matt Bianco 1984 excellent CD)

 

- alternatively, someone with more programming smarts than me could figure out if a music player could be modified to cache an entire selected album from a library located on HD into RAM before starting playback?

 

- maybe there could be an AppleScript to do this when a title has been selected (cache to RAM Disk the entire album containing the selected title; commence playback from the cached version, perhaps starting before entire cache write process is complete, in order to minimize playback delay)

 

---

 

specific objective: instead of using add-on flash memory solutions (expensive / slow write time / etc) to store music files for playback, define a portion of the computer's existing built-in RAM as storage for audio files which is wired (locked-down, ie not subject to virtual memory paging) and exempt-from-I/O-caching.

 

best world: again, my primary listening mode is playback of full album at a time. So, keep music library (in my case, several TeraBytes) on lower-cost, high-redundancy media (RAIDs, drobos); album selected for playback would be cached in its entirety before playback starts

 

simplest: music player (iTunes, Cog, Play etc) could have its memory management modified to use a specified RAM Disk as playback buffer with increased size to hold much more than a few seconds (ie a 800MB RAM Disk)

 

harder: some type of automation so that title selection in music player software would cause a cache write process to pull the entire associated album from main library HD and write it to RAM Disk.

 

this RAM space (a RAM Disk?) could be used simply as space from which stored files could be played back

 

i am absolutely not a programmer, so specific questions have to do with:

 

1/ creating RAM Disks which are wired in memory (ie non-pageable)

2/ creating RAM Disks which somehow will be exempt from having OS X cache disk I/O

 

---

 

so, let us presume for sake of discussion in this thread that there are viable reasons to play back computer-based audio from semiconductor memory instead of hard drives.

 

we'd all love to have nice big fast SSDs in order to playback from big libraries stored in solid-state memory instead of on spinning disks (insert references here to threads / posts about SSD vs HD, etc etc)

 

some of us however don't have the disposable income, but do have several PowerBooks with PC Card and ExpressCard slots, and upwards of 2G of built-in RAM.

 

---

 

step 1: i have been experimenting with playing back music stored on memory cards in the slots (PC Card etc). In absolutely unscientific, uncontrolled, non-rigorous and hence unacceptable to tweakheads testing (no ABX, no jitter analyzer bench equipment; stripped-down OS, no active application after boot except music player (iTunes, Cog or Play); standardized playback path via 2 different paths (USB > Transit > TOS > DAC1; FW > FF400 > SPDIFcoax > DAC1)) -- I detect what I perceive to be a positive difference in playback, generally in sonic areas one would associate with improvements due to improved system jitter handling (especially low-frequency recreation). so seems like the play-from-RAM stuff indeed has legs, onward to

 

step 2: store music on a small RAM Disk created from a portion of the Mac's existing internal RAM. I use Esperence DV to create a RAM Disk. a CD or concert's worth of tunes is manually copied to the RAM Disk (very fast); iTunes or other player (ie Cog or Play for FLAC files) uses that as playback source. Seems to be pretty good, just like the option in step 1 above; in any event, seems to provide similar sonic benefit versus sourcing files from HD.

 

step the next: some way to simplify / automate what is currently a manual process [ "i wanna hear "Stop Making Cheese"" > manually copy files to RAM Disk > point iTunes to thise new files via new playlist or whatever > play ] >> to become >> [ select title from iTunes existing library, or point to title on HD > (script?) > iTunes copies selected album of files to RAM Disk > iTunes plays cached copy in RAM Disk ]

 

---

 

RAM Disk issues (hence the questions for UNIX- and OS X-savvy programmers)

 

1/ creating RAM Disks which are wired in memory (ie non-pageable)

2/ creating RAM Disks which somehow will be exempt from having OS X cache disk I/O

 

1/ avoiding OS X Virtual Memory paging of RAM Disks -- is there some way to protect a RAM Disk by defining it to be "wired" memory so that data will never be paged out to hard disk?

 

2/ i've seen information implying that OS X caches disk I/O for RAM Disks. this would have effect of a speed slowdown AND would use more system memory. OK perhaps on a MacPro with 32GB, less OK on a PowerBook with 2GB. is there some way to define that a particular device (ie this RAM Disk) could be exempted from such I/O cacheing?

 

---

 

any and all constructive thoughts gratefully appreciated.

 

[BEEMB]

 

Emmodad,

 

I read your post with interest.

 

I moved to PC playback from OSX a good number of months ago (not for music playback reasons) so I can't really answer your question.

 

Some playback software on the PC already does exactly what you're asking, loading (maybe not albums) songs into RAM prior to play back. XXHighEnd is one such example.

 

I accept that there may be an argument for improving sound quality by reducing the number of moving parts involved in playback, and in a few years solid state hard drives will be far cheaper than they are now. However, I've never seen any proof of this and in a computer system I'm not sure there can actually be a difference - if the data is not there, if a 1 or a 0 is missing, it's corrupted.

 

If your reasons are for something other than sound quality then please disregard my post. As I know hard disks can fail over time.