SACD Ripping using an Oppo or Pioneer? Yes, it's true!

[mindset]

I'm able to stream the iso to my Mac, which is convenient. However, iso extraction to DSF seems quite a bit slower than on Windows. Is this due to the need to package sacd_extract with Java? Any method of extracting more rapidly on OS X? If not, I can always bounce the iso to Windows or Linux for extraction, though that would be a little less convenient, or just wait for the longer extraction on OS X.

The build configuration file in the original sacd_extract github is missing the compiler optimization option for both OS X and Linux. As a result sacd_extract Linux binaries available on the web (including those packaged in iso2dsd) are much slower than they should be. I don't have a Mac so I cannot confirm this for OS X, but probably the same is the case for OS X binaries as well.

I tried compiling sacd_extract for Linux with compiler optimization turned on by modifying the build config file. That gave more than 3x speed boost in dsf extraction from the same iso and PC.

[mindset]

I have a couple of questions about optimization. I'm not a developer and don't really know what I'm doing, but have compiled things before in FreeBSD, Linux, and, using XCode, for an OS X native app (Audirvana when it was open source). Thus I can follow "cookbook" instructions, but don't know how to recover from any but the most obvious elementary errors. So this being said, mindset, can you tell me what optimizations you selected? And does anyone know whether XCode will compile sacd_extract on its own, or whether I need something like Homebrew?

 

Thanks all.

 

It is probably much easier to use command-line tools (just cmake, make, and gcc) to compile sacd_extract as described in the build instruction than setting up XCode for that. You can get the latest source tree from the sacd-ripper github. Build instruction is in readme.rst. Scroll down to "SACD Extract Build Instructions". You will see that there are very few steps for compilation as pretty much everything is automated. In order to enable compiler optimization, add -O3 after -pipe in line 30 of tools/sacd_extract/CMakeLists.txt before running cmake. At least for Linux this makes a huge difference, and looking at CMakeLists.txt, I believe this is the case for OS X.

 

Just for reference,

For the same ISO and machine (Ubuntu 16.04 on Core i7-5557U=dual core). Both ISO and output directory are local SSD.

Custom build with optimization: 5.14 MB/sec

Custom build without optimization: 1.45 MB/sec

Binary in iso2dsd_Linux_v3.zip: 1.45 MB/sec

 

Hopefully you will get a similar speed boost on OS X.

[mindset]

Edit: Nope, if I don't want to do a lot of fiddling with environment variables, etc., looks like Homebrew, MacPorts, something like that. Any preference amongst the crowd for either, or another alternative?

 

How about running a windows binary with wine? This doesn't involve any compilation. I tried this on Linux. It doesn't run as fast as the native optimized binary but still runs more than 2X faster than the non-optimized Linux native binary. I believe wine is available on OS X as well.

 

Command used:

wine sacd_extract.exe -s -ia.iso

 

Comparison using the same ISO and machine (Ubuntu 16.04 on Core i7-5557U=dual core):

Custom build with optimization: 5.14 MB/sec

Custom build without optimization: 1.45 MB/sec

Binary in iso2dsd_Linux_v3.zip: 1.45 MB/sec

Windows binary in iso2dsd_V7.zip run with wine: 3.84 MB/sec

[mindset]

21 hours ago, Antone said:

 

 

The sleep issue is a real drag.  I hope someone figures out how to keep the thing awake, I was thinking of trying to use a streaming service like hulu and watching videos while it rips, but it won't let me use my personal HULU account???

 

Not sure if this is related, but for that sleep issue, have you tried turning on the Quick Start Mode of the player and start ripping AFTER putting the player in sleep?  If Quick Start Mode is enabled, the UNIX system in the player keeps running even during sleep (power light = OFF) and you can even rip SACDs in sleep.

 

I have been happily ripping using BDP-S390 for more than a year with this method (plus firmware hack for pure USB-memory-less operation based on Malcom Stagg's study) though I don't have such a big SACD like yours.

 

By the way, I have also tried BDP-S5100 and BDP-S590 and these also work, too.

[mindset]

First of all, I started with the browser exploit which allows execution of arbitrary program from a USB drive to gain telnet access.  (google Malcom Stagg for the exploit.  It's easy to find.).  Then, I ran sacd_extract_160, insert a disc, put the player in sleep, and rip over LAN.  I don't remember doing anything special besides this.  I am not sure how different this is from the popular Pioneer 160 approach people here are using.

 

Then, I thought it was inconvenient to do this each time I reboot the player so I switched to firmware modification for USB-memory-less operation.  I might have gone straight to firmware modification for some of the players I mentioned.  (Sorry, it has been 1.5 years since the time I was playing with them)

 

USB-memory-less operation: one can modify the firmware so arbitrary programs (such as sacd_extract_160, telnetd, ftdp, etc) in the little non-volatile disk space (/3rd_data) are executed before the main player program (bdpprog) so once the player powers up, it's ready for ripping.  Again, please refer to Malcom Stagg's webpage for the method.  Unfortunately it is not very straightforward as it involves extracting the firmware file system, extracting it, modifying it, rebuilding the firmware, and packing everything together.

 

[mindset]

OK.  I found three S5100's in my closet (used them for some other experiments) and one of them is still carrying a factory firmware.

 

Autoscript pretty much worked except that I had to replace /mnt/sda1 with /mnt/sda for the USB drive mount point (could be because of the disk format), and I needed your workaround (=touching the SACD playback setting) OR my workaround (=putting the player in sleep) after disc insertion to be able to rip a disc.

 

So it looks like you and I independently came up with different workaround for the same goal.

 

[mindset]

By the way S5100 doesn't rip any faster than S590 or S390.  The firmware on my S5100 is the latest = M15.R.0197.

 

Another thing:

This might have been discussed earlier, but I see many people asked for help, and their problems seem to be related to the way USB stick is formatted which can affect the mount point while autoscript assumes /mnt/sda1 as the mount point.  I see that the mount point of a USB stick is not fixed and it even changes each time it is reinserted.  Maybe changing Autoscript like this will help?

 

Original:

CLI(CLI_exec cp /mnt/sda1/AutoScript/sacd_extract_160 /)

 

New:

CLI(CLI_exec cp `ls -d /mnt/sd*`/AutoScript/sacd_extract_160 /)

[mindset]

14 minutes ago, Phthalocyanine said:

So this command  looks for any folder in the mnt directory starting with sd (which will catch all the variant mount assignments)

and copies sacd_extract within.  Clever.  Have you tested it out in practice?

 

Yes, I tested the new command.  It worked for me.

 

17 minutes ago, Phthalocyanine said:

What else have you got in that closet?

Have you got any of the SACD playing Sonys from 2014 like the BDP-S6200 or from 2015 like the BDP-S6500 to test out?

 

Unfortunately, I have got only S5100, S590, S390, and S1100.  S1100, which doesn't support SACD, has the same optical drive as S5100's but I couldn't get it to rip an SACD.  It does recognize an SACD as SACD (according to bdpprog log) but something (probably software) is preventing it from using the SACD layer.

 

For S6200 and newer, the firmware structure has changed, so we can't even extract the firmware using Malcom Stagg's method.  This makes me think that there might have been lots of changes to the firmware which could potentially include sealing the back doors like the autoscript and browser's USB binary execution.

[mindset]

42 minutes ago, Phthalocyanine said:

But it no longer can read the USB drive either.  I confirmed with telnet ls command that the player no longer see sda or sd-anything in the mnt folder when it is in sleep mode.

 

This is a fatal flaw for my preferred method of ripping, which is "Local" ripping.  In this method,  I change directory to the sacd_extract_160 in the USB drive and issue the telnet command for the player to run the program from there and the player copies the ripped .iso into that folder on the USB drive.

 

Oh, I see.  It's opposite for me.  Ripping over LAN has been always my preferred method (I have setup a script so I can get .iso ripped, extracted, and stored in proper locations in my LAN in one shot) .    I can confirm that the USB drive disappears during sleep.  If you need to rip locally, the sleep method is not for you unless some one can find a way to "wake up" USB during sleep

[mindset]

On 4/21/2018 at 6:53 PM, Phthalocyanine said:

One important thing I learned trying to do your method is to warn beginners to make sure that the "quick start" setting is OFF.

 

With "quick start" on, the player never really shuts down and it is hard to start from scratch if something goes wrong in the ripping process.  Most importantly, with "quick start" on, the USB drive keeps getting re-named sdb1, sdc1, sdef, etc. every time you reinsert it and you can't get it back to sda1, which you need for the stock scripts.  (I understand now why you wrote that special script.)

 

The only problem left with that is that depending on the way USB flash drive is prepared, it can be mounted as /mnt/sda1 or /mnt/sda (I have seen this).  I think modification of the AutoScript like I mentioned before (copied below) will solve that problem and it provides the freedom of whether or not to enable the Quick Start Mode.

 

CLI(CLI_exec cp `ls -d /mnt/sd*`/AutoScript/sacd_extract_160 /)

 

[mindset]

4 hours ago, Phthalocyanine said:

The Sony S390 is low too, so it might need it as well.  I haven't tested it without the line.

 

I have never needed to insmod splitter.ko on S390.  However, it doesn't mean it's not needed.  The fact is that splitter.ko is already inserted by something before the AutoScript is run.  Therefore, when AutoScript tries to insert it, it just fails to do so and just moves on.

 

As a proof,

One can manually remove splitter.ko from the kernel by:
rmmod /lib/modules/2.6.35/BDP/splitter.ko

 

then the player will not be able to rip (the whole player will freeze when ripping starts).

 

 

[mindset]

3 hours ago, mindset said:

I have never needed to insmod splitter.ko on S390.  However, it doesn't mean it's not needed.  The fact is that splitter.ko is already inserted by something before the AutoScript is run.  Therefore, when AutoScript tries to insert it, it just fails to do so and just moves on.

 

 

Correction.  It looks like the main player program (bdpprog) inserts splitter.ko when a disc is inserted.  I see this in the output of bdpprog upon disc insertion:

 

[NOTIFY] send_notification:0x3ff
[DRV_INIT_PROG] driver_notify, argc:2
[DRV_INIT_PROG] vDriverInit, level:2
[DRV_INIT_PROG] vDriverInit, to insert splitter module!
[DRV_INIT_PROG] vDriverInit, insert splitter module!
[DRV_INIT_PROG] insert_kernel_module splitter.ko, inserted:0xf0, to insert:0x1
[DRV_INIT_PROG] LoadFileToDram, file_size:0xbb300
[DRV_INIT_PROG] insert_kernel_module, dram_pointer:0x79260000

 

[mindset]

57 minutes ago, MikeyFresh said:

The player woke up, and opened the tray, however it should be noted the tray closed again all on it's own within about 1-2 seconds. So fast I didn't have time to remove the disc (be aware of that if you try to take the disc you just ripped out of the tray you have almost no time to do so before the drawer starts to close again).

Try removing the following line from AutoScript (and AutoScript.TSS just to be safe):

 

CLI(CLI_drv.ir.rx.sq 0xaf000)

 

This line opens the tray (and possibly closes it), and is unnecessary for sacd_extract to work.

 

Alternatively, if you are using the server mode, you can remove the USB stick once sacd_extract is loaded by the player .   sacd_extract will keep running even after ripping or sleep as long as quick start mode is enabled.  There will no problem with sequential ripping.

 

 

[mindset]

8 hours ago, hfl said:

I have successfully ripped about 50 older SACDs. These are single layer meant to be played only on an SACD player. Of that group, 3 failed using the telnet with sacd_extract_160 files. I have seen a reference to some SACDs not working with sacd_extract_160. Can someone provide some insight into how to deal with this issue? Is there another version of sacd_extract that would read these discs?

 

I believe you are doing local ripping on the player side.  For local ripping, It seems special characters are not handled properly because character code conversion doesn't work properly on the player side (you are probably seeing a bunch of error messages regarding this).  If you rip a disc in the server mode, character conversion happens on the client side in which case character codes are most likely translated properly.  At least that's what I have seen and here are some of the ripping results without any special treatment:

 

 

I use Linux exclusively for ripping.  I am not sure about Windows.

 

[mindset]

37 minutes ago, Phthalocyanine said:

Yes iso2DSD in Windows using the server method can handle the special characters.  That's what I was trying to say in my previous post.

Thanks. So, it looks like the special character issue exists only in local ripping.

 

38 minutes ago, Phthalocyanine said:

By the way, what software do you use for your server method ripping under Linux?

I use sacd_extract.

[mindset]

27 minutes ago, wl_lam said:

insmod: can't insert '/tmp/fileOGL2cs': Device or resource busy

[0]: install_modules: mknod/insmod filed

rmmod: can't unload 'sacd_read': unknown symbol in module, or unknown parameter

[0]: Can not install modules

 /mnt/sde1/AutoScriptSACD #

sacd_extract_160 is already running in the server mode so you cannot run another sacd_extract_160 for ripping.  If you want to do local ripping you need to comment out the line with CLI(CLI_exec /sacd_extract_160 -S &) in your AutoScript (and possibly AutoScript.TSS).

[mindset]

A little off from @wl_lam's current direction, but for flexibility I feel it is better to always run sacd_extract_160 in the server mode, which can actually be used for local ripping as well by pointing to 127.0.0.1 (=local host) as the server address like this:

 

sacd_extract_160 -i 127.0.0.1:2002 -I

 

This way, one can do local/remote ripping without changing anything on the player side.  I tried this on my S390, and confirmed working.

 

[mindset]

1 minute ago, Phthalocyanine said:

@mindset

 

This sounds great but what location is it ripped to locally using this method?  Is it ripped to a USB stick or local storage in the player? Please explain further.

 

The output will be saved under the directory sace_extract_160 executed at.  In other words, it is no different than the conventional local ripping method.

[mindset]

4 minutes ago, Phthalocyanine said:

If you used it as part of a script on a USB drive I suppose you could set it to automatically rip to the drive without having to use telnet.

You would have to insert the SACD first, then do the music setting toggle, then insert the USB drive with the script.

(I think someone had proposed something like this a while back.)

This method would not be very convenient if you were doing more than one SACD at a time (and there are problems associated with re-inserting a USB drive without rebooting, as we know.)

I might not be understanding your concern correctly, but the following steps should just work for sequential ripping. 

 

1. Insert a USB stick & let it strat sacd_extract_160 in the server mode

2. Insert a disc & do your trick

3. run sacd_extract_160 -i 127.0.0.1 -I

4. Remove the disc 

5. Go to 2. 

 

OR 

 

1. Insert a USB stick & let it strat sacd_extract_160 in the server mode (if not already running)

2. Insert a disc & do your trick

3. run sacd_extract_160 -i 127.0.0.1 -I

4. Remove the disc 

5. Remove the USB stick

6. Go to 1.

 

17 minutes ago, Phthalocyanine said:

 

@mindset

Is there no tricky way to trick sacd_extract to rip the .iso to a directory other than the one sacd_extract is in?

The idea is to have sacd_extract in permanent memory in the player and then rip to an attached USB drive.

Just cd to the directory you want save the result at and and run sacd_extract_160 from there by specifying it with its absolute path (like /sacd_extract_160 or /3rd/sacd_extract_160 depending on where it is).

 

[mindset]

27 minutes ago, Phthalocyanine said:

If you used it as part of a script on a USB drive I suppose you could set it to automatically rip to the drive without having to use telnet.

You would have to insert the SACD first, then do the music setting toggle, then insert the USB drive with the script.

(I think someone had proposed something like this a while back.)

This method would not be very convenient if you were doing more than one SACD at a time (and there are problems associated with re-inserting a USB drive without rebooting, as we know.)

Oh now I see.  Maybe you are saying that for each USB insertion, one automatic ripping.  The possibility of this is no different for the method I mentioned compared with the conventional local ripping method.

[mindset]

14 minutes ago, Phthalocyanine said:

1.  cd /mnt/sda1

2. mnt/3rd_data/sacd_extract_160  -I.

 

?

 

Pretty much.. except that you need a slash before mnt in 2. like /mnt/3rd_data/sacd_extract_160 -l

[mindset]

1 hour ago, Phthalocyanine said:

For automatic ripping to USB without telnet it seems you would have to remove and re-insert the USB for each new SACD.  (And if the USB drive assignment changes, then the script on the USB drive will no longer work, unless one uses one of your tricky scripts.)

 

Automatic ripping is possible.  I just tried it and it works (insert a USB stick & ripping automatically starts) with some script changes with a caveat that there is no easy way to assign a unique .iso file name for sequential ripping because of the special character issue.  I thought I would use a time stamp as a .iso filename but the player is missing the essential command (=date) for that because it is carrying a very light version of Linux.  There is still a way, but it's not worth my effort.

 

In my opinion, when someone has come to the point to do sequential ripping, remote ripping is the way to go.  It is some learning curve for some people who are not familiar with networking/Linux but is worth the learning effort because:

 

1. It doesn't involve telnet

2. Sleep mode technique works (just a matter of pressing a power button instead of navigating through menus with a  remote control to switch the DSD playback mode)

3. There is no need to physically retrieve the USB stick after ripping

4. There is no filename special character issue

 

I do this on regular basis for sequential ripping:

 

1. Start sacd_extract_160 in the server mode in whatever way

2. Insert a disc

3. Press the power button to put the player in sleep

4. Run sacd_extract -i <player_address>:<port> -I on a remote PC

5. Eject the disc

6. Go to 2.

 

Note: wireless (on the player side) may be slow.  With wired connection, remote ripping is as fast as local ripping.

 

[mindset]

9 hours ago, BluRay444 said:

The file was fairly large, and I decided to do some rips to multichannel only individual files to save some space, and was surprised to find that this had the opposite effect- the sum of the individual files for each disc were roughly twice the size of the ISOs. Interesting. So now I'm back to ISOs. Hopefully jRiver is configurable so that if I connect another instance of jRiver to it on the LAN, it will be able to stream the multichannel or stereo version based on the remote jRiver instance system's audio capabilities.

I believe you ended up with larger files than the original .iso because you dumped the multichannel tracks to dsf or didiff with DST decompression.  Audio data on SACD is normally (especially for those with multi channel tracks) compressed by DST with a nominal compression rate of around 1/3.   If you want to keep the data compressed, dsdiff is the only option as dsf format doesn't allow compressed audio.

 

9 hours ago, BluRay444 said:

Any advice, thoughts, etc about integrating SACD ISOs into the jRiver infrastructure will be appreciated

I believe there is a better forum for this kind of question. 

[mindset]

47 minutes ago, emiliocb said:

And that small possibility of error in the SACD, is even less with one player better than another, or is it a random issue that does not depend on the player?

The foundation of SACD is DVD which uses much more reliable error detection and correction than those of CD, so chances of getting non-bit perfect rip is "practically" zero "if" ripping succeeds.  In other words, if a disc has uncorrectable defects, that will be much more likely be detected than CDs (without relying on external check sum sources).  Therefore, unless players are messing with the read data, there should not be any difference between players.

 

Some people have mentioned file system in SACD, but the way the album is separated into track is similar to that of Audio CD.  The entire audio of an album is linearly recorded (which is essential for smooth read operation), and the TOC at the beginning of the disc contains pointers (offsets) to tracks.

 

[mindset]

5 hours ago, mansr said:

The disc contains a filesystem that is carefully laid out such that the audio data is in one continuous block, same with DVDs. The data on audio CDs is less structured, which is why otherwise identical rips done on different drives can, for instance, have track breaks shifted by some constant amount.

Yes, but this is far from typical "file systems" though its definition is very loose.  There is no concept of files or directories in SACDs while DVDs (even video DVDs) do.  I agree that data on SACDs data is better structured and ripping repeatability is much higher though.