I have started my own journey into Backblaze storage

[Rsmaximasr]

Yes, I have hands on experience with IBM SSA disks, to EMC Symmetrix systems, Netapp filer systems, LSI storage subsystems, and my last 10 years was designing/implementing and testing enterprise solid state storage technologies into some of the biggest enterprises in the world. 
 

My last 2 years, I started working and demonstrating around the country the use of CEPH, which is a unified storage system. If you’re into technology, look into CEPH, which CEPH users will also state you don’t need NAS or SAN. 
 

Do you know why SAN and NAS were invented? Back in the early days, disk drives were very expensive and once a hdd was attached to a machine, and it wasn’t being utilized, you were paying a lot of money for nothing. HDDs were isolated islands that couldn’t be utilized. Now, hdds are cheap.

In a perfect world, checksums/buffering systems and other methods would help protect your data. We aren’t in a perfect world especially in home systems. For example, how many connections (usb or  Ethernet) for home systems, hba’s for commercial SAN systems? 100% of home systems use 1 connection to the SAN/NAS/DAS, whereas in commercial environments would have many hba’s or network connections to the storage. I’ve had to do restores from tape/disk using EMC/Netapp/LSI systems. For failures, you can have a failure in the usb/Ethernet/FireWire/fibre connection that corrupts data, the OS could send garbage down the line, or you can be hacked where your data is changed (no checksum will fix this) or a user modifies  data incorrectly (again no checksum will fix this).

I know ZFS, wasn’t my favorite. It has some nice features. I preferred ASM for the last 15 years, which is much faster and had better features. It doesn’t matter what OS you are using, I’ve used most of them (IBM mainframes, Solaris, aix, most enterprise Linux, and OSX, never windows because that was the worst OS and still is.

 

 

 

 

[The Computer Audiophile]

5 minutes ago, Rsmaximasr said:

Yes, I have hands on experience with IBM SSA disks, to EMC Symmetrix systems, Netapp filer systems, LSI storage subsystems, and my last 10 years was designing/implementing and testing enterprise solid state storage technologies into some of the biggest enterprises in the world. 
 

My last 2 years, I started working and demonstrating around the country the use of CEPH, which is a unified storage system. If you’re into technology, look into CEPH, which CEPH users will also state you don’t need NAS or SAN. 
 

Do you know why SAN and NAS were invented? Back in the early days, disk drives were very expensive and once a hdd was attached to a machine, and it wasn’t being utilized, you were paying a lot of money for nothing. HDDs were isolated islands that couldn’t be utilized. Now, hdds are cheap.

In a perfect world, checksums/buffering systems and other methods would help protect your data. We aren’t in a perfect world especially in home systems. For example, how many connections (usb or  Ethernet) for home systems, hba’s for commercial SAN systems? 100% of home systems use 1 connection to the SAN/NAS/DAS, whereas in commercial environments would have many hba’s or network connections to the storage. I’ve had to do restores from tape/disk using EMC/Netapp/LSI systems. For failures, you can have a failure in the usb/Ethernet/FireWire/fibre connection that corrupts data, the OS could send garbage down the line, or you can be hacked where your data is changed (no checksum will fix this) or a user modifies  data incorrectly (again no checksum will fix this).

I know ZFS, wasn’t my favorite. It has some nice features. I preferred ASM for the last 15 years, which is much faster and had better features. It doesn’t matter what OS you are using, I’ve used most of them (IBM mainframes, Solaris, aix, most enterprise Linux, and OSX, never windows because that was the worst OS and still is.

 

 

 

 

Hi @Rsmaximasr, you’re bring back some memories now :~)

 

I setup a Symmetrix system to asynchronously mirror data across the country, as one of my last projects before leaving the corporate world. 
 

I’m still struggling to see how you propose we solve our needs using DAS. I know it can be done, but it’s unpleasant as heck in my opinion, compared to a Linux appliance built for this purpose. 
 

- Many machines accessing music at a time

- Simple drive replacement when one or more fail

- Music access from outside the network

- Encrypted offsite backup (automated)

- Backup to USB disks (automated)

- SFTP access from outside

 

[Rsmaximasr]

I have many endpoints and also use Roon ARC for streaming to the cars. Remember, music data is small. I’m used to building large databases with thousands of concurrent users, now that’s real data throughput and had all hba’s in a large AIX server saturated. I also setup CEPH over 7 servers with 100Gb networks and saturated the network using MySQL. You can stream to 20 concurrent  endpoints with Roon  and you might get 1 or 2% disk utilization, half of 1 rotation of the hdd can read a whole album into memory. DAS access is the fastest without the overhead of RAID. RAID 5/6 is terribly slow on writes, ok for reads. RAID 1 or 10 is expensive and can be as fast as DAS. 
You can build a RAID 1 using multiple drives on any apple computer. You can purchase a cheap jbod from owc with or without softraid to handle RAID 0 thru 5. 
If using Linux use mdadm over a bunch of das disks. 
All I’m saying is that there are many options out there. I loved ASM which can be implemented over DAS, the best feature of EMC were bcv’s. Check out CEPH.

[AudioDoctor]

32 minutes ago, Rsmaximasr said:

I have many endpoints and also use Roon ARC for streaming to the cars. Remember, music data is small. I’m used to building large databases with thousands of concurrent users, now that’s real data throughput and had all hba’s in a large AIX server saturated. I also setup CEPH over 7 servers with 100Gb networks and saturated the network using MySQL. You can stream to 20 concurrent  endpoints with Roon  and you might get 1 or 2% disk utilization, half of 1 rotation of the hdd can read a whole album into memory. DAS access is the fastest without the overhead of RAID. RAID 5/6 is terribly slow on writes, ok for reads. RAID 1 or 10 is expensive and can be as fast as DAS. 
You can build a RAID 1 using multiple drives on any apple computer. You can purchase a cheap jbod from owc with or without softraid to handle RAID 0 thru 5. 
If using Linux use mdadm over a bunch of das disks. 
All I’m saying is that there are many options out there. I loved ASM which can be implemented over DAS, the best feature of EMC were bcv’s. Check out CEPH.

 

Perhaps you don't realize that I have no desire to learn what all those acronyms are much less how to use them. This is not my job this is my hobby for fun and I want to keep it that way.

[AudioDoctor]

5 hours ago, Jud said:

What would motivate me to use a NAS?

 

I have no idea Jud, we all make these decisions for ourselves and the only person I can speak for is myself.

[The Computer Audiophile]

1 hour ago, Rsmaximasr said:

I have many endpoints and also use Roon ARC for streaming to the cars. Remember, music data is small. I’m used to building large databases with thousands of concurrent users, now that’s real data throughput and had all hba’s in a large AIX server saturated. I also setup CEPH over 7 servers with 100Gb networks and saturated the network using MySQL. You can stream to 20 concurrent  endpoints with Roon  and you might get 1 or 2% disk utilization, half of 1 rotation of the hdd can read a whole album into memory. DAS access is the fastest without the overhead of RAID. RAID 5/6 is terribly slow on writes, ok for reads. RAID 1 or 10 is expensive and can be as fast as DAS. 
You can build a RAID 1 using multiple drives on any apple computer. You can purchase a cheap jbod from owc with or without softraid to handle RAID 0 thru 5. 
If using Linux use mdadm over a bunch of das disks. 
All I’m saying is that there are many options out there. I loved ASM which can be implemented over DAS, the best feature of EMC were bcv’s. Check out CEPH.

 

I think we are on different wavelengths. None of us are concerned about throughput to multiple devices. We are concerned with access for multiple devices, which means everything from UPnP to SMB to SFTP to Roon, etc...

 

it sounds like you've setup what's effectively a NAS, but gone through the trouble to recreate the wheel. Describing how little throughput music actually uses, then talking about how slow RAID 5/6 are, is a bit strange, but again, I think we are on different wavelengths. 

 

A purpose built linux appliance that can do everything one needs versus a kludge desktop with all those pieces added on, doesn't seem like much of a content for me. I'll take the NAS every time. Unless shown why I shouldn't. 

[Jud]

1 hour ago, AudioDoctor said:

 

I have no idea Jud, we all make these decisions for ourselves and the only person I can speak for is myself.

I did go on to answer that rhetorical question. 🙂 Two reasons, first because I want something that is purpose built to stay running 24/7 (my current desktop isn’t, so I have to remember to turn it on when I want remote access to my local music collection); and second because I might get some geeky fun out of building a NAS if I decided to go that route instead of buying.

[AudioDoctor]

22 minutes ago, Jud said:

I did go on to answer that rhetorical question. 🙂 Two reasons, first because I want something that is purpose built to stay running 24/7 (my current desktop isn’t, so I have to remember to turn it on when I want remote access to my local music collection); and second because I might get some geeky fun out of building a NAS if I decided to go that route instead of buying.

 

In my case, I can split the cost of two identical NAS systems and Backblaze 3 ways between business, myself, and my brother, who all share in the expense and use of the NAS.

[Jud]

2 hours ago, AudioDoctor said:

 

In my case, I can split the cost of two identical NAS systems and Backblaze 3 ways between business, myself, and my brother, who all share in the expense and use of the NAS.

I liked B2 when I had it, so just trying to figure out whether it would be easier to get you to hire me or adopt me.

[Jud]

3 hours ago, The Computer Audiophile said:

None of us are concerned about throughput to multiple devices.

That seems to be one of the two key distinctions in my reading between Ceph and ZFS. Ceph would appear to work better with a system distributed across multiple nodes (usually servers).

 

The other distinction is that Ceph is more configurable, but by the same token that means more complexity.

 

I personally think I would be happier with a single server and easier configuration.

[AudioDoctor]

1 hour ago, Jud said:

That seems to be one of the two key distinctions in my reading between Ceph and ZFS. Ceph would appear to work better with a system distributed across multiple nodes (usually servers).

 

The other distinction is that Ceph is more configurable, but by the same token that means more complexity.

 

I personally think I would be happier with a single server and easier configuration.

 

TrueNAS makes it stupidly simple to do this stuff with TrueNAS Scale and TrueCommand

 

https://www.truenas.com/truecommand/

 

edit: I did pay for a year of 24/7 phone support so that if I have questions I can get help and hopefully learn enough from that year that I won't need it anymore.

[Dan Gravell]

I'm currently using Wasabi - works out well. Used rclone to upload my library and it worked pretty seamlessly.

[AudioDoctor]

in an estimated 3 hours my Backblaze backup will be complete... Finally.

[Jud]

On 8/14/2023 at 5:40 PM, AudioDoctor said:

 

TrueNAS makes it stupidly simple to do this stuff with TrueNAS Scale and TrueCommand

 

https://www.truenas.com/truecommand/

 

edit: I did pay for a year of 24/7 phone support so that if I have questions I can get help and hopefully learn enough from that year that I won't need it anymore.

 

Scale is relatively new. Anything you've noticed that's a missing capability you'd like to see, or that's not completely solid?

[AudioDoctor]

1 minute ago, Jud said:

 

Scale is relatively new. Anything you've noticed that's a missing capability you'd like to see, or that's not completely solid?

 

I am probably the worst person to ask, but no. Everything I want to do, short of running Minimserver on it, it does.

[The Computer Audiophile]

On 8/14/2023 at 6:40 PM, AudioDoctor said:

edit: I did pay for a year of 24/7 phone support so that if I have questions I can get help and hopefully learn enough from that year that I won't need it anymore.

That by itself is a huge selling point. When I have a QNAP issue, support is good but it can take a week or more to get it resolved via messaging. 

[Rsmaximasr]

If you use Apple products, you will be using APFS and truenas had issues with this a while back. Also you read in these posts that you might get support within a week for a freeware app, that’s uncalled for.

For home use, and if I needed to use RAID, I’d use Apples built in disk utilities to build a RAID 0,1,10,01 or jbod. If I need anything more robust, I’d buy an older Mac Pro and get the pcie raid card. Or just run virtualbox with Linux running mdadm.

[AudioDoctor]

5 hours ago, The Computer Audiophile said:

That by itself is a huge selling point. When I have a QNAP issue, support is good but it can take a week or more to get it resolved via messaging. 

 

I am sure I am the smallest of potatoes to IxSystems, creators of TrueNAS and the hardware they sell, but they never make me feel like it and are super helpful.

[AudioDoctor]

Just now, Rsmaximasr said:

If you use Apple products, you will be using APFS and truenas had issues with this a while back. Also you read in these posts that you might get support within a week for a freeware app, that’s uncalled for.

For home use, and if I needed to use RAID, I’d use Apples built in disk utilities to build a RAID 0,1,10,01 or jbod. If I need anything more robust, I’d buy an older Mac Pro and get the pcie raid card. Or just run virtualbox with Linux running mdadm.

 

I have my time machine backup living on my TrueNAS backup without problems.

[AudioDoctor]

Backblaze is increasing prices effective October 3rd.

 

https://www.backblaze.com/blog/2023-product-announcement/

 

edit: Taking your data from Backblaze and moving to a new provider is now free. The increase is 1$/TB/month for B2 and 9$/month for personal, up from 7$. Click the link if you want more specifics, it's not a huge increase.

 

edit 2: This is important for Personal backups as well

Free One Year Extended Version History: Also effective October 3, all Computer Backup licenses may add One Year Extended Version History, previously a $2 per month expense, for free. Being able to recover deleted or altered files up to a year later saves Computer Backup users from huge headaches, and now this benefit is available to all subscribers. Starting October 3, log in to your account and select One Year of Extended Version History for free. 

[Jud]

32 minutes ago, AudioDoctor said:

Backblaze is increasing prices effective October 3rd.

 

https://www.backblaze.com/blog/2023-product-announcement/

 

edit: Taking your data from Backblaze and moving to a new provider is now free. The increase is 1$/TB/month for B2 and 9$/month for personal, up from 7$. Click the link if you want more specifics, it's not a huge increase.

 

edit 2: This is important for Personal backups as well

Free One Year Extended Version History: Also effective October 3, all Computer Backup licenses may add One Year Extended Version History, previously a $2 per month expense, for free. Being able to recover deleted or altered files up to a year later saves Computer Backup users from huge headaches, and now this benefit is available to all subscribers. Starting October 3, log in to your account and select One Year of Extended Version History for free. 

 

This makes B2 very close to the same price as  Computer Backup (personal unlimited) for me, and with free data egress from B2 it might even be cheaper in the event I lost my data at home. So I'll likely go back to B2. I prefer B2's greater availability of tools for moving data and checking its status at Backblaze, and the fact that it works with all my OSs (personal is either just Windows or just Windows and MacOS, I forget which).

 

[AudioDoctor]

49 minutes ago, Jud said:

 

This makes B2 very close to the same price as  Computer Backup (personal unlimited) for me, and with free data egress from B2 it might even be cheaper in the event I lost my data at home. So I'll likely go back to B2. I prefer B2's greater availability of tools for moving data and checking its status at Backblaze, and the fact that it works with all my OSs (personal is either just Windows or just Windows and MacOS, I forget which).

 

 Personal works with Mac OS, I know that for a fact.

[Dan Gravell]

The move to free (ish) egress in B2 is positive - it makes the storage much more flexible.

[AudioDoctor]

Using Forklift I can use an Applet to sync my mounted NAS Music Share to Backblaze with a few mouse clicks. I do this when I add music to my NAS.

 

 

 

 

[AudioDoctor]

What I uploaded was this album which I was surprised to not already own in CD quality for 10 GBP, about 12+ USD, from Bandcamp.