APFS in High Sierra: Disappearing disk space WTF?

[wgscott]

I just got my iMac up and running again (having replaced Apple's defective power supply and nightmare 1TB fusion drive with third-party alternatives), so I formatted my new internal SSD with APFS.  I restored from my "time machine" backup, and filled about 91% of the new 500GB disk.  I decided to move some stuff out of the way, so I deleted about 130 GB of files.  The free disk space goes down about 2 GB.  WTF?

 

So then I did a little bit of internet checking, and it turns out with 10.13 and APFS, Apple implemented non-optional local backups.  I like to do my "time machine" backups once a day, when I feel like it. So I turned it off.  Apple fascistically decided it is for my own good to fill my hard drive with this junk.  You can delete it, but it comes back every hour.

 

To see if you have any of these soaking up hard drive space, issue this command:

 

sudo tmutil listlocalsnapshots /

In my case it returned a single such snapshot:

 

com.apple.TimeMachine.2018-01-31-191554

 

I then deleted it with the following command (note the syntax):

 

sudo tmutil deletelocalsnapshots 2018-01-31-191554

 

Now instead of 91% occupancy, I have 50% free space.

 

Until Apple decides to stick it to me again.

 

They removed the tmutil option of turning this off.  

 

Anyone have a workaround?  This is just nuts.  I wonder if all them folks who like to turn off "extra" processes in OS X have a problem with this?  Nothing like writing several hundred gigs of data to a disk to slow stuff down and potentially create noise.

[kirkmc]

First, you need to know that the Finder's available space is not the amount of free space on a drive; it's the amount of free space if things were deleted that could be deleted. 

 

Go to Apple Menu > About This Mac > Storage, Look at your startup drive. You'll see an Available amount, but this is the amount available if any optimization features were active, such as auto-deleting iTunes videos, the Trash, moving files to iCloud, etc. 

 

If you run df in Terminal (when at /), you'll see the amount of storage used by each of your discs, and you'll see your local TM snapshots. (One thing I don't understand is why those snapshots exist on my iMac, which has a TM drive connected to it; I thought they used to only be on laptops, or computers whose TM disk wasn't present.) (Also, tmutil shows 24 local snapshots; df only shows six.)

 

For my 250 GB startup SSD, it shows this:

 

Filesystem                                                     512-blocks        Used      Available Capacity

/dev/disk1s1                                           488555312   410369808   70587208    86%

 

It says that the disk is 86% full. However, Finder shows it as having about 62 GB free, or about 25%. 

 

But those snapshots don't necessarily take up any space, because of hard links. So deleting snapshots may not free up space, which is fictional anyway. I deleted six of mine, and it only freed up 2 GB. (And I've been working on a video project yesterday and today, so it wouldn't surprise me if there were changed files that big that would be different in the various snapshots.)

 

It sounds like in your case you had a huge snapshot, with files that weren't just hard links. Or that something is weird on your disk. It's hard to know. 

[kirkmc]

In the Apple support document you link to, it says:

 

"That's why Finder and Get Info windows don't include local snapshots in their calculations of the storage space available on a disk."

[wgscott]

I used df -h  to keep track of disk usage.  The single backup file did not appear in the output of df.  The disk usage as displayed in disk utility agreed with df (probably just a gui display of df output).  I confess I didn't look at Finder. (I initially suspected file corruption, like unliked inodes, but disk util said I was ok.)

 

I had 91% disk usage after my restoration (I restored to a disk half the original's size).  Then I pruned it by removing directories containing about 180 GB of data.  df  reported only about 2GB difference.  So I think the huge backup was made when I deleted those directories (the unix way).  Presumably it keeps the inode and just hides it from the user, and the snapshot is some sort of reference to that, because the whole thing was instantaneous.  When I deleted the snapshot, it took awhile to recover all that disk space, so I think only then were the inodes being deleted.  I am going to play around a bit and see if this also happens deleting big files via Finder; maybe this is a unix level (rm -rf ) thing?

[kirkmc]

Yes, I think there may be a lag, in part because the snapshots probably contain hundreds of thousands of files. There is probably some sort of error checking that occurs, rather than just wiping them immediately. 

 

I would not, however, mess with rm -rf; that's asking for trouble. :-) 

 

Don''t forget that some of the space may be taken up by VM files. 

[wgscott]

I have about 24 GB that are still "purgeable" but no reason to mess with that.

 

rm -rf   has been my friend for almost 30 years.  

 

On the unix level, what normally happens when you use rm is that it unlinks the name of the file to the inode, but keeps the inode, so it is hidden. (Under some circumstances you can undo the rm command, as long as the inode in question doesn't get over-written.)  I think all Apple is doing is making those deleted inode links recoverable using time machine.

[wgscott]

Today I cannot repeat this (either via command-line or Finder).  I am back to thinking it is a bug, rather than a feature.  In any case, should anyone run into this, the above commands in the OP allow one to recover the disk space.

[Ralf11]

did you report it?

[wgscott]

No.  They will just blame it on using a Samsung SSD instead of using their crappy fusion drive.

[HowieBeck]

This thread is leaving me hopeful... I've been on with 'tech support' with someone at Apple for weeks, and they cant solve this for me. I'm new to using terminal, but I'm having to download files and programs now, as my disk is full, and is devouring it's own hard drive space constantly. Could someone please let me know how to do these commands without messing anything up?? I'm losin it. Thanks. HB

[wgscott]

@HowieBeck

 

Sorry, I only saw this now.  Did your issue ever get resolved?

 

If not, open up /Applications/Utilities/Terminal.app

 

 

Open up a window, and type in the following command:

 

sudo tmutil listlocalsnapshots /

It should list one or more local backups, if this is indeed the problem.  If it does, just use the syntax described in the first post.

 

These also get created to make scheduled time machine backups go faster, but in that case you should only have one, not a whole pile of them.

[diecaster]

Here is a shell script to automate the deletion process:

 

#!/bin/bash

/usr/bin/tmutil  listlocalsnapshotdates / |grep 20|while read f; do tmutil deletelocalsnapshots $f; done
echo "Done!"
 

 

[wgscott]

I'd pipe through

grep -v Snapshot

 

[diecaster]

2 hours ago, wgscott said:

I'd pipe through

grep -v Snapshot

 

 

Why??

[wgscott]

Why grep the 20?

 

Why give the full path to tmutil in the first command but not the second?

 

Why not use "sudo tmutil" in the delete command so that it actually works?

 

Why copy/paste things from the internet without testing them yourself?

[diecaster]

I did test it...and it works perfectly and has for months.

[diecaster]

Actually this works great too since "/usr/bin" is always in the list of paths the shell will search for commands:

 

#!/bin/bash

tmutil listlocalsnapshotdates / |grep 20|while read f; do tmutil deletelocalsnapshots $f; done
echo "Done!"

 

You don't need "sudo" because the "tmutil" verbs "listlocalsnapshotdates" and "deletelocalsnapshots" do not require root privileges.

 

Why not "grep 20"?

[wgscott]

You are absolutely right.  Sorry.  I could have sworn it required a sudo when I deleted it a month ago...

 

grep 20 will fail in 2100  ;-)