[Solved] "System" is using lots of Disk Space on Mojave and High Sierra
Like @johnfbraun mentioned in a recent episode, I've now started seeing "System" using over 100GB on my APFS-formatted High Sierra and Mojave systems. I've been running all kinds of utilities to find the answer, to little/no avail.
This morning, with DaisyDisk, I realized that my Drive Genius logs were taking up ~20GB. Those were mostly in ~/Library/Logs/com_prosofteng_logs/DriveGenius/dp. I don't launch Drive Genius often these days, so I assume these are from its Drive Pulse app (which I'm not keen on anyway). I opened that folder and saw a few very large .tlg files in there. Before just manually deleting them, I launched Drive Genius... and watched it wipe them away.
Perhaps Drive Genius is built to "process" these, but only after launch?
Anyway, that's one small dent in my System Storage usage (I still have 152GB used there). What have you found that helps?
I would run though what this blog post says and see if it helps:
Thanks, @brianmonroe. I had done quite a few of those things, but not all.
I really think DaisyDisk is better for this than OmniDiskSweeper, as much as I'm surprised to see myself type that. When doing "Scan as Administrator" DaisyDisk actually showed me numbers that match what Apple shows: nothing is hidden or missing. I'm a little sad about that because I thought maybe there was 100GB just waiting to be deleted but, alas, no... I'm really filling up my drive.
Things like iCloud Photo Library's cache at 25GB (which shouldn't cache since I store my photo library on an external drive!), Chrome's "File System" at 15GB, and my 50GB Downloads folder (oops!) are taking up space, for sure.
Glad to heat that the mystery has been solved. It sounds like you need a much larger internal/boot SSD and to change your default downloads folder for you various browsers you use.along with your Mail Downloads Folder:
As I have found many people forget that folder exists and it does get cluttered up with all sorts of junk that normally can be deleted.
@brianmonroe — I wish it were just my Mail Downloads folder. That one is pretty small... it's my ~30-year email archive that eats most of that space! Maybe it's time to consider MailSteward, but I really am concerned about long-term access when moving my mail out of my "Mail", if that makes sense.
Interestingly, here are two screenshots, one from DaisyDisk and the other from OmniDiskSweeper. Same drive, completely different results. DD shows 348GB for "Users" while OmniDiskSweeper shows 247GB. And it's not a different user account, digging in on both shows my user account as 347.1GB and 246.8GB, respectively. Insane.
Both were run in "Administrator" mode so should "see" everything and yet... the proof is in the pictures.
In looking a little deeper at ODS vs. DD (albeit still quickly), the issue seems to be a discrepancy between them in the size of my ~/Library folder. And, looking in there, seems ODS doesn't see my ~/Library/Mail folder at all. I have both apps added to Mojave's "Full Disk Access" list but obviously something's missing.
My guess is that they're using different engines/methods of determining usage, perhaps both with du albeit different permissions. Regardless, seems like even running ODS in sudo mode and with Full Disk Access enabled still doesn't help it keep from missing a lot of stuff.
Honestly, my guess is that du would also fail at this. So I did some testing.
du -sh ~/Library/Mail results in "du: Mail: Operation not permitted", so I tried:
sudo du -sh Mail and entered my password.
Same rejection, so I added Terminal to the Full Disk Access list and then it worked. Seems whatever ODS is doing isn't actually taking advantage of Full Disk Access, even though the app is there. ¯\_(ツ)_/¯ Actually, maybe now that I've added Terminal to the FDA list, maybe ODS will work properly. Running a test. I'll report back.
In watching my Terminal session that controls ODS, seems that lsbom is another tool it uses. Haven't seen du show up, but it might be happening too quickly to see.
That's exactly what I was seeing as well when I ran the du command on my user/home directory and I am also in agreement that there is a good chance that the developers of DD and ODS have decided to not present error messages when they do not have permissions to access files/folders as I am sure that they did not want to get lots of tech support calls dealing with users who would not understand.
I am sure that both DD and ODS are using different methods as your tests have proven or they would both line up as they are both looking at the same source for their data.
I am curious if the free DiskInventory X would give you a more accurate number assuming you gave it full disk access?
While this is fun to get sorted out, I still think the smart move at this point is to pickup a larger SSD to swap out your internal drive with as 500GB is just not enough for power users like you. Depending on how long you are planning on keeping your system you could look at a 1TB or a 2TB SSD as they both have come down in price a lot.
Ok. I finished some testing here.
DaisyDisk is definitely the simplest one to run in terms of seeing everything. It alerts you if it needs Full Disk Access, and can scan all your files with "Scan as Administrator" without having to use the Terminal/sudo trick.
Omni Disk Sweeper will warn you if you need to add it to Full Disk Access, but it won't scan files belonging to other users on its own. You can use the Terminal/sudo trick, but if you do that then ODS doesn't know to warn you because it's running "inside" Terminal, and requires you to add Terminal to Full Disk Access in Mojave.
In the end, with both configured properly, they'll both see everything, though.
Yes. you do "need" a larger SSD. Just because you are a power user/Dave The Nerd 🤓 and can work around these issues, I just know that having dealt with these things in the past you are making a lot more work for yourself in the future by using working around the issue versus just getting more internal storage for your files.
Yes. you can manually direct iTunes, Photos, Mail and other apps to use other drives for their storage but soon you will come up against a wall that can not be worked around easily or at all as not all applications are well built and many I have had to deal with on both macOS and Windows assume your main partition is where your files live. Now I know we are taking macOS here but still the same lazy devs exist that make those assumptions and will continue to do so.
Plus, I assume the time you are spending now and have spent in the past working around the limitations has a cost that is more than the cost of the storage. For my clients I have been putting in 1TB SSD's to replace their 1TB HDD's even if they are not yet using the storage as I know that I can always take those drives out in the future when the system I upgraded gets retired and either put the drive in an external enclosure or put it in to a newer system later on.
I am glad you got a clear picture of what is going on with your storage as it is important that no matter what you ultimately decide to do now and the future you need to know if your space issues are because of extra files you want to keep around or cruft that has been generated over the years either by the system or because of extra software was installed that you do not need or can do without.
Good to know that DD is the better option as long as you give it the correct permissions.
Yes. While all of that is true. It is also true for people who have laptops like MacBook Air/Pro computers, they will (in many cases) need to still think about purchasing as much internal local storage as they need or better yet, just use the built in optimize storage options with macOS and iCloud to not worry about it as using external storage for your main files with a laptop is normally a non starter as it restricts the portability of the device as every time you want to take it anyplace, you have to bring your drive(s) with you. That can become a pain to deal with.