76

I want to delete some files/directories from my Time Machine Partition using rm, but am unable to do so. I'm pretty sure the problem is related to some sort of access control extended attributes on files in the backup, but do not know how to override/disable them in order to get rm to work. An example of the error I'm getting is:

% sudo rm -rf Backups.backupdb/MacBook/Latest/MacBook/somedir
rm: Backups.backupdb/MacBook/Latest/MacBook/somedir: Directory not empty
rm: Backups.backupdb/MacBook/Latest/MacBook/somedir/somefile: Operation not permitted

There are a number of reasons I do not want to use either the Time Machine GUI or Finder for this. If possible, I'd like to be able to maintain the extended protection for all other files (I'd like not to disable them globally, unless I can re-enable once I've done my work).

2
  • I've gotten closer. First, I needed to disable ACLs on my Time Machine disk. Previously, one would use fsaclctl to do this, but Snow Leopard does not include this. I grabbed a copy of the binary from an older version of OSX and ran this: % sudo fsaclctl -p /Volumes/tmvol -d I then tried to use "sudo rm -rf" to remove a directory, but still ran into issues with some files (though others went away fine). Specifically, it failed on soft links. Very strange. On the plus side, the links take up almost no space. On the negative side, I still have the directories sticking around.
    – Tim
    Jul 14, 2010 at 16:26
  • It seems that the accepted answer is quite dangerous when deleting snapshots, or folders in snapshots, like your rm -r Backups.backupdb/MacBook/Latest/MacBook/somedir; if you agree, then please accept Arne's answer?
    – Arjan
    Aug 26, 2015 at 16:44

9 Answers 9

130

To work around "operation not permitted" errors, use the Time Machine Safety Net "bypass" program:

sudo /System/Library/Extensions/TMSafetyNet.kext/Contents/MacOS/bypass rm -rfv /Volumes/[disk]/Backups.backupdb/[path]

In 10.8 Mountain Lion, bypass moved into 'Helpers':

/System/Library/Extensions/TMSafetyNet.kext/Helpers/bypass

In 10.10 Yosemite, bypass moved here:

/System/Library/Extensions/TMSafetyNet.kext/Contents/Helpers/bypass

Beware when using this to delete specific snapshots: as Time Machine uses hard links, using rm -r on folders might also affect older and newer snapshots of the same machine. (See other answers referring to tmutil delete to safely delete a specific snapshot.) Using rm to delete all snapshots for a single machine is okay though. And so is using rm to delete a specific file, which will only remove that hard-linked file from the snapshot(s) you specify, assuming the file is not in a hard-linked directory as then you'd actually remove the file from all those hard-linked directories.

12
  • +1!! This helped me. I didn't even try the others because this looked like "the right way" and indeed, it allowed me to resolve the issue with no further questions. Thanks!
    – tripleee
    Feb 28, 2012 at 13:59
  • I created an account just to +1 you. I have been looking for a better solution to this, and this was it. It had been driving me crazy. Thank you.
    – CWSpear
    May 13, 2012 at 21:46
  • 3
    Awesome. This works. (although I leave out the 'v' option to avoid seeing an entry for each of a few hundred thousand files) i.e.: sudo ...bypass rm -rf /Volumes/... Nov 13, 2012 at 1:54
  • 7
    This is an extremely dangerous method for managing Time Machine files; Time Machine uses hard-linked directories to reference folders that haven't changed since the previous backup, however the rm process doesn't understand these and will follow these links and remove files from them. This means you could potentially delete files that are also in older and newer backups than the one that you selected, potentially causing irreparable damage to the backup(s). Arne Stenström's recommendation of using tmutil is by far the superior solution.
    – Haravikk
    Dec 3, 2013 at 18:59
  • 2
    To extend on Haravikk's comment a bit: using rm for hard-linked files is okay, but for hard-linked folders it's not. Kent's answer mentions the same problem. And in What is the Unix command to create a hardlink to a directory in OS X? someone wrote for 10.5 in 2010: "Deleting is a different story: if you go about it the usual way to delete directories, you'll delete the contents. So you must "unlink" the directory: unlink new_hard_link". So only use this to delete all backups (snapshots) for a given machine.
    – Arjan
    Aug 26, 2015 at 16:24
30

BLUF (bottom line up-front):

sudo tmutil delete _snapshot-dir_

Using sudo [chmod][chmod(1)] -R -N _folder_ to remove all ACLs from a folder hierarchy doesn't work on the files and folders in Time Machine's Backups.backupdb, because of the TM Safety Net mechanism and the criteria described in this 318 Tech Journal post (but possibly not exactly as described).

(Before learning this from looking up the Safety Net mentioned in Eric W's answer (which works), I had only tested on a folder cloned from a subfolder of a TM backup, and there chmod worked. But trying chmod on a folder in an actual TM backup gives the "Operation not permitted" error.)

Of possible use:

In Mac OS 10.7+, there's a tmutil command (which I haven't tried, since I'm still on Snow Leopard). It has a delete verb, which according to the description "can delete snapshots from backups that were not made by, or are not claimed by, the current machine" (where a "snapshot" is a dated folder representing a single incremental backup). It isn't clear to me if this means that it can't delete snapshots which are made by or claimed by the current machine.(?)

6
  • 2
    Indeed, tmutil is useful, and will allow you to delete backups (tmutil delete /Volumes/DISK/Backups.backupdb/HOST/DATE_FOLDER). However, you still need Eric's bypass trick to delete the "Backups.backupdb" folder itself.
    – mivk
    Jun 30, 2012 at 13:27
  • 1
    My OSX 10.8.3 (Mountain Lion) did not require bypass. Just sudo tmutil delete <snapshot-dir>. The popular bypass rm answer is outdated.
    – John Mee
    May 9, 2013 at 22:33
  • Is it possible to use tmutil to delete just some files from a snapshot? It didn't work for me (Invalid deletion target (error 22)) so I went with bypass instead. Dec 6, 2014 at 14:39
  • +1 for BLUF. Did you get that from (the totally awesome) Manager-Tools? :)
    – Olie
    Dec 4, 2015 at 20:18
  • Also: I keep finding references to a Backups.backupdb directory that I'm not seeing on my backup drive (a Time Capsule). I just have things of the form MachineName.sparsebundle. Did the format change? I'm trying to use TM's delete backup command, but it's been stuck on 99.99% progress bar for several hours.
    – Olie
    Dec 4, 2015 at 20:20
16

A warning about using the bypass command to remove an old backup: if the deleted backup has folders that are exactly the same in earlier or later backups, then files might be deleted from earlier or later backups as well!

Time Machine not only uses hard links for unchanged files, but also uses hard links for folders in which no files were added, changed or deleted at all. This results in something like:

/2014-11-06/folder/file1
                  /file2
                  /file3
/2014-11-13/folder/file1 = hard link to file /2014-11-06/folder/file1
                  /file2 (changed; new inode)
                  /file3 = hard link to file /2014-11-06/folder/file3
/2014-11-20/folder/ = hard link to folder /2014-11-13/folder/
/2014-11-27/folder/ = hard link to folder /2014-11-20/folder/

With the above, deleting any file from /2014-11-06/folder/ is fine, and only affects the backup for that date. The hard link reference counts are decreased, so the "inode" for file2 will be removed, but inodes for file1 and file3 will still have a reference count of 1 due to the later backups. Hence, rm -R /2014-11-06 is fine too.

However, removing any file from either /2014-11-13/folder/, /2014-11-20/folder/ or /2014-11-27/folder/ will effectively remove it from all those 3 folders.

The problem is that rm -R does not care about hard-linked folders. It just recurses into any hard-linked folder it finds, boldly deletes all its files, and then removes the empty folder.

So: when removing an old backup one should not recurse into a hard-linked folder and delete its contents. Instead, one should only remove the hard link for the folder itself. So, rather than rm -R use tmutil delete as explained in Arne's answer.

As an aside, it seems that the OS X unlink command cannot be used on folders: "only one argument, which must not be a directory, may be supplied". The OS X API can remove hard-linked folders, and so can GNU Coreutils, like installed using Homebrew.

Finally, to prove all of the above, a test-case (OSX 10.6.8):

sh-3.2# ls -lFa 2014-11*/Users/USERNAME/Library/Safari/TopSites.plist 
-rw-r--r--@ 2 USERNAME  staff  1551 10 30  2014 2014-11-06-012454/Users/USERNAME/Library/Safari/TopSites.plist
-rw-r--r--@ 2 USERNAME  staff  1551 10 30  2014 2014-11-13-024438/Users/USERNAME/Library/Safari/TopSites.plist
-rw-r--r--@ 2 USERNAME  staff  1551 10 30  2014 2014-11-20-014044/Users/USERNAME/Library/Safari/TopSites.plist
-rw-r--r--@ 2 USERNAME  staff  1551 10 30  2014 2014-11-27-025033/Users/USERNAME/Library/Safari/TopSites.plist

Note that the number of links for each occurrence is 2 (second column). Let's remove the first occurrence:

sh-3.2# /System/Library/Extensions/TMSafetyNet.kext/Contents/MacOS/bypass unlink 2014-11-06-012454/Users/USERNAME/Library/Safari/TopSites.plist 
sh-3.2# ls -lFa 2014-11*/Users/USERNAME/Library/Safari/TopSites.plist 
-rw-r--r--@ 1 USERNAME  staff  1551 10 30  2014 2014-11-13-024438/Users/USERNAME/Library/Safari/TopSites.plist
-rw-r--r--@ 1 USERNAME  staff  1551 10 30  2014 2014-11-20-014044/Users/USERNAME/Library/Safari/TopSites.plist
-rw-r--r--@ 1 USERNAME  staff  1551 10 30  2014 2014-11-27-025033/Users/USERNAME/Library/Safari/TopSites.plist

So, after unlinking one of the files, the number of links dropped to 1 for each occurrence, though the file is still shown 3 times. No problems yet. Remove the first occurrence again:

sh-3.2# /System/Library/Extensions/TMSafetyNet.kext/Contents/MacOS/bypass unlink 2014-11-13-024438/Users/USERNAME/Library/Safari/TopSites.plist 
sh-3.2# ls -lFa 2014-11*/Users/USERNAME/Library/Safari/TopSites.plist 
ls: 2014-11*/Users/USERNAME/Library/Safari/TopSites.plist: No such file or directory

Now all are gone. Apparently the file TopSites.plist was last changed 2014-11-06 and hard-linked on 2014-11-13 as then some other files were added, changed or removed in the Safari folder. Next, the contents of the Safari folder did not change in the subsequent two backups, so on 2014-11-20 and 2014-11-27 the Safari folder was hard-linked to the previous backup.

Indeed, the 4 folders only use 2 inodes (first column):

sh-3.2# ls -lFaid 2014-11*/Users/USERNAME/Library/Safari/
648651968 drwxr-xr-x@ 86 USERNAME  staff  2924  9 10 16:06 2014-11-06-012454/Users/USERNAME/Library/Safari//
650804457 drwxr-xr-x@ 86 USERNAME  staff  2924  9 10 16:07 2014-11-13-024438/Users/USERNAME/Library/Safari//
650804457 drwxr-xr-x@ 86 USERNAME  staff  2924  9 10 16:07 2014-11-20-014044/Users/USERNAME/Library/Safari//
650804457 drwxr-xr-x@ 86 USERNAME  staff  2924  9 10 16:07 2014-11-27-025033/Users/USERNAME/Library/Safari//
10
  • 1
    Some background: for files, hard links work as expected; only the hard link you want to delete is removed. Like: touch file1; ln file1 file2; ln file2 file3; ls -li; rm file2; ls -li will only remove a single hard link. But for folders, in What is the Unix command to create a hardlink to a directory in OS X? someone wrote for 10.5 in 2010: "Deleting is a different story: if you go about it the usual way to delete directories, you'll delete the contents. So you must "unlink" the directory: unlink new_hard_link". This is probably still valid.
    – Arjan
    Aug 26, 2015 at 16:39
  • The man page for unlink (in 10.6.8) says that it cannot be used on directories: When the utility is called as unlink, only one argument, which must not be a directory, may be supplied.
    – Kent
    Aug 26, 2015 at 22:31
  • Hmmm, mystery. Even more so, your answer is important: don't use [bypass] rm -r on hard-linked directories. (But I don't have to explain that to you.)
    – Arjan
    Aug 27, 2015 at 7:08
  • I just experimented a little more. Running bypass unlink FILE has the same (unintended?) consequence as bypass rm FILE. The identical FILE is removed from all backups, not just the single location that was specified. And, unlink will NOT take a directory or more than one file as an argument (10.6.8 server; but, I don't think this will have changed through the newer OS versions)
    – Kent
    Sep 10, 2015 at 0:00
  • Wow, I'm quite surprised by your bypass rm FILE and bypass unlink FILE removing all identical files, which does not match what one sees for touch file1; ln file1 file2; ln file2 file3; ls -li; unlink file2; ls -li, nor for touch file1; ln file1 file2; ln file2 file3; ls -li; /System/Library/Extensions/TMSafetyNet.kext/Contents/Helpers/bypass unlink file2; ls -li. I'm never going to remove something from a backup...
    – Arjan
    Sep 10, 2015 at 6:47
4

Time machine works like rsnapshot. It creates a tree of hard links for each new backup. Hard links to files already existing in a previous backup use very little additional space. Only when the last hard link to a file is removed is the file actually deleted from the filesystem.

Removing an entire individual backup won't hurt. You're just removing hard links. No other backup will be affected. But that can be accomplished via tmutil.

One scenario in which it may be necessary to bypass protection is to remove a specific file from all backups (and the reason why I ended up on this post).

My backup disk is full. I have a very large file (many gigabytes) that has been backed up for months. There is one physical copy of it, but many snapshots with hard links to that copy. To actually get rid of that file, I need to remove the hard link from every backup.

Note that the inode number is the same for all hard links to the same file.

% cd /Volumes/WD\ 500G\ USB/Backups.backupdb/csm-laptop
% ls -li */Macintosh\ HD/Users/csm/vm.img
...
2740350 -rw-r--r--@ 28 csm  staff  42949672960 Feb 17 16:12 2015-05-08-005636/Macintosh HD/Users/csm/vm.img
2740350 -rw-r--r--@ 28 csm  staff  42949672960 Feb 17 16:12 2015-05-08-015812/Macintosh HD/Users/csm/vm.img
2740350 -rw-r--r--@ 28 csm  staff  42949672960 Feb 17 16:12 2015-05-08-030036/Macintosh HD/Users/csm/vm.img
2740350 -rw-r--r--@ 28 csm  staff  42949672960 Feb 17 16:12 2015-05-08-041307/Macintosh HD/Users/csm/vm.img
2740350 -rw-r--r--@ 28 csm  staff  42949672960 Feb 17 16:12 Latest/Macintosh HD/Users/csm/vm.img

(Latest is just a symlink to the last dated directory)

% sudo bypass rm -f */Macintosh\ HD\Users\csm\vm.img

The file is removed from all backups, and space is returned. If the file has been changing over time, each backup will have a full copy and the space returned will be enormous.

1
  • As for "Removing an entire individual backup won't hurt. You're just removing hard links. No other backup will be affected.": if this refers to snapshots within backups for a single machine, then given Kent's (old) answer using rm -r might be (or: have been) dangerous. Better be safe than sorry and use tmutil for that.
    – Arjan
    Aug 26, 2015 at 16:41
3

Note: Due to the "TM Safety Net" mentioned by Eric W, this answer doesn't work for the case of a Time Machine backup, which the question specifically pertains to. But for most every other case, the information about how to get rid of ACLs is relevant.


There's no need to use ACL tools copied from an older OS.

Use ls -le to view ACLs and chmod to alter them.

For more info, type man chmod and look under ”ACL Manipulation Options” .

The command to remove all ACLs from a folder hierarchy is:

chmod -R -N foldername
0
1

If you're not executing the command as the user that "owns" the backup, you're going to have a hard time deleting from the command line. I just had this issue with a migration, and we had to ditto the entire Time Machine backup (1tb+) and format the drive before we could get any sort of access to it - and trust me, I tried everything to override the permissions.

3
  • 2
    sudo should give me access to all files as I'm running as administrator. I'm pretty sure this is an ACL issue and I'm working on that.
    – Tim
    Jul 13, 2010 at 15:09
  • 1
    Nic, a few years late, but in case you missed it: see Eric's answer.
    – Arjan
    Aug 26, 2015 at 15:13
  • @Tim: this assertion would even be untrue on other unixoid systems, given the immutable attribute (chattr). What root will give you theoretically is to circumvent any security net. Nov 19, 2015 at 17:37
0

If you want to delete all files in a folder and not just specific files, you can accomplish this by adding the folder to Time Machine's exclusion list. (System preferences -> Time Machine -> Options. Drag the folder here.)

Next time you perform a backup, copies of that folder will be removed from previous backups.

Now, if you really want to do this from a CLI, there's a way, albeit a little cumbersome.

  1. Make a backup of /Library/Preferences/com.apple.TimeMachine.plist
  2. Copy /Library/Preferences/com.apple.TimeMachine.plist somewhere where you can toy with it.
  3. Cd into whereever you put it.
  4. Execute
    plutil -convert xml1 com.apple.TimeMachine.plist
    to convert it from binary form.
  5. Open the converted plist in preferable text editor, search for "skippaths"
  6. Insert a new line in that section, formatted as <string>/Path/To/Exclude</string>
  7. Save and exit, convert back by executing
    plutil -convert binary1 com.apple.TimeMachine.plist
  8. Copy your edited plist back into /Library/Preferences/
  9. Start a backup by executing
    /System/Library/CoreServices/backupd.bundle/Contents/Resources/backupd-helper -auto

Edit: When you perform step 9, all copies of the newly excluded folder will be erased from previous backups.

To remove the exception, copy your backup back into /Library/Preferences.

7
  • I'm not trying to exclude a directory. I'm trying to delete a directory from an existing backup and to do some from the commandline. Really, I'd just like to figure out how to allow rm to work within a Backups.backupdb directory.
    – Tim
    Jul 12, 2010 at 20:30
  • OK, perhaps it wasn't clear from my instructions that the folder is actually deleted from the backups when performing the above steps. See my edit. But if it's not the solution you are looking for, then I suppose you could always su - root and then rm -rf the folders, but I think fiddling with something as precious as backups in that way is something one should generally try to avoid.
    – Frost
    Jul 13, 2010 at 6:27
  • But adding it as an exclusion would delete all copies of it from all the backups. Also, though this is a TimeMachine backup, it is not from the machine that I'm doing this work on, so it is no longer an active TimeMachine. sudo has the same effect as "su - root" and then "rm -rf". I'm pretty sure this is failing because Apple has added a level of security to the filesystem beyond simple *nix permissions.
    – Tim
    Jul 13, 2010 at 15:11
  • This might sound stupid, but I'm not entirely sure that sudo has the same effect as su - root on a mac. I seem to recall an occasion when I was trying to delete something where sudo didn't suffice, but sudo - root did the trick.
    – Frost
    Jul 14, 2010 at 12:58
  • 3
    @Frost- I don't think Time Machine works the way you suggest. I just tried excluding a folder, then running TM, and the older backups of that folder are still present in TM. Or perhaps its behavior has changed since you posted this nearly a year ago. Jun 1, 2011 at 16:20
0

You can make ls list extended attributes in a long view using the -@ flag. It will list ACLs when you supply the -e flag. So, you can find out what you're dealing with by using ls -lea@ DIR.

Judging by my local Time Machine backups, it looks like Time Machine applies extended attributes with metadata about the newest and oldest snapshots. The data stored by the xattrs looks to be a binary plist. These seem innocuous.

Time Machine also looks to apply ACLs to certain directories it knows about, such as the ones placed in a standard user directory. There are two kinds of ACLs potentially standing in your way: those applied directly to the file or directory that deny delete, and those applied to a parent of the file that deny delete_child.

Unfortunately, Mac OS X does not supply the user utilities getfacl and setfacl specified by POSIX.2c to view and manipulate ACLs. To mess with ACLs, you'll have to do some programming; see the acl(3) manpage.

0

I needed this combination and order of commands for removing leftover files while removing a time machine directory on an external drive. Only use these if no other option worked for you.

  1. Start in Single User Mode (hold CMD+S on boot)
  2. sudo chflags -R nouchg <path to some.backupdb>
  3. sudo chflags -R noschg <path to some.backupdb>
  4. sudo chmod -R ugo+rw <path to some.backupdb>
  5. sudo chmod -R -N <path to some.backupdb>
  6. sudo rm -rf <path to some.backupdb>

Explanations: Why does chown report "Operation not permitted" on OS X?

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .