locked files on HFS+ home partition shared between OSX/Linux

Question

I dual boot into Arch Linux and OS X 10.6 on my MacBook pro. I synced my UID between both OSes and created an HFS partition (with no journaling) to use as a shared home/Users partition. For the most part it works just as I'd expect, but sometimes when I'm booted into OS X certain files are "locked" (when I get info on a particular file the "Locked" box is checked under the "General" pane. I can resolve the issue by manually unchecking the box) and/or I get "Operation not permitted" when I try deleting or chmod'ing a file. In both cases I don't see anything out of the ordinary on the permission bits displayed with ls -l, except for a trailing '@' character in the position where the sticky bit would normally occur:

-rw-r--r--@  1 myuser  mygroup   296 Mar 29 11:44 myfile

This '@' character shows up on ALL normal files, so doesn't seem to be linked to the locked/operation not permission situation.

On the Linux side of things I never have permission problems. To the best of my limited knowledge and experience with ACLs I've not found any ACLs on any of the files in question.

For what it's worth, I do most of my file editing using emacs (Aquamacs in OSX), is it possible it is setting weird permission bits?

1. What is the "locked" setting that OS X uses and does it have a permission bit equivalent (so at the very least I could recursively unlock all files in my home directory from the terminal)
2. why might some, but not other files get "locked" when booting into OS X
3. what is the meaning of the '@' character?

Answer 1

I have been running into the same issue too.

My understanding from information that I read here, and on other places, is that it's a linux kernel bug in the hfsplus module. It adds random user flags to files. There are two flags that prevent editing/deleting files: uchg and uappnd. These are the two bad guys. They can be applied to a file or even to a parent directory.

Flags are displayed with:

$ ls -laO /Volumes/my-volume

Flags can be removed recursively with:

$ man chflags

$ chflags -R nouchg,nouappnd,noopaque,dump /Volumes/my-volume

NOTE: I remove also the opaque and nodump flags. I don' t need no flags.

Answer 2

The @ means that file has "extended attributes" (extra metadata, abbreviated "xattrs") attached to it in the filesystem. To see the list of xattrs attached to the files, do ls -l@ in Mac OS X.

Classic Mac OS had the concept of "Finder Info" which was a small fixed (non-extensible) set of metadata that all files on an HFS volume had. This included the "type and creator codes", and the "Finder flags", including the "locked" bit, the "visible" (hidden) bit, and several others. Mac OS X has basically deprecated the old Finder meta-data, but on the occasions where it's still needed, it now gets attached to the file's record in the filesystem as an xattr. Those locked files you're seeing almost certainly have this Finder info xattr attached, so that the state of the old Finder "locked" bit can be recorded.

My Snow Leopard system has a /usr/bin/xattr command that seems to have no man page, but it does have a usage statement if you invoke it with -h. Note that xattr -l filename can be useful to get a hex/ASCII dump of the values of the xattrs attached to a file.

Mac OS X's built-in commands for viewing and manipulating the old Finder info xattr from the terminal include GetFileInfo(1) and SetFile(1).

Update:
I have no good explanation for why those files are getting locked, but my hunch would be that whatever HFS support software you're running in Linux is either misunderstanding the purpose, and the deprecated status, of the old Finder lock bit and setting it when it shouldn't, or it's intentionally using the lock bit as a mechanism to map some kind of Linux filesystem semantic concept onto HFS.

The Finder lock bit was intended as a way for users to manually lock their own files so they didn't accidentally modify or delete them, and was not meant as a mechanism for process-level file locking to avoid multiple processes writing to the same file at the same time. That is, it was not supposed to be a replacement for fcntl(2) or flock(2). At the time the Finder lock bit was designed, the Mac was not a multiprocessing system.

Update 2: It could also be that Aquamacs is abusing the old Finder lock bit to carry out emacs' file locking wishes.

Answer 3

I've found a workaround. It's seems to be a race condition in the hfsplus kernel module, caused by non atomic access to userflags. I've disabled it and the userflags will ever be zero, unlocked, ok for me.

fs/hfsplus/inode.c near line 248:

    inode->i_mode = mode;

/* FIXME commented out because of unreliable results, needs mutex_lock (?) */
//    HFSPLUS_I(inode)->userflags = perms->userflags;
    if (perms->rootflags & HFSPLUS_FLG_IMMUTABLE)

fs/hfsplus/catalog.c near line 79:

            perms->rootflags &= ~HFSPLUS_FLG_APPEND;

/* FIXME commented out because of unreliable results, needs mutex_lock (?) */
//    perms->userflags = HFSPLUS_I(inode)->userflags;
    perms->mode = cpu_to_be16(inode->i_mode);

You could build a custom kernel, but I use dkms:

$ cd /usr/src
$ tar xjpvf linux-source-*.tar.bz2 linux-source-*/fs/hfsplus
$ cp -R linux-source-*/fs/hfsplus hfsplus-YOUR_VERSION
$ vi hfsplus-YOUR_VERSION/inode.c
$ vi hfsplus-YOUR_VERSION/catalog.c
$ vi hfsplus-YOUR_VERSION/dkms.conf (see below for the content)
$ su
# dkms install hfsplus/YOUR_VERSION

/usr/src/hfsplus-YOUR_VERSION/dkms.conf:

NAME=hfsplus
VERSION=YOUR_VERSION
PACKAGE_NAME="$NAME"
PACKAGE_VERSION="$VERSION"
MAKE[0]="make -C ${kernel_source_dir}
  SUBDIRS=${dkms_tree}/${NAME}/${VERSION}/build modules"
BUILT_MODULE_NAME[0]="hfsplus"
DEST_MODULE_LOCATION[0]="/kernel/fs/hfsplus"
REMAKE_INITRD=y
AUTOINSTALL="yes"

Note: The installation fails for me, if I don't cd into /usr/src .

To uninstall:

# dkms remove hfsplus/YOUR_VERSION --all

Environment: MacBookPro7,1, Core 2 Duo, SATA NVidia MCP89 AHCI, Mac OS X 10.6, Debian GNU/Linux, Kernel 2.6.28, 2.6.29, 3.0, 3.1, 3.2.

Answer 4

This is a Linux kernel bug, fixed in 3.4 (patch).

I've had the same problem using pure Unix utilities. Namely, I've backed up my Mac OS X hard drive from Xubuntu 12.04 live using rsync. After restore, many folders were randomly locked, including directories in git repositories (and I highly doubt git would do that). You can see those attribs with ls -lO. Doing that on my backup shows that these bits have essentially random values:

# ls -ldO /Volumes/HFS+Backup/Users/pgiarrusso/*
drwx------   31 pgiarrusso  staff  uchg,nodump,opaque         1054 Aug 13 02:00 /Volumes/HFS+Backup/Users/pgiarrusso/Desktop
drwx------   36 pgiarrusso  staff  nodump                     1224 Jul 22 16:04 /Volumes/HFS+Backup/Users/pgiarrusso/Documents
drwx------  108 pgiarrusso  staff  uappnd                     3672 Aug 13 11:43 /Volumes/HFS+Backup/Users/pgiarrusso/Downloads
drwx------   13 pgiarrusso  staff  uappnd,uchg,opaque          442 Jul 22 05:04 /Volumes/HFS+Backup/Users/pgiarrusso/Dropbox
drwx------   53 pgiarrusso  staff  -                          1802 Aug 12 00:58 /Volumes/HFS+Backup/Users/pgiarrusso/Library
drwx------   11 pgiarrusso  staff  uchg,nodump,opaque          374 Jul 22 17:25 /Volumes/HFS+Backup/Users/pgiarrusso/Movies
drwx------   13 pgiarrusso  staff  uappnd,uchg,nodump          442 Jun 10 12:05 /Volumes/HFS+Backup/Users/pgiarrusso/Music
drwx------   15 pgiarrusso  staff  uappnd,nodump,opaque        510 Jun 10 12:05 /Volumes/HFS+Backup/Users/pgiarrusso/Pictures
drwxr-x---   11 pgiarrusso  staff  opaque                      374 Jul  6 15:33 /Volumes/HFS+Backup/Users/pgiarrusso/Public
drwxr-xr-x   34 pgiarrusso  staff  uappnd,uchg,opaque         1156 May 27 12:39 /Volumes/HFS+Backup/Users/pgiarrusso/Sites
drwxr-xr-x    2 pgiarrusso  staff  uappnd,nodump,opaque         68 Jun 10 21:43 /Volumes/HFS+Backup/Users/pgiarrusso/VirtualBox VMs
-rwxr-xr-x    1 pgiarrusso  staff  uappnd,nodump,opaque       1703 Feb 19  2012 /Volumes/HFS+Backup/Users/pgiarrusso/bash-prompt.sh
drwxr-xr-x   22 pgiarrusso  staff  -                           748 Aug 10 19:47 /Volumes/HFS+Backup/Users/pgiarrusso/bin
lrwxrwxrwx    1 pgiarrusso  staff  nodump,opaque                37 Sep 27  2011 /Volumes/HFS+Backup/Users/pgiarrusso/default.sfx -> /Users/pgiarrusso/opt/rar/default.sfx
-rw-r--r--    1 pgiarrusso  staff  uappnd,uchg          1375563169 Aug  2 18:52 /Volumes/HFS+Backup/Users/pgiarrusso/heapdump-1343925310626.hprof
drwxr-xr-x   22 pgiarrusso  staff  uappnd,nodump               748 Aug  1 22:15 /Volumes/HFS+Backup/Users/pgiarrusso/opt
drwxr-xr-x    7 pgiarrusso  staff  uappnd                      238 Apr 19 20:00 /Volumes/HFS+Backup/Users/pgiarrusso/share
drwxr-xr-x   35 pgiarrusso  staff  nodump,opaque              1190 Aug 10 00:06 /Volumes/HFS+Backup/Users/pgiarrusso/tmp

I've compared this to the same directory on a working filesystem, and those bits are not set.