How can Mac OS X save details about the URL from which a file has been downloaded?

Question

WRT this topic What should I do about com.apple.quarantine? , I want to know how can OS X learn and save those information in the @ field. I'm not looking for a way to disable this feature, but for a code (or pseudocode, idea, whatever) regarding the implementation of this thing. Thank you in advance.

Answer 1

As Daniel mentioned, the browser can store the source URL (and also the referrer) of a downloaded file in the extended attribute com.apple.metadata:kMDItemWhereFroms.

First Apple added this to Safari for Mac OS X 10.4, then it was added to Chrome/Chromium [Issue 22289], and there is an outstanding enhancement request to add it to Firefox [Bug 337051].

As for the actual code to implement this, check out the Chromium patch.

Answer 2

It's set by the application downloading the file. No magic download detection, just an attribute set on the downloaded file by the downloading application.

I just downloaded a zip file using wget, Google Chrome, Firefox and Safari, and, as expected, the wget download has no extended attributes, the browser downloads have, as they're aware of that feature. Interestingly, the Firefox download set quarantine, but forgot to set com.apple.metadata:kMDItemWhereFroms, the information where the downloaded file was originally located (its URL). Safari and Chrome set both, wget neither.

Answer 3

That information is stored in what are called "extended attributes". John Siracusa has an excellent article discussing extended attributes when they were first introduced, in OS X 10.4: http://arstechnica.com/apple/reviews/2005/04/macosx-10-4.ars/7.

It appears there is no longer a man page for the 'xattr' command line tool, but 'xattr -h' should show basic help.

A POSIX/BSD-level API is available in /usr/include/sys/xattr.h which defines C functions such as getxattr(), setxattr(), removexattr(), listxattr(), and their FILE-based cousins. (These commands, unlike 'xattr(1)', do have man pages).

Answer 4

After downloading any file, running xattr file will show you what the browser has stored in the extended attributes:

xattr mydownload.gz
com.apple.metadata:kMDItemWhereFroms
com.apple.quarantine

Next, more details:

xattr -p com.apple.quarantine mydownload.gz 
0000;4cee9d4b;Safari;24064D6B-9854-46BC-AF73-5DE5F8042D0B|com.apple.Safari

xattr -p com.apple.metadata:kMDItemWhereFroms mydownload.gz 
62 70 6C 69 73 74 30 30 A1 01 5F 10 24 68 74 74
70 3A 2F 2F 73 75 [and so on]

...or, in readable format:

mdls -name kMDItemWhereFroms mydownload.gz
kMDItemWhereFroms = (
    "http://some-url"
)

You can set those (or any attribute you want) yourself too:

xattr -w some-name some-value mydownload.gz

The following works too, even though is has different results when running the above commands again:

xattr -w com.apple.metadata:kMDItemWhereFroms http://example.com mydownload.gz

And for the com.apple.quarantine attribute, according to understand:

Typically, the Quarantine data consists of a set of 4 semicolon-delimited values (a set of decimal digits of an unknown purpose - typically 0000, an 8 character opaque ID, the name of the application that created the Quarantined file, and the application's UTI (Uniform Type Identifier) prefixed with a pipe symbol)

For example, for two different files downloaded using Mozilla FireFox, this looks like:

com.apple.quarantine: 0000;4b392bb2;Firefox;|org.mozilla.firefox
com.apple.quarantine: 0000;4b38d820;Firefox;|org.mozilla.firefox

Using Automator's Folder Actions you could run such commands for new files in a specific folder, if that's what you after.