0

I'm not very technical, just a humble user. What I'd like to do is have a way to add user defined attributes to a file, that I could then use to search on or control which files are displayed. e.g. add an attribute "foo" (for a file type like .pdf or .txt), and arbitrarily assign a value to "foo" for any file of that type, so I could display in a file list e.g. only files that have a "foo" attribute with a value of "bar". For me this would make finding and organizing things a heck of a lot easier.

I have discovered the "attr", "getfattr" and "setfattr" terminal commands that appear to do the first part of what I want. The second part though, accessing these attributes from within a file manager like Nemo or Nautilus doesn't seem to be possible. I've searched for Nemo extensions, Nemo attributes, xattr, user_attr, all to no avail. Nothing I have found can access or display the extended attributes defined with the above commands.

I'm aware there are applications to manage particular file types like music, video, picture etc types that also have extended attributes, but they are bespoke aps that I'm assuming manage their attributes internally for the files and folders they have been given access to, and do not use the extended attributes functionality of the file system itself.

Have I missed something? Is it possible to access user attributes from within a stock file manager? All advice gratefully received

1

What you want are known as extended attributes on Linux (and many other UNIX-like systems), forks on macOS, and alternative data streams on Windows. Most platforms have support for this type of thing in some form or other, though it's often dependent on the filesystem supporting a way to store them (most you're likely to encounter do).

The thing is though, they tend to have very specific and limited usage in practice, largely because most people either don't want to count on them, or don't even know they exist.

Actual usage consists of:

  • On macOS, there's a special fork called the resource fork for every file. This is used to store structured metadata and other associated data. For example, most executables on macOS store their program icons in their resource fork. Rather significantly, data in resource forks may be indexed by Spotlight, so you can theoretically utilize that to some etent for what you're talking about doing.
  • On Windows, there's a special alternate data stream (that I can't remember the name of right now) that is used to store the origin domain name for files downloaded from the internet. This is used to trigger the 'Are you sure you want to do this?' prompts when you try to run a file downloaded from the internet.
  • On Linux, file capabilities (security attributes that give special additional permissions to an executable) are stored as an extended attribute.
  • On Linux, SELinux and most other mandatory access control systems use extended attributes internally to store security contexts (special tags used to determine what resources have access to that file).
  • A number of distributed file systems use extended attributes to store internal metadata about a given file (for example, keeping track of where other copies of the same file are located).
  • Malware may use alternate data streams, extended attributes, or forks to hide data from users, as they are not usually user visible.

Beyond that there's actually not much practical support for them. Most good backup and archiving tools support saving and restoring them, but most other user applications just ignore them, preferring to either use internal databases or inherent support for structured metadata that's built into the file formats (see below) to store metadata relevant to the files in question, as those allow them to work everywhere, not just on systems that support this at a filesystem level.


Edit in response to comments:

Most file formats that are not plain text, as well as some plain text file formats (such as HTML) support storing some form of structured metadata in the file itself. Usually, this is limited to metadata that's relevant to the type of data in the file (for example, for audio files this typically includes stuff like a track title, artist name, album name, and similar things useful for cataloging music), but some formats either allow free-form tagged data (for example, HTML technically allows for this with it's <meta> tag in the document head) or a 'comment' tag that simply stores arbitrary text (ID3 and EXIF both have this, with it normally being used to store a text description of the media).

Trying to describe how these work here is not practical due to the huge variety of different approaches taken, but here are some links that include basic descriptions of what some of the more common ones do:

  • ID3 is the tag format used for MP3, AIFF, and WAV audio files. Most of the same fields are used in the tag formats provided by other audio container formats (for example, FLAC, WMA, Ogg, etc).
  • Vorbis comments are the other big audio tag format, used by all formats that utilize the Ogg container format (Vorbis, Speex, Theora, and Opus) as well as FLAC.
  • Exif defines a format for metadata tagging for images and audio which is very widely used with image files (not so much for audio though). When you open photos from a digital camera, this is how you're able to see stuff like when the photo was taken, what the exposure settings were, where it was taken, and possibly also get color correction applied.
  • RDF is used by a variety of document formats (including PDF) either as embedded metadata or occasionally as detached metadata in it's own file. It's technically a 'meta-format' in that it doesn't specify any tags but leaves that up to the user (so you can theoretically use it for anything).
  • The HTML <head> element is an example of how this type of metadata is handled in a plain text format. It's used in HTML documents to store the document title, arbitrary tagged metadata (including things like the document author or character encoding), and links to stylesheets, scripts, and other external resources relevant to the document.
  • PNG files are a good example of a custom metadata handling approach, with the extra metadata being stored as 'ancillary chunks'. It supports embedding a block of Exif metadata under its own tag, but also supports a number of other tags covering a handful of things that Exif does not inherently.

Note that in many cases, especially media files (images, audio, and video), some file managers support displaying the contents of specific well known and widely used metadata tags as part of the file listing. Windows Explorer for example can display most of the standard audio file tags (stuff like artist, album, and track names, track length, and track and disc numbers) in it's 'Details' view when showing audio files, and you can even sort on the contents of those tags.

| improve this answer | |
  • Thanks Austin for your comprehensive and detailed reply, much appreciated. Could you explain how structured metadata is built into the file format, as in the example you made of ID3 tags? Are there any other file formats that have predefined metadata, like pdf or txt, or office formats like ods and odt? I find it curious that somebody somewhere a long time ago thought extended attributes was a good idea (and I agree) and implemented them in various file formats, as you have noted. And yet nobody really uses them except in a few specialist areas. But what do I know... :-) – mogplus8 Jul 26 at 2:31
  • @mogplus8 Edited to provide some more info about file metadata formats. – Austin Hemmelgarn Jul 26 at 13:21
  • Wow! Thanks again Austin, a wealth of information to sort through and understand. – mogplus8 Jul 27 at 14:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.