28

I've just discovered UltraSearch and was blown away by its file and folder search speed. It's instantaneous. And doesn't use any indexing service. It simply uses the NTFS Master File Table, which already stores all the filenames on the NTFS partition.

The question is, why isn't this capability way more popular among file managers, and Windows Explorer Search (Win+F) to begin with?

2

3 Answers 3

29

Because of Security!

That's the real reason. (And the only real reason, in my opinion -- it's not that hard to make a reader for major file systems, although it's by no means easy; making a writer is the real challenge.)

A program like this bypasses the entire (file) system's security infrastructure, so only an administrator (or someone else who has "Manage Volume" privileges) can actually run it.

So obviously, it wouldn't work in many scenarios -- and I don't think Microsoft (or any other big company) would ever consider making a product like this and then encouraging users to run as administrators, because of the security ramifications.

It would be theoretically possible to make a system which runs in the background and filters out secured data, but in practice it would be a lot of work to get correct and without security holes for production.

By the way I haven't used UltraSearch, but I'd written a very similar program myself a few years ago which I open-sourced just last month! Check it out if you're interested. :)

9
  • 1
    This does not feel like a right reason. OS can give a view for unsecured search just like a DMBS. An API or a restricted view should give public access to public files. And if the file table does not know anything about security of different directories than it is probably bad design on the OS design end
    – LifeH2O
    May 24, 2015 at 13:01
  • 1
    How the performance hit can be more than scanning directories? Only the security of inner directories will need to be checked. I don't know how much can be done with windows file table.
    – LifeH2O
    May 24, 2015 at 18:31
  • 1
    @LifeH2O: Have you considered how complicated it is to "check" something? Users belong to multiple groups, groups and users can each have allow/deny/neither permissions on either some directory on the chain or on the file itself, and you have to figure out the effective permissions for the current user on each file using its ACL. Now add to that the synchronization required with the kernel's security manager subsystem, and you're going to get massive performance hits just "checking" all the files.
    – user541686
    May 24, 2015 at 18:50
  • 2
    You need to provide something authoritive indicating what you are saying, otherwise people can't differentiate speculation from information. I agree with others, this is purely speculation.
    – Sam Hobbs
    Dec 4, 2016 at 3:20
  • 1
    And the only real reason, in my opinion - what about if the files don't actually exist? stackoverflow.com/a/13937860/478656 e.g. a backup system which presents virtual files which you can enumerate the filesystem through the normal API calls and see them, but they really reside on a backup store and have no entries in the MFT. Or PSProviders in PowerShell, or files in the Offline Folders client-side-cache, etc. Dec 14, 2016 at 20:35
6

File managers have to be able to support every single filesystem that could be encountered. As such, they have to call into the VFS via its API. There is no (sane) way to return a large array from an API call, which results in the file enumeration being serial regardless of the presence of a MFT/FAT/superblock.

2
  • 1
    If you were a programmer then you would know how APIs manage large amounts of data such as you say. And no, a search program is not required to support multiple file systems.
    – Sam Hobbs
    Dec 4, 2016 at 3:23
  • @user34660: They have two choices: 1) Use enumeration. 2) Run very slowly when handling very large datasets. And a search tool that only supports a single filesystem is of very limited utility. Dec 4, 2016 at 3:31
3

File indexing service is for user who would like to search content (most likely text) and metadata of files, not merely filename. That's why it takes a long time go walk through all the files and the index built from such services is big and relatively slow. You can disable indexing service in Windows but windows explorer is stupid enough to keep searching file content after filenames. As Ignacio Vazquez-Abrams said the file managers cannot take advantage from low-level file system.

Not the answer you're looking for? Browse other questions tagged .