Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I'm using a software that regularly scans thousands of files (probably the name and modification time). This takes quite a while since the files are accessed via a network share. Both machines run windows xp sp3.
Are there any simple, reliable tweaks to speed up the directory scan?

share|improve this question
    
What application are we talking about? –  Cary Schwartzman Feb 2 '11 at 20:54
    
Bdb at work ...but somehow I don't think that makes a difference ;-) –  VolkerK Feb 2 '11 at 20:54

4 Answers 4

up vote 1 down vote accepted

If you're working with directories that contain hundreds or thousands of files or subdirectories (or even a couple dozen files per directory, all with similar names), you're probably running into collisions with the 8.3 DOS-compatible filenames. There is a Microsoft KB article with instructions on how to improve directory enumeration performance by disabling 8.3 filename creation.

You can also disable the Last Accessed timestamp to improve performance. There's a small chance this may cause problems with other programs, but if you do notice problems you can easily re-enable the Last Accessed timestamp.

You may be able to get away with applying these changes to just the machine that's hosting the network share, but you could apply the changes on both machines just to be safe.

If you're running an antivirus program on both machines, you should also try disabling on-access scanning temporarily on both machines to see if that improves performance. In my experience, McAfee antivirus causes an insane performance hit every time you touch a file, but some other antivirus apps (such as CA and Avast) don't seem to have as severe a performance hit, even with on-access scanning enabled. Most antivirus programs allow you to specify directories to exclude from on-access scanning.

share|improve this answer
    
Both tips helped a lot, though I stick with only disabling the 8.3 filenames for now. –  VolkerK Dec 12 '09 at 9:04

A few ideas immediately come to mind:

  • See if it is possible to cache file data locally, within your application. That way it'd only have to do the scan once.

  • Upgrade the speed of your network. Gigabit ethernet PCI cards are $30-$40 CAD in my area. If you want a router, that's another $100. If it's only one other computer, however, then you can get away with running a piece of crossover (P2P) cat6 ethernet cable between the two.

  • The slowness may also be a function of the speed of the two computers in question. CPUs? Amount and speed of memory? What model of hard drives?

share|improve this answer

Accessing files over the wire is always going to be slower than if you access a local disk. You might be able to increase the performance of the scan if you set it up to run the scan locally. Whenever you start pulling files across the wire you will start noticing these types of slowdowns due to the number of files being accessed.

If you run the same scan remotely it will have to setup the connection to transfer/access each file which is why it is probably so slow.

It is possible to tweak the protocol you are using to access the files, but you will see much more dramatic results just moving the process to a local scan versus a network scan.

Something to note here, many enterprise class backup solutions also include a locally installed component that serves this very same purpose. The files can be transferred in a push versus a pull across the network which is a significant improvement.

share|improve this answer
    
It's not possible to change the location of the files or the way it is accessed, it's just the way this application works. You create a network drive, set the application's data dir to that network drive et volià you have a so-called multi-user installation. Works ok on a fresh install. But over time the number of files grows exceedingly and with it the time it takes to perform that (often used) operation. –  VolkerK Aug 6 '09 at 18:11

I have a similar issue. I was thinking about running an FTP server on the remote machine and mapping a FTP Drive mapping instead. I haven't tried it yet but I bet it would get better performance. If it doesn't then I might just use a FTP client to access it and just abandon microsoft drive mapping altogether. I was going to use NULL Ftp Server to try the scenerio.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.