Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Imagine a large SAN with one hundred thousand files in a unorganized directory structure.

The files will be moved to a new SAN with an organised directory structure. Some files will be renamed when they are moved. Also many files have references, eg Excel Spreadsheets are interlinked and these will break after the move.

We need to keep a map of From and To file paths encase we need to refer back to the archive, eg to fix a broken link.

I've done research to see if there was anything on the market but no luck. Hence I've begun writing an application that allows users to drag and drop files and records to a SQL dB:

enter image description here

I'm not 100% sure on this approach. Does anyone know if there is a better way to do this? Is there anything in Windows?

  1. We were thinking of scanning the old and new directories and doing a map that way. Unfortunately this wont fly as the file name could change and the Create/Modified etc Dates will be today for the moved files.

  2. We were thinking of having some unique identifier in each file but that idea never got traction as where would you store the unique identifier for different file types?

  3. Possibly a AutoHotKey script?

share|improve this question

2 Answers 2

up vote 1 down vote accepted

I built an automated deployment system in C# for one of my clients. This issue looks similar except in scale.

Things that were taken care of in ADS:

  1. Automatic checkout from SVN.
  2. Traverse through all the folders.
  3. Each folder can be Included or Excluded from rollout.
  4. If a folder is included, all files and sub folders are included as well. Same goes for exclusion.
  5. Just like inclusion, each folder can be relocated to a new location in deployment folder.
  6. Everything was drag-and-drop, (thankfully, deployment scripts don't have to be modified that often).
  7. Most critical part was binary file comparison to copy only modified files. <-- This looks like the most important thing for you as well. Since I was only dealing with code files like C# or perl code files, this was an easy check:
    1. Take files on left and right
    2. Remove all whitespaces.
    3. Compare files one byte at a time.

In your case, unless the files are rather huge, you can use the same approach. If files are huge (assuming these are data files), a simple header/footer comparison should do the trick.

We were thinking of having some unique identifier in each file but that idea never got traction as where would you store the unique identifier for different file types?

This is what I had started with as well, but my problem was not this serious so I scrapped it. Technically, each FileInfo object was wrapped around in a specialized class that had a specific field of type GUID. I was saving this array of serialized classes in XML files.

<FIItem>
    <ID>*some guid*</ID>
    <FileInfoObj>*file info which contains full file path*</FileInfoObj>
</FIItem>
share|improve this answer
    
+1 This is good and storing FileInfo (serialized) rather than FilePaths is much better. I dont need to do binary comparison of files, just keep track. The program was going to work on-the-fly WYSIWYG and not by creating a deployment script but I will present this to colleagues, thanks heaps –  Jeremy Thompson Aug 14 '12 at 23:48

Found a software product that is designed to automatically find and fix broken links when files are moved or renamed and when directory structure changes. See LinkFixer Advanced

share|improve this answer
    
I have used this tool it has some pros (does binary analysis) and cons (cost). See here stackoverflow.com/questions/10924606/… –  Jeremy Thompson Mar 18 '14 at 23:06

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.