5

I have ~300 files that need to be truncated (I need to remove a specified number of bytes from the beginning of the file). I can do it one by one using a hex file editor, but given the number of files I have to process that would be a rather overwhelming task.

Is there an automated solution for this? (Operating system is Windows 7 64bit.)

1
  • 1
    Your best bet is to use PowerShell. It should be pre-installed with the O.S. I'm not on a Windows machine now, but I'll come back with an answer tomorrow, if noone comes up with one. Here's a clue though: sans.org/windows-security/2010/02/11/…
    – user127350
    Oct 8, 2012 at 21:31

3 Answers 3

5

dd has a skip option.

Per file You can use dd if=MyFile of=my_new_file skip=BytesToSkip

Optionally in a loop (e.g. in bash in the target directory)
for a in * ; do echo processing $a ; dd if=$a of=$a.shorter skip=300 ; done

Adjust skip with the correct number of bytes, KB or MB
If the files are large playing with the block size (bs=X) might speed things up.

3
  • Tell that to Windows... :)
    – user127350
    Oct 8, 2012 at 21:29
  • There was no win7 tag yet when I started my answer. Anyway, how big are the files? Small enough for a quick move to a unix host? Large enough to warrant installing extra tools?
    – Hennes
    Oct 8, 2012 at 21:44
  • 6
    dd is available for Windows too. In CMD the loop could be done like this: for %a in (*.*) do dd if=%a of=%a.shorter skip=300. Oct 8, 2012 at 21:46
4

Here you go...

Powershell code:

$PATH = "d:\My Dir"
$BYTES_TO_TRIM = 10

$files = dir $PATH | where { !$_.PsIsContainer }

foreach ($file in $files) {
    Write-Output "File being truncated: $($file.FullName)"
    Write-Output "  Original Size: $($file.Length) bytes"
    Write-Output "  Truncating $BYTES_TO_TRIM bytes..."
    $byteEncodedContent = [System.IO.File]::ReadAllBytes($file.FullName)
    $truncatedByteEncodedContent = $byteEncodedContent[$BYTES_TO_TRIM..($byteEncodedContent.Length - 1)]
    Set-Content -value $truncatedByteEncodedContent -encoding byte -path "$($file.FullName)"

    Write-Output "  Size after truncation: $((Get-Item $file.FullName).Length) bytes"
    Write-Output "Truncation done!`n"
}
2
  • 1
    This script gives the following error: "Exception of type 'System.OutOfMemoryException' was thrown.."
    – dtmland
    May 8, 2014 at 19:55
  • @dtmland I didn't thought of 100 GB files when I wrote it. If you want you can come with a byte-by-byte reading solution.
    – user127350
    May 10, 2014 at 18:26
0

Without loading the whole file into memory:
eg. remove 3 leading bytes from all log files and save copy to 1_<same name>:

set files=*.log
set bytes=3
for %i in (%files%) do set /a size=%~zi-%bytes% >nul & powershell gc "%i" -Encoding byte -Tail !size! ^| sc 1_"%i" -Encoding byte

Tested in Win 10 cmd

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .