Automatically extracting text files via commandline and combining them into a single text file

Question

In-Depth Explanation of Problem:

I'm trying to automate a process on my computer and having some difficulty. Each day, I'm emailed a zip file that contains a txt file. I have a script in place that automatically takes the attachment and dumps it into a local directory.

Now, I'm trying to figure out how I can get the contents of the txt file from the directory and append it to a "master file". So basically, how it works now is I have an archive emailed to me each day of [TODAY'S_DATE].zip. A script moves that archive to a folder. I then have a .bat file that extracts the contents to a folder named [TODAY'S_DATE] and moves the archive to a folder titled "Completed". I need to know how to take the text out of the file as it is extracted and dump it into a "Master.txt" file that will continuously get larger.

This would avoid the need of extracting the files to their own directories and manually copying the text from them and into the master file.

My setup:

• Using Windows 7
• Using 7zip Command Line for extraction
• Using .Bat file to extract file from archive to directory

Problem:

• Need to take txt from archive and merge it into a "master.txt" file.

Current contents of .bat file:

7za x *.zip -o*

COPY /Y *.zip " \Completed\"

@echo extraction complete

I really appreciate any help that can be offered. I know this was really long-winded, but often when I see these types of questions, not enough detail is presented. Thank you again.

Answer 1

You can append the contents of one file to another file with the "type" command and the ">>" operator like so:

type todaysfile.txt >> masterfile.txt

You might first have to insert a new line into your masterfile, so the new content starts at its own new line, like so:

echo. >> masterfile

If you do not know the filenames, you can append every file in the directory "completed" like so:

type completed\* >> masterfile.txt

"type" and "echo" will output things, and the ">>" operator tells it to append that output to a file.

Source: type, echo, command redirection

Answer 2

It's simple:
copy file1.txt+fileX.txt intome.txt or copy singlefile.txt+intome.txt

EDIT: just read your comment for your post. So you don't know the file names? Let me extend the stuff above... just a sec :)

EDIT2: I had to search for it and I think the easiest you can do is by not knowing the file name is: copy /b *.txt newfile.txt as of A command-line or batch cmd to concatenate multiple files

Answer 3

I’m not sure that I understand your comment about not knowing the name of your file.  I guess your situation is that you have a directory structure something like this:

|       ︙
|
+---current
|       730643_594352.txt
|
+---master
|       master.txt

where you don’t know the name (730643_594352.txt) of the current file –– but it is the only file (or at least the only .txt file) in the current directory, so *.txt works.  If that’s it, why not

ren current\*.txt current\today.txt

and then operate on it by name (today.txt)?  If you need to preserve its name as received in the emailed zip, make a parallel temp directory and do

copy current\*.txt temp\today.txt

Answer 4

You have a new file named [TODAY'S_DATE].zip that is moved into a folder by some process.

I assume that:

1. The file is actually named something like: 2013-0511.zip, and it is different every day.
2. The current directory at the time the .bat file is run is the same folder as where the new .zip file has been created.
3. There is only ever one .zip file to process at any given time.
4. The folder that the .txt file is extracted into, is created by the unzip process. In other words, the destination folder is named inside the .zip file so you do nothing special to create the destination folder in commands within the batch file.
5. The name portion of the .zip filename (so '2013-0511' in my example above) is the same as the name of the folder inside the .zip file and the same as the name of the folder the .txt file is extracted into.
6. The full path and filename of the "Master.txt" file is constant.

Here is a batch file that should work for you:

@echo off

rem    define the full path and filename of the "Master.txt" file
set "zzmaster=C:\Master\Master.txt"

for %%f in ("*.zip") do call :work "%%~dpf" "%%~nf"

rem        other processing here..., then goto :EOF
rem        recommend moving or deleting the .zip file so it is not present 
rem        the next time this is run, or use MOVE instead of COPY below


rem        cleanup environemnt variables if necessary
set "zzmaster="
set "zzpath="
set "zzname="
set "zztxt="


goto :EOF



:work
rem:work %1, %2 will contain the Path and Name of the .zip file (extension is of course .zip)

set "zzpath=%~1"
set "zzname=%~2"

rem        this will extract the .zip file as you specified before
7za x "%zzname%.zip" -o*
COPY /Y "%zzname%.zip" "\Completed\"
@echo extraction complete

rem        you know the folder that the .txt file was extracted into is the sane as %zzname%
rem        this will look in the new folder created by the unzip process and get all files with an extension of .txt
rem        this can easily be adjusted if it is not what you want.

for %%g in ("%zzname%\*.txt") do call :doappend "%%~nxg"


rem        optional: delete the folder that contained the extracted .txt file(s)
rem rd "%zzname%">nul 2>&1

goto :EOF



:doappend
rem:doappend %1 is the name.ext of the file to append to master

set "zztxt=%~1"

rem        optional: if Master file already exists, add a blank line to the end of Master file
if exist "%zzmaster%" echo.>> "%zzmaster%"


rem        append the contents of the extracted .txt file to the end of the Master file
type "%zzname%\%zztxt%" >> "%zzmaster%"


rem        optional: delete the extracted .txt file
rem del "%zzname%\%zztxt%">nul 2>&1


goto :EOF

Check the "optional" code to see if it makes sense for you.

If my "assumptions" are not correct or if you need something changed, let me know.

Answer 5

I want to thank everyone who took the time to answer. I did more digging in 7Zip's documentation and tested some things and eventually arrived at the answer.

I'm sure many of the answers on this page can be used in various ways to achieve the same result, but I wanted to avoid as much filehandling as possible.

What was needed was a switch from 7zip (-so). This allows you to extract the file to STDOUT and you can redirect the stream to any file you want.

This is what it looks like:

7za e *.zip -y -so >> masterlist.txt

This allowed me to skip extracting the file to a directory altogether which helps to save disk space and file handling. If you have a structure to the directory and you know the filename (I didn't have that luxury), then you can use:

7za e *.zip -ir!PATH\FILENAME.txt -y -so >> masterlist.txt

Also, in case anyone wants to see the working .bat file, this is what I have.

7za e *.zip -y -so >> masterlist.txt
MOVE /Y *.zip  Completed
@echo extraction complete

Line 1: 7zip extracts the single file and sends the output to STDOUT and that is then appended (>>) to masterlist.txt.

Line 2: The zip archive is moved to the Completed folder so it will not be processed again in the future.

Line 3: It lets you know the extraction is complete, although you probably won't see this (at least with the way I'm using it).

I hope this helps someone out. :-)