0

Essential info: I have a file "DB_OUTPUT.TXT" with 304 lines that I need to turn into 304 files (one per line). Each line contains many special characters and may be up to tens of thousands of characters long. For these reasons, I'm having difficulty using a cmd.exe batch file (which limits the amount of input) and the echo command (which would try to execute each special character, short of me having to escape them all).

I also have a file "DB_OUTPUT_FILENAMES.TXT" containing a distinct filename for each line-soon-to-be-file from "db_output.txt". So line 1 of DB_OUTPUT.TXT needs to be the body of a new file with a name equal to line 1 of DB_OUTPUT_FILENAMES.TXT.

Extra info: As you may have guessed, DB_OUTPUT.TXT is output from a database; it contains 304 records with 6 or 7 columns at a fixed width with the last column being a SQL query. Each of these lines (db records) will be used as a script to create new database objects, which is why the special characters need to be preserved.

Question: Is there a way to do this in a batch-like fashion? I'd be happy with either a Windows solution or a Linux one.

2
  • Question: Are you against using a language like Python, PHP, etc to do this? Or do you have a requirement that it needs to be done purely in Bash or a CMD batch? Either way, this miiiight be a question best handled by StackOverflow.
    – dotVezz
    Nov 5, 2013 at 22:03
  • Not against a language; just happened to be more familiar with a shell-like environment. Thanks for the direction.
    – MollyOQ
    Nov 14, 2013 at 19:37

2 Answers 2

0

split should work (just tested on a binary file which is even worse than what you describe):

split -l 1 DB_OUTPUT.txt

This creates a heap of files starting with an "x". To batch rename them, start with

ls x* > oldnames

Double check that the number of lines agrees:

wc -l oldnames DB_OUTPUT_FILENAMES.TXT

This should give twice the same number of lines if your file name list has the proper length. Finally,

( paste oldnames DB_OUTPUT_FILENAMES.TXT ) | while read i; do mv $i; done

(Note that this assumes that the output filenames don't contain spaces.)

0

Here is a shell script:

#!/bin/sh

test "$#" -ne 3 && { echo arguments: datafile namefile dstdir ; exit 1 ; }

data="$1"
names="$2"
dstdir="$3"

test -e "$dstdir" || mkdir "$dstdir"

while read fn ; do
    read -r lin <&3
    echo -n "$lin" >"$dstdir/$fn"
done <"$names" 3<"$data"

You can run the script in a Unix-like environment like Linux, Mac OS X or Cygwin under Windows. Store the script for example into a file splittofiles. Do chmod a+x splittofiles.

Run it: splittofiles DB_OUTPUT.TXT DB_OUTPUT_FILENAMES.TXT output. The resulting files will be created in the directory output.

The script should be polished by adding more error-checking conditions. Program in Python mentioned in the note can be faster than this script.

1
  • read by default treats \ specially. The switch -r has to be added so that read does not do so. Nov 6, 2013 at 22:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .