Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Essential info: I have a file "DB_OUTPUT.TXT" with 304 lines that I need to turn into 304 files (one per line). Each line contains many special characters and may be up to tens of thousands of characters long. For these reasons, I'm having difficulty using a cmd.exe batch file (which limits the amount of input) and the echo command (which would try to execute each special character, short of me having to escape them all).

I also have a file "DB_OUTPUT_FILENAMES.TXT" containing a distinct filename for each line-soon-to-be-file from "db_output.txt". So line 1 of DB_OUTPUT.TXT needs to be the body of a new file with a name equal to line 1 of DB_OUTPUT_FILENAMES.TXT.

Extra info: As you may have guessed, DB_OUTPUT.TXT is output from a database; it contains 304 records with 6 or 7 columns at a fixed width with the last column being a SQL query. Each of these lines (db records) will be used as a script to create new database objects, which is why the special characters need to be preserved.

Question: Is there a way to do this in a batch-like fashion? I'd be happy with either a Windows solution or a Linux one.

share|improve this question
    
Question: Are you against using a language like Python, PHP, etc to do this? Or do you have a requirement that it needs to be done purely in Bash or a CMD batch? Either way, this miiiight be a question best handled by StackOverflow. –  dotVezz Nov 5 '13 at 22:03
    
Not against a language; just happened to be more familiar with a shell-like environment. Thanks for the direction. –  MollyOQ Nov 14 '13 at 19:37

2 Answers 2

up vote 0 down vote accepted

split should work (just tested on a binary file which is even worse than what you describe):

split -l 1 DB_OUTPUT.txt

This creates a heap of files starting with an "x". To batch rename them, start with

ls x* > oldnames

Double check that the number of lines agrees:

wc -l oldnames DB_OUTPUT_FILENAMES.TXT

This should give twice the same number of lines if your file name list has the proper length. Finally,

( paste oldnames DB_OUTPUT_FILENAMES.TXT ) | while read i; do mv $i; done

(Note that this assumes that the output filenames don't contain spaces.)

share|improve this answer

Here is a shell script:

#!/bin/sh

test "$#" -ne 3 && { echo arguments: datafile namefile dstdir ; exit 1 ; }

data="$1"
names="$2"
dstdir="$3"

test -e "$dstdir" || mkdir "$dstdir"

while read fn ; do
    read -r lin <&3
    echo -n "$lin" >"$dstdir/$fn"
done <"$names" 3<"$data"

You can run the script in a Unix-like environment like Linux, Mac OS X or Cygwin under Windows. Store the script for example into a file splittofiles. Do chmod a+x splittofiles.

Run it: splittofiles DB_OUTPUT.TXT DB_OUTPUT_FILENAMES.TXT output. The resulting files will be created in the directory output.

The script should be polished by adding more error-checking conditions. Program in Python mentioned in the note can be faster than this script.

share|improve this answer
    
read by default treats \ specially. The switch -r has to be added so that read does not do so. –  pabouk Nov 6 '13 at 22:39

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.