Recursive Batch File

Question

I have a file that looks this:

head1,head2,head3,head4,head5,head6
a11,a12,keyA,a14,a15,a16
a21,a22,keyB,a24,a25
a31,a32,keyC,a34
a41,a42,keyB,a44,a44
a51,a52,keyA,a54,a55,a56
a61,a62,keyA,a64,a65,a66
a71,a72,keyC,a74
some message

Objective: Write list of unique keys to a text file. For example, the result for the file described above should be:

keyA, keyB, keyC

Here's the pseudocode I would like to implement in batch file recur.bat

1. Read second line of inputfile
2. If no key exist on second line, return else continue
3. Append keyX to list
4. FINDSTR /v keyX inputfile
5. Pipe results to recur.bat

I don't know if this is the most efficient way to do this without using actual programming language.

Any suggestions for actual batch file code?

Answer 1

^{Separate answer because this is, essentially, a different solution}

So, here's a cmd/batch version.

@echo off
type NUL>output.txt
for /f "tokens=1,3 delims=, skip=1" %%a in (input.txt) do (
    if "%%b"=="" goto :eof
    findstr "^%%b$" output.txt > NUL
    if ERRORLEVEL 1 echo %%b>>output.txt
)

It creates output.txt, then reads input.txt using for and , as a delimiter. The first line is skipped.

The first token (tokens=1,3) needed to be read to stop it on the some message line, since it would just skip the line and continue if the requested token (tokens=3) did not exist - and would never run the if. %%a is the first requested token (1), %%b is the second (3).

a71,a72,a73,a74
a71,a72,keyC,a74
some message
a71,a72,keyD,a74

keyD will not be read, since it stops at the line before (does not have a third token). However, a73 will be counted.

This uses findstr on the output file to check if the key is already in there - not efficient at all, but it works. The RegEx is used to make sure it matches the whole line; if one key is contained entirely within the other key it won't match (i.e. keyA will not match key). Output is piped to NUL to keep things quiet.

And if the key is not in the output file, it will append it.

Answer 2

You could use PowerShell, which comes with Windows Vista or later:

$keys = @( );
Import-Csv input.txt | ForEach-Object { 
    if (!$_.head3) {
        $keys | Out-File output.txt;
        break;
    } else {
        if (!($keys -contains $_.head3)) {
            $keys += $_.head3;
        }
    }
}

This may be slow on large amounts of data, since it is using an array ($keys) to hold and check unique keys. An alternative method is to write everything into a text file, sort it, and run it through Get-Unique. Another alternative is to use a hashtable (wouldn't help with memory usage, but would be faster than checking if something exists in an array).

This uses Import-Csv, which will take the first line as the headings. It then passes an array of objects (lines) to ForEach-Object. $_ is a variable referencing each object (line). .head3 is the property with the name head3, as defined in your example data as the column containing the keys. It checks if there is a value for this column in this line; if not it outputs to a file and quits as per your pseudocode. Note that non-key values may be accepted. If you have/need stricter rules for what is a key, you can check length, or do some RegEx pattern matching, etc..

a71,a72,a73,a74
a71,a72,keyC,a74
some message
a71,a72,keyD,a74

Currently, a73 counts as a key (it's in the third column, head3). The program will end at some message, since it does not have a third column, and will not read keyD.

If the line does have a key column, it checks if the key already exists in the array and, if not, adds it. Note that -contains is case insensitive. If this is a problem, it can be changed.

So you'll probably have to replace input.txt output.txt and head3 with the correct names. This was the simplest solution that does not modify the order of the data, though faster ones are possible if necessary.

Answer 3

In Bash (you can use Cygwin if you are on Windows) this is easy:

1.) replace , by newline (\n)

sed -i 's/,/\n/g' superuser.txt

Before you had:

head1,head2,head3,head4,head5,head6 a11,a12,keyA,a14,a15,a16 a21,a22,keyB,a24,a25 a31,a32,keyC,a34 a41,a42,keyB,a44,a44 a51,a52,keyA,a54,a55,a56 a61,a62,keyA,a64,a65,a66 a71,a72,keyC,a74

Now you have:

head1
head2
head3
head4
head5
head6 a11
a12
keyA
a14
a15
a16 a21
a22
keyB
a24
a25 a31
a32
keyC
a34 a41
a42
keyB
a44
a44 a51
a52
keyA
a54
a55
a56 a61
a62
keyA
a64
a65
a66 a71
a72
keyC
a74

2.) Search for "key", sort the results and eliminate duplicates

grep -F key superuser.txt | sort | uniq

Gives you:

keyA
keyB
keyC