I have bunch of CSV files to process daily. They have a constant form with 19 COMMAS to appear in every file. In the file, there is a mixture of strings, timestamps, integers and float digits. The file always ends with CR-LF set.
Each file has only 1 line of that text, that provides a set of information that is then processed later on.
Problem appears that from time to time (once every 1000 files) the device that is responsible for generation of that file creates INCOMPLETE file. Some information is missing.
I need to separate those files from all the good files before processing them further.
After spending some considerable amount of time trying to figure out the most robust way to resolve that I came to the conclusion, that since the file is incomplete, it has some COMMAS missing.
So I want to count the amount of COMMAS appearing in the CSV file. Correct file should have 19 commas, any problematic files that I have discovered for the last half year would have LESS than that. They also have CR-LF missing at the end of the line. They simply end with comma after some value and that is it.
Example of a good CSV file:
STRING1,STRING2,2017-01-20 17:34:08,53.808536,-7.789231,19.5,3,0,STRING3,2017-01-20 17:34:19,2,0,7.9,2,0,1,0,0,0,0
Example of incorrect CSV file:
STRING1,STRING3,2017-01-12 10:11:09,53.779093,-7.494274,
My question is: How can I count amount of COMMAS in a single CSV file, and if this amount doesn't match defined number to perform a certain operation (like move this incorrect file into ERRORS folder, that I could deal with later). If the file has the correct amount of COMMAS, just let it be. So another words I need to move Incorrectly formatted CSV files to an ERRORS folder and leave the correctly formatted CSV files as they were.
FYI: CSV files have always only 1 line of data.
I have experimented with certain solutions, that are trying to find and count instances of certain strings in a TXT file, but it is getting me nowhere. Maybe because my string is just a COMMA sign... I don't know.
I would much appreciate you help in that matter.