Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I've googled and looked and explored, but it seems most editors configurable syntax highlighting definitions are based on keywords and such. I need one that is based on columns and lines, basically something to highlight different fields in a flat file.

It needs to be more complicated than one line fits all however, there are several different "types" of lines.

A sample is shown below

A9999000055555 333333     55554444422222210102009000000333333
B44444555550000
B44444555550000
B44444555550000
B44444555550000
C9999        0004

Except for "0004" (# of B records) and "10102009" (a date) all other sequential numbers would be actual alpha-numerics representing different user input fields. The first letter is usually designated as a Line type.

The intent of this question is to allow humans to verify the "correctness" of the files format. Typically these files are normally created in an automated way, but business rules change, and old mainframe programs have to be updated (and what ends up happening is not pretty). So the idea is to give those who create these files a way to double-check themselves that the file meets the format. I do not intend for ANY business rule verification here, purely format.

share|improve this question
1  
Can you post a sample of the flat file so we can see the structure? Put several lines, if some depend on the previous ones. –  Snark Oct 15 '09 at 20:12
    
What type of platform are you comfortable working on? Does it need to be a Windows solution? I would usually lean toward sed/awk/vim for this sort of thing, because it's what they're really good at, being designed when all computer data looked like your flat file. One more question: What's the purpose of editing such a file? Are you taking it somewhere, like into a database or putting it back in something old, like a mainframe application? –  atroon Dec 22 '09 at 22:03
    
The format itself has been a standard for years, and will not be going away anytime soon, because many customers still have mainframes. For this reason, I was hoping to give customers (and me) an "easy" way to view and verify the format of the file before they send it. Windows would be the preferred platform. –  hova Dec 23 '09 at 17:35

10 Answers 10

You could write a script to add HTML tags to do the coloring. sed (works on lines) and awk (works on columns) might help, especially on linux. Or a perl script. You could use regular expressions to do pattern matching.

You'll also have to add the minimal HTML code to convert it to a HTML file (header and body tags).

share|improve this answer
    
The intent is not to make HTML, but to provide guidelines when editing the actual files. –  hova Oct 16 '09 at 18:44
    
Editing in what ? Vim (regex), notepad2 (primitive parsers), ... ? –  ldigas Oct 16 '09 at 22:27
    
@Idigas Editing in ANYTHING that is able (potentially) to perform the above and also save it (sans highlighting markup). Just a simple text editor. –  hova Oct 19 '09 at 17:08

Vim! Start with there instructions for changing syntax highlighting and make your way from there. Here's what I think you're saying:

  • If a line starts with "A" it will have a series of 12 characters; a space; six chars; some spaces; many chars
  • If a line starts with "B" it will have a series 14 chars;
  • If a line starts with "C" it will have 4 chars; some spaces; 4 chars

So you can definitely define your highlighting rules based on those requirements.

share|improve this answer
1  
It will take me a month to surmount VIM's Learning Cliff, and that's just starting out, not to even get started on highlighting –  hova Dec 22 '09 at 17:12
    
Okay, so you don't like vim. But it would work in this situation. –  Amanda Mar 20 '10 at 23:24

I asked about what the intended use of such a utility would be and based on a couple keywords ("view and verify before they send") you basically want to do the following:

  1. Make sure the file is in standard format with the right number of columns (and rows, maybe)
  2. Give some stats like total number of records, total number of items sold, date range represented in file, etc, etc.
  3. Save the file to either a central server or to a waiting area for upload later.

The right way to accomplish this, in my opinion, (especially because, as you say, many customers still have mainframes and still get data in formats like this) is to use a custom front-end to get the information about the data to your customer.

The way to do that? You have essentially 2 options: either a) write a file-parser yourself in Java, C#, or C++ (shudder) to give a 'preview' of the data contained in the files. Or b) hire a member or recent graduate (check a portfolio first!) of computer programming classes to code a parser for you. Or the boss' nephew, or your sister's nerdy boyfriend, etc. I don't know what experience or interest you have in such a thing, nor anything about your background, so please forgive me if I'm making undue assumptions. As with so many other things in the world of Technology, you can have any two of fast, cheap, and good.

The lowest cost and fastest option of those presented so far, which span the range of possibilities I'm aware of as a technology professional, would be to bite the bullet, learn some vim (use GVim for Windows) and use syntax highlighting filters there. Second would be to make a series of scripts in something like sed or awk.

Since the input to your problem is essentially a stream of text chopped off at intervals (the line breaks) the processing of the data has to handle input in the same way, and very few programs do so now, because, as you say, this is data generated by an old mainframe program.

Excel might also be of use in importing, but all the lines have to be formatted the same way, so that's still not going to do what you want. One additional thing that comes to mind is that you might be able to use Access to parse such a file and use some VBA to create record counts and compile statistics, but as far as overlaying highlights, that would be less easy. If you know any COBOL developers, this would be a great 1-2 night project for one of them. It might even turn into an open source project on sourceforge.net to free data from the clutches of the mainframe!

share|improve this answer
    
#1 Is the most important part of this, the intent being to ensure that the file meets the format and errors are easy to spot. #2 is not so accurate as the total # of records and such are part of the file itself, and do not need to be computed, just highlighted. #3 is way beyond what I was looking to do, but is rather what will happen to the file via the customers own means (usually automated). I was hoping I would not have to code up a solution myself in C#, but alas, it surprises me that given the time and duration of flat files there is not such a thing already. –  hova Dec 23 '09 at 20:55
    
Ah, but it's not surprising...it's human nature. 'This data came out of the computer, therefore, it's correct' is a surprisingly prevalent attitude. Also 'If there's anything wrong with this file, they'll catch it at HQ'. My sympathies. –  atroon Dec 23 '09 at 21:13

I think you're unlikely to find a solution that doesn't involve programming.

Any text editor with syntax-highlighting features more sophisticated than simple keyword matching will necessarily be more complicated, as it is in Vim and Emacs.

For Vim, you can define a highlighting file like this:

syntax clear A B C

syntax match A /^A\%(\a\|\d\)\{13\} \d\{6\} \{5\}\S\+$/
syntax match B /^B\%(\a\|\d\)\{14\}$/
syntax match C /^C\%(\a\|\d\)\{4\} \{8\}\d\{4\}$/

highlight A guifg=darkgreen
highlight B guifg=darkblue
highlight C guifg=royalblue

and load it with :source myhighlight.vim

Then the lines will be highlighted when they're correct, or left unhighlighted if they don't match the regular expression.

It gets more complicated if you want to highlight particular fields within those lines.

Personally, I'd be looking at ways of invoking some script from the editor, which could do all the checking for me. That way, you could use any editor and not be forced to use one which has the highlighting flexibility you need.

share|improve this answer

If I wasn't going to code a front-end for the file, I would probably use Notepad++ as it has a way to define your own language, complete with highlighting.

Download / Main site

All about the user-defined language section

Another example / walkthru

Hopefully these will help you get started.

share|improve this answer
1  
Yeah, I've slogged through that already, it's useless as it's based on keywords. I have no keywords, just positions and a few identifying letters. –  hova Dec 22 '09 at 17:00
    
Ah.. in that case your best bet may either by the VIM solution or simply creating a custom front-end UI for this file. –  mpeterson Dec 22 '09 at 22:26

It looks like your columns are space delimited?

Have you tried just importing them into Excel, and then just applying highlighting/colours to the columns?

If that won't cut it then as others say, you're probably looking at programming. You could probably write a fairly simple Perl or VBScript that'll take that text file and output a syntax highlighted HTML file or similar using something like regexes to match the patterns that you're looking for?

share|improve this answer

You could try something like Monarch

share|improve this answer

I had a similar requirement and I found the solution using one of the samples given in ":sytanx help" screens.

Below is the sample flat file with 2 records of same type. Here record header is "01" which is the first column. After header record there are 4 columns with fixed lenghts given below:

+-----+-----+-----+-----+-----------------+
| No. | Beg | End | Size| Description     |
+-----+-----+-----+-----+-----------------+
|    1|    1|    2|    2|HEADER           |
|    2|    3|    7|    5|Column 2         |
|    2|    8|   11|    4|Column 3         |
|    2|   12|   15|    4|Column 4         |
|    2|   16|   17|    2|Column 5         |
+-----------------------------------------+

Sample records:

01AAAAABBBBCCCCDD
01PPPPPQQQQRRRRSS

" Syntax definition and highlighting for Record 01
:sy region rec1 matchgroup=rec1 start="^01" end="$" contains=r1col1,r1col2,r1col3,r1col4 keepend
:hi rec1 ctermbg=red guibg=red

" Syntax definition and highlighting for next 4 columns from Record 01
:sy region r1col1 matchgroup=r1col1 start="\%3v" end="\%8v" contained
:sy region r1col2 matchgroup=r1col2 start="\%8v" end="\%12v" contained
:sy region r1col3 matchgroup=r1col3 start="\%12v" end="\%16v" contained
:sy region r1col4 matchgroup=r1col4 start="\%16v" end="\%18v" contained

" Highlighted alternate columns with same color
:hi r1col1 ctermbg=green ctermfg=black
:hi r1col2 ctermbg=green guibg=green
:hi r1col3 ctermbg=darkblue ctermfg=white
:hi r1col4 ctermbg=green ctermfg=black

This way we can define "regions" for more records (say Record starts with "02") with different number and widths of columns which may present in the same flatfile but will get highlighted based on the record header.

share|improve this answer
1  
What software are you referring to here in you answer? –  slm Jan 8 '13 at 6:25

You could look at the RecordEditor RecordEditor it is not a Text editor (it displays the data in a table). It also has single record display + other views available.

The default is to select the record layout but the there is a "prefered layout" which will display each record using the correct layout for the record.

It does allow you to visually check the file

share|improve this answer

if you are still looking for an answer, Text Master Data Editor might be of some help.

share|improve this answer
    
Welcome to Super User! Generally we like answers on the site to be able to stand on their own - Links are great, but if that link ever breaks the answer should have enough information to still be helpful. Please consider editing your answer to include more detail. See the FAQ for more info. –  slm Apr 4 '13 at 18:11

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.