Questions tagged [parsing]
The parsing tag has no usage guidance.
108
questions
34
votes
10
answers
15k
views
How can I parse extremely large (70+ GB) .txt files?
I have several .txt files with >30 million lines each, and anywhere from 20 to 40 "columns" (some comma-separated, some space-separated, all ASCII with lines separated by a new-line). I ...
- 457
20
votes
8
answers
10k
views
Automatic parsing of citation text in academic references
Is there any software (or pseudo-code) which can automatically scan a piece of text (either pasted into the tool, or read from a .doc/.pdf) and identify citation data using standard formats? The data ...
- 864
16
votes
8
answers
185k
views
Copy/Pasting data from SQL Server to Excel splits up text into multiple columns?
I've got a problem pasting data from the result grid of SQL Server 2005 to an Excel 2007 spreadsheet.
I have a query in SQL Server that returns 2 columns (a number column and a text column). On one ...
- 803
8
votes
5
answers
44k
views
sed: extracting value of a key-value pair in a URL query string
I am trying to use sed to extract the value part of one of the many key-value pairs in a URL's query string
This is what I am trying:
echo 'http://www.youtube.com/watch?v=abc&g=xyz' | sed 's@^...
- 583
8
votes
3
answers
55k
views
Import json data into Excel
I have a text file in json format and want to read it into Excel. A very simplified example of the json file has the following structure:
{ [
{ 'a': 10, 'b': 20 },
{ 'a': 20, 'b': 22 },
{ 'a': ...
- 201
7
votes
8
answers
1k
views
Remove linux file named with set of shell responsive characters
I've created a file named \;:$"\' to test a software of mine. I ended up with an error, because I cannot delete the file property. I'm trying to find a precise character combination to remove it via ...
- 227
7
votes
4
answers
23k
views
Extracting a list of values from JSON file to Excel or a text file
I want to extract usernames from a JSON data file.
[{"username": "Cobra", "user_id": 146231486, "event_type": 2,
"title": null, "class_id": 4211, "war_state" : null,
"superpower_expire_date": ...
- 71
7
votes
1
answer
8k
views
Is the ".sha256" file format formally defined somewhere? How should it be parsed?
I see a bunch of FOSS projects which have ".sha256" files. They look something like this:
dsdfdfdsffdfsdfdsfdsfdsfdsfdsfds23r2ewrefdefdsfdsgfdsgffgfkgdfgg *meow.exe
...
- 71
6
votes
3
answers
2k
views
Breaking a file down of strings, into separate files each based on the first letter. BASH
Alright, so I have a file full of thousands of strings. Each one on it's own line. I want to make a script that will allow me to take this file, call it list.txt, and take the items from each line, ...
- 1,774
6
votes
2
answers
28k
views
Powershell: Get inital character string from file name and create directory from string, then move files
I have a folder with the following file names:
00150005D201110172338427995.vpf
00150005D201110180005318058.vpf
00150013D201110180014448082.vpf
00150013D201110180022268098.vpf
...
- 61
5
votes
3
answers
36k
views
How append a string at the end of all lines?
I'm trying to add a string at the end of all lines in a text file, but I have a mistake somewhere.
Example:
I've this in a text file:
begin--fr.a2dfp.net
begin--m.fr.a2dfp.net
begin--mfr.a2dfp.net
...
- 53
4
votes
5
answers
36k
views
How to split a text file into multiple text files
I have a text file called entry.txt that contains the following:
[ entry1 ]
1239 1240 1242 1391 1392 1394 1486 1487 1489 1600
1601 1603 1657 1658 1660 2075 2076 2078 2322 2323
2325 2740 2741 2743 ...
- 736
4
votes
2
answers
5k
views
Excel Does Not Parse Quoted String containing comma in CSV File Correctly
I have a CSV file that with quoted string that contains comma. When I open it in excel it still uses the comma inside the strings and parse it in different columns. Can someone help?
T7, 7, "331-8463 ...
- 1,688
4
votes
3
answers
12k
views
How to use grep to match domain names?
I have a list of urls like:
hxxp://url.com/subpage.html
hxxp://www.url2.com/index.php
hxxp://subdomain.url3.com/somepage.php
...
How can I use grep to match the domain names only?
All the urls have ...
- 53
4
votes
1
answer
2k
views
Shell scripting: Way to parse parameters from command line
myscript [-a a-arg] [-c c-arg] [-b] [-e] somedirectory
Given that I want my shell script be invoked at the command line using the above parameters - where [these brackets] denote that they are ...
- 2,091
4
votes
1
answer
2k
views
How do I delete BOTH lines of a duplicate pair in a text file in bash?
I want to delete both lines of a duplicate pair in a text file.
There is no way to do this using sort -u file or awk '!a[$0]++' file which both delete duplicated lines.
Somehow I would have to ...
- 51
4
votes
1
answer
4k
views
How can I parse an XML file from the command line (for GeekTool)?
I'd like to find a Terminal command that can pull the file at http://api.twitter.com/1/statuses/user_timeline.xml?screen_name=SOMEUSERNAME&count=1 and parse it to find a user's Twitter status. ...
- 1,176
4
votes
1
answer
3k
views
How do I parse file paths separated by a space in a string?
Background: I am working in Automator on a wrapper to a command line utility. I need a way to separate an arbitrary number of file paths delimited by a single space from a single string, so that I may ...
- 41
3
votes
4
answers
43k
views
How can I get the line count of a text file?
I don't want it to delete the rest or anything but just to quickly report the number of lines the text I just entered is included.
How can I quickly find "the number of lines in this text that ...
- 2,566
3
votes
1
answer
5k
views
Machine readable list of files in rar archive
I need a way to get a parsable list of all files in a .rar archive without extracting them, in bash or python.
What I've tried:
rar l *.rar
7z l -slt *.rar
I've also looked at patool in python, but ...
- 133
3
votes
2
answers
2k
views
Match exactly (and only) the pattern I specify in a grep command
Usually, grep searches for all lines containing a match for the pattern/parameter I specify.
I would like to match just the pattern (i.e. not the whole line).
So, if a file contains the lines:
We ...
- 3,521
2
votes
3
answers
1k
views
Extracting links from a numeric range of web pages
I would like to extract links from a numerical sequence of pages like this:
http://example.com/page001.html
http://example.com/page002.html
http://example.com/page003.html
...
http://example.com/...
user237698
2
votes
3
answers
2k
views
Excel notation that gives FIND the ability to look for multiple strings
So I've been working with a spreadsheet that has a list of schools names. I pull data from this spreadsheet to create a roster that limits the size of text in each cell. In order to get the school ...
- 6,101
2
votes
2
answers
290
views
Get first two strings of each filename in a directory as a txt file
I need to clean up strings like this (where I read in a group of image names), where I only want the first two strings (currently I'm using, in Windows 7 cmd line - dir /a/b/p > textfile.txt):
Acaena ...
- 167
2
votes
4
answers
771
views
Parsing command line output delimited by underscore
I've got some command which output looks like that:
some_command
Current view: username_token1_token2_token3_4_token4_2
How could I parse the "token3_4_token4_2" part out of the string?
- 135
2
votes
1
answer
544
views
What does this grep command actually do?
I'm trying to get a hang of grep. I've got the following command from a GeekLet script that someone made for getting the weather info off some website:
curl -s 'http://thefuckingweather.com/?zipcode=...
- 123
2
votes
2
answers
1k
views
Batch-convert lit -> txt
Is there a tool for Windows to batch-convert a folder of .lit files to a different format, preferably plain-text?
- 175
2
votes
1
answer
4k
views
How to properly unescape strace output
How do I unescape strings outputted by strace? I see some advices to use printf '%b' for that, but that doesn't work correctly for me in some cases. Consider echo -ne '\037\061':
$ strace -e write ...
- 135
2
votes
1
answer
11k
views
Parsing and reformatting date string using PowerShell
Given a CSV-file containing the following line:
HEAD;1;49999;8-10-2017;;;
.. I need to reformat the date given as d-m-yyyy into fixed length dd.mm.yyyy and/or yyyy-mm-dd (fixed, naturally implying ...
- 239
2
votes
3
answers
9k
views
Parse string using batch script
How can I parse string using batch script?
Goal is to save in a array everything under Import: and strip out #head for example --> //MPackages/Project/config/abc.txt and //Packages/Project/config/cde....
- 127
2
votes
1
answer
41k
views
Batch to extract a word from a string inside a text file
I have a database.txt file with this content:
40001 16 DATAMAN Jimbo WS2
I want to extract the third word, as delimited by whitespace ("DATAMAN" in ...
- 35
2
votes
2
answers
7k
views
Difference between an executable and data file format?
A few quick things out of the way as I know these type of questions often have a malicious background.
I am not trying to execute something in a file format (jpg)
I am not trying to simply achieve ...
- 599
2
votes
2
answers
205
views
Deleting list of files with non-standard file names
I have recently had to rescue files from a bad HDD, and am in the middle of cleaning up the mess. Currently I'm using a duplicate cleaner to delete any restored files that have duplicates on my backup....
- 23
2
votes
1
answer
2k
views
Is it possible to fetch and parse email using mailutils+shell scripting only in Ubuntu server headless 15.10?
In my hoping to minimize the number of packages each needing their own configurations, I'm wondering if there is a way for me to retrieve email for parsing by script without installing any more email ...
- 83
2
votes
1
answer
909
views
Deleting music file duplicates but retaining iTunes metadata
Back when I first imported the majority of my iTunes library I had the copy files to iTunes Media folder when adding to library option checked, so basically my whole music library was duplicated on ...
- 201
2
votes
0
answers
427
views
seach for a word in .txt files and write few line into new .txt file
Please help me to create a script to perform task as per described below. i have 2 files. A.txt and B.txt
Content of A.txt as per below
ITEM
name TICKY
title nice coffe drink
type DRINK
ITEM
...
- 21
2
votes
1
answer
851
views
Counting duplicates lines from a stream
I'm currently parsing apache logs with that command:
tail -f /opt/apache/logs/access/gvh-access_log.1365638400 |
grep specific.stuff. | awk '{print $12}' | cut -d/ -f3 > ~/logs
The output is a ...
- 131
2
votes
2
answers
2k
views
Parse incoming mail in Thunderbird?
I need to parse the title of incoming mails and do some simple checks like "does it include this word/number".
I'm familiar with AutoHotkey, batch, PHP...
Any ideas? I can't seem to find a plugin ...
- 249
1
vote
2
answers
992
views
Parsing the result of a command in a Linux bash script
I am writing a small bash script in Linux to control Pulseaudio inputs/outputs.
What I want to do is to route all sink-inputs, except one, into one or another sink. The sink-input I don't want to ...
- 141
1
vote
3
answers
2k
views
How Do I Parse a String?
I am new to bash, and I am creating a script that loops through the files in a directory and based on part of the filename, does something with the file, so far I have this:
#!/bin/bash
DIR="/Users/...
- 879
1
vote
1
answer
4k
views
Get variable values from config file
I have one config file with some variables eg below but my command only read some variable value, I cant read all values in config file. Seems only read values with numbers.
what is better way to read ...
- 476
1
vote
2
answers
171
views
How to retrieve a name from another file with a Bash script?
I would like to grab a name out of another predefined file and use it within my shell script.
The file being read would be in ./docs/description.org
and the format (a table in .org format) is:
......
1
vote
3
answers
8k
views
Fast extraction of key-value pairs in Linux command-line
I have a large (tens of GBs) text file with data in key=value;foo=bar;baz=quo format.
The number and order of keys can change from line to line.
I need to process this file and extract values for ...
1
vote
2
answers
601
views
Parsing two input files and comparing line existence
I need to take two text files and check to see if a line that exists in File A doesn't exist anywhere in File B.
Each line in File A that does not exist in File B should be copied to an output log.
...
- 11
1
vote
1
answer
106
views
Pulling dollar amounts from DoD Contract Awards
I'm going through Department of Defense press releases (here) looking for dollar amounts greater than $500 million. Each document is between 1,000 and 15,000 words, with dollar amounts inline with ...
- 289
1
vote
3
answers
9k
views
How to parse a webpage into a text or Excel file?
I'm trying to figure out a way to grab 4 - 5 lines of text from multiple pages of the same site, the information is always in the same place in the webpage. How would I go about getting that ...
- 189
1
vote
2
answers
529
views
Capturing and parsing xterm cut buffer in emacs
I'd like to make an indirect control for Emacs (in Ubuntu Linux) so that I can swipe some text in an xterm with the cursor, cutting it into the X cut buffer, then run a macro in Emacs to parse the ...
- 177
1
vote
1
answer
41
views
Is it theoretically possible to create a log parser/analyzer with the quality of manual parsing without use of an AI? [closed]
I am trying to setup a good log analyzer for our tomcat application. I was able to create a basic parser and analyzer with python regex and panda stats feature.
The parser basically parses the ...
- 277
1
vote
1
answer
244
views
Parsing errors due to special characters in dns txt records
I have trouble parsing dns txt records retrieved by a script I wrote. The script takes a records, ns records and txt records and writes them to a .csv file. I am using semicolon to separate entries ...
1
vote
1
answer
217
views
How to find all websites matching the pattern website.edu/~<some_string>?
It's common among university lecturers to have their own web pages on the university domain and share resources there. If the name of a professor is John Smith then he his website may be harvard.edu/~...
- 13