5

So I have a log file from Twitch chat and I want to make a word cloud, to do this I only want the texts after their name eg.

[Jun 01 2015 02:23:58 UTC] swimsphinx: test1
[Jun 01 2015 02:24:15 UTC] swimsphinx: test2
[Jun 01 2015 02:27:36 UTC] swimsphinx: lol

I want to process the whole file so that the date and name are removed so i'm left with 3 lines

test1
test2
lol

Is this possible in windows CMD? Or is there something else i should do?

3
  • Would you mind running powershell instead of CMD? – tanantish Jun 16 '15 at 6:19
  • sure, anythingg – friend man Jun 16 '15 at 6:29
  • You can do this with any programming language with the power of regex – Dudemanword Jun 16 '15 at 16:46
6

(EDIT: Just for clarity, as noted by David, this isn't going to work in the generic windows cmd interpreter. This is more of a powershell one-liner)

You could do something like this:

cat chat.log | %{$_ -replace "\[[^]]*\] +[^ ]*: ",""} > words.txt

Translated: read the contents of chat.log, then pipe that into a command to replace anything that looks like a timestamp/username thingthe front of each line and put the result into words.txt

I took a guess as well, that for odd cases like:

[Jun 01 2015 02:24:15 UTC] swimsphinx: test2 [Jun 01 2015 02:24:15 UTC] swimsphinx: message

you want to get

test2 message

rather than

test2 [Jun 01 2015 02:24:15 UTC] swimsphinx: message

6
  • @friendman "Is this possible in windows CMD?" - Please share how this works in cmd ... this look like a bash (or cywin) solution which is not what you asked for. – DavidPostill Jun 16 '15 at 6:49
  • @DavidPostill I asked in the comments if powershell was an option (and apparently, it is) and the question is "Is this possible in windows CMD? Or is there something else i should do?" – tanantish Jun 16 '15 at 6:54
  • @DavidPostill Just for clarity as well, added an edit to my answer to make it clear I'm not going for a cmd solution. This is a powershell solution – tanantish Jun 16 '15 at 7:00
  • Fair enough. It wasn't obvious to me that it was a powershell script :( – DavidPostill Jun 16 '15 at 7:01
  • @DavidPostill I must admit it's pretty terse, *NIX-like syntax (and regexps are never that pretty) but I've found it a surprisingly handy pattern for throwaway things like this. – tanantish Jun 16 '15 at 7:06
1

Similar to DavidPostill's answer, but you can use a wildcard to remove everything before a substring.

@echo off

setlocal enabledelayedexpansion
for /f "tokens=*" %%a in (%1) do (
    set line=%%a
    echo !line:*swimsphinx: =!
)
endlocal

Pass the name of the text file into the script when you call it (or replace %1 with your text file name).

1

How do I filter specific text from a log file?

[Jun 01 2015 02:23:58 UTC] swimsphinx: test1
[Jun 01 2015 02:24:15 UTC] swimsphinx: test2
[Jun 01 2015 02:27:36 UTC] swimsphinx: lol

If the lines are always in the same format you can use the following batch file.

test.cmd:

echo off
Setlocal EnableDelayedExpansion
for /f "tokens=4 delims=:" %%a in (log.txt) do (
  set _txt=%%a
  echo !_txt:~1! >> filtered.txt
  )

input:

C:\test>type log.txt
[Jun 01 2015 02:23:58 UTC] swimsphinx: test1 abc
[Jun 01 2015 02:24:15 UTC] swimsphinx: test2
[Jun 01 2015 02:27:36 UTC] swimsphinx: lol

C:\test>

output:

C:\test>type filtered.txt
test1 abc
test2
lol

C:\test>

notes:

  • hard coded with input file log.txt and output file filtered.txt, you could pass these names into the batch file as parameters.
  • with the format you provided the text you want to extract is the 4th token when the delimeter is a colon (:).
  • set _txt=%%a followed by !_txt:~1! is used to strip the space following the 4th :.
0

Here comes my personal choice using PythonPy. Works across multiple OS and you can reuse your Python knowledge. Even if you don't know Python learning it would be easier than spending time with Awk & Sed stuffs.

Here's how you need to start.

  1. Install PythonPy for windows

    pip install pythonwpy

  2. Use this command type test | wpy -x 'print(x.split(" ")[-1])' to parse your log file.

Here is what I tried and got:

nehemiah@neo ~> type test.txt
[Jun 01 2015 02:23:58 UTC] swimsphinx: test1
[Jun 01 2015 02:24:15 UTC] swimsphinx: test2
[Jun 01 2015 02:27:36 UTC] swimsphinx: lol
nehemiah@neo ~> type test.txt | wpy -x 'x.split(" ")[-1]'
test1
test2
lol

3. You can learn much more exciting features for Windows here and for Linux/OSX here

0

Not a cmd solution and loving Powershell but this is actually one of vim's fortes.

%s/.*:_ 

and you're done.

_ being the space character

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.