3

Ignoring timestamps, how may one remove non-chat duplicates in the following? The chats have 2 formats

  1. beginning by a nickname surrounded by angle brackets and
  2. starting with a nickname followed by " tells you: ".

I'd prefer if it was done in Notepad++ but I also have access to multiple utilities thanks to Cygwin.

Original

[16:29] You see a sheep; it looks like it weighs about 98.
[16:30] You see a sheep; it looks like it weighs about 100.
[16:52] anonymized tells you: Do you know the bank yet?
[17:11] Only anonymized may access the corpse for now.
[17:12] Only anonymized may access the corpse for now.
[17:14] <anonymized> You can do it later.
[17:14] <anonymized> The dagger for example
[17:15] <anonymized> The dagger for example
[17:15] <dynv> hi
[17:32] gnome has been killed by anonymized and dynv
[17:32] The corpse is too far away.
[17:32] The corpse is too far away.
[17:33] anonymized: now is gets dangerous

Desired result

[16:29] You see a sheep; it looks like it weighs about 98.
[16:30] You see a sheep; it looks like it weighs about 100.
[16:52] anonymized tells you: Do you know the bank yet?
[17:11] Only anonymized may access the corpse for now.
[17:14] <anonymized> You can do it later.
[17:14] <anonymized> The dagger for example
[17:15] <anonymized> The dagger for example
[17:15] <dynv> hi
[17:32] gnome has been killed by anonymized and dynv
[17:32] The corpse is too far away.
[17:33] anonymized: now is gets dangerous

Thank you kindly

2 Answers 2

2
  • Ctrl+H
  • Find what: ^\[.+?] (?!<\w+>|\w+ tells you:)(.+)\K\R\[.+?] \1
  • Replace with: EMPTY
  • TICK Wrap around
  • SELECT Regular expression
  • UNTICK . matches newline
  • Replace all

Explanation:

^                   # beginning of line
    \[.+?]              # time stamp in square brackets followed by a space
    (?!                 # negative lookahead, make sure we haven't after:
        <\w+>               # nickname surrounded by angle brackets 
      |                   # OR
        \w+ tells you:      # nickname followed by " tells you:"
    )                   # end lookahead
    (.+)                # group 1, 1 or more any character but newline, (the text)
    \K                  # forget all we have seen until this position
\R                  # any kind of linebreak
    \[.+?]              # time stamp in square brackets followed by a space
    \1                  # back reference to group 1 (the text)

Screenshot (before):

enter image description here

Screenshot (after):

enter image description here

1
  • Thanks. Yesterday I did it manually taking ~10 mins, and I just ran your regex that showed I missed 1 line.
    – DynV
    Aug 3, 2022 at 19:25
0

I generally have the opinion that a procedural approach is easier to understand than fully regex-based magic, even if it takes more than one line:

#!/usr/bin/env python3
import re
import sys

prev = None
for line in sys.stdin:
    line = line.strip()
    if m := re.search(r"^\[\d+:\d+\] (.+)$", line):
        text = m.group(1)
        if re.search(r"^<\S+> ", text):
            print(line)
        elif re.search(r"^\S+ tells you: ", text):
            print(line)
        elif text != prev:
            print(line)
        prev = text
    else:
        print(line)
#!/usr/bin/env perl

my $last = "";
while (my $line = <>) {
    if ($line =~ /^\[\d+:\d+\] (.+)/) {
        my $text = $1;
        if ($text =~ /^<\S+> / || $text =~ /^\S+ tells you:/) {
            print $line;
        } else {
            print $line unless ($text eq $last);
        }
        $last = $text;
    } else {
        print $line;
    }
}

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .