0

I have some entries in notepad++ ive never seen before. I have highlighted squares on many lines with SGCI SSA PU1 PU2 MW and SPA within which if copy/pasted here all translate to/become ``

I'm looking for a way to remove these from the entries. Google indicates they are "control characters" but there are far too many for me to try and remove manually.

tried [\x00-\x09\x0B-\x0C\x0E-\x1F] but apparently it did not/does not cover the offending characters.

enter image description here

here is an example of some lines just in case my original post wasn't clear enough.

enter image description here

Here is the file itself.

https://www.dropbox.com/s/lymgnxy9p0atp8x/TrophyID.txt?dl=0

  • Is it possible for you to share the file or part of it for others to have a look? Because replacing them the way @Toto proposed should have worked. – Reddy Lutonadio Jul 3 at 21:14
  • Yes, im afraid it's due to user error somehow. he's helped me several times and is always spot on. I added a link to the file to the OP. – klepp0906 Jul 3 at 22:01
  • @klepp0906 Please do not link to a direct file download, instead pasting the file's contents onto PasteBin, then linking to it. (Linking to a direct file download is an unnecessary security risk for any trying to help.) – JW0914 Jul 4 at 12:24
  • notepad++ has a menu entry for encoding which lets you change to UTF-8. The answers are well-meaning but ultimately not that useful. – Thomas Dickey Jul 4 at 12:54
  • @JW0914 roger that! – klepp0906 Jul 5 at 13:15
3

All these characters are UTF8

  • Ctrl+H
  • Find what: [\x{0080}-\x{0099}] or [\x00-\x09\x0B-\x0C\x0E-\x1F]
  • Replace with: LEAVE EMPTY or whatever you want
  • CHECK Wrap around
  • CHECK Regular expression
  • Replace all

Explanation:

[                 # character class
    \x{0080}        # from character http://www.fileformat.info/info/unicode/char/0080/index.htm
  -               # upto
    \x{0099}        # character http://www.fileformat.info/info/unicode/char/0099/index.htm
]                 # end character class
[                 # character class
    \x00-\x09       # hex 00 to 09
    \x0B-\x0C       # hex 0B to 0C
    \x0E-\x1F       # hex 0E to 1F
]                 # end character class

You can adapt the range to fit exactly your needs.

Screenshot (before):

I've taken some lines from your example file.

enter image description here

Screenshot (after):

Here I've used XXX as replacement to see where the replacement has been done.

enter image description here

| improve this answer | |
  • this did the trick, perhaps i should have provided more information in the OP from the get-go. Once again, thank you for your time and help! – klepp0906 Jul 4 at 11:57
  • 1
    @klepp0906: You're welcome, glad it helps. – Toto Jul 4 at 12:35
  • @Toto Great answer, but its layout/presentation is discombobulated. – JW0914 Jul 5 at 13:28
2

SGCI or 'Single Graphic Character Introducer' (U+0099) and PU2, or rather 'Private Use Two' (U+0092) are part of the 'Latin-1-supplement' block which goes from [\x80-\xFF]. Here you can see all the characters in this block.

So to remove both SGCI and PU2 you need to find:

[\x99\x92]

Replace by nothing.

| improve this answer | |
  • I've extented to more characters, there are also STS, CCH and some other. – Toto Jul 4 at 10:15
  • 1
    thank you for the reference. I'll slowly pick up this gnarly regex stuff. that reference in comparison to your expression and toto's affords me the ability to manipulate what is kept and what is omitted in the future. filed away! I flagged his as the answer as it evidently came in first but both work perfectly. – klepp0906 Jul 4 at 11:57
1

It is possible that "Show All Characters" and/or "Show White Space and TAB" are enabled. Disable them by going to View -> Show Symbol, then selecting them.

IMG:

| improve this answer | |
  • unfortunately they were not, thank you though. I'll know to look here if I get any related funny-business in the future. – klepp0906 Jul 3 at 20:51
0
  • Ctrl+H
  • Find what: [\x00-\x09\x0B-\x0C\x0E-\x1F]
  • Replace with: LEAVE EMPTY
  • CHECK Wrap around
  • CHECK Regular expression
  • Replace all

Explanation:

[                   # character class
    \x00-\x09           # hexa 00 to 09
    \x0B-\x0C           # hexa 0B to 0C
    \x0E-\x1F           # hexa 0E to 1F
]                   # end character class

Screenshot (before):

enter image description here

Screenshot (after):

enter image description here

| improve this answer | |
  • unfortunately this did not work. 0 occurrences were replaced. unsure if relevant - but your before screenshot shows none of the highlighted strings/characters that are giving me trouble. I assumed the regex blanket/covered them all - evidently not the case. Please let me know if anymore information will help! I edited my OP with a few images. – klepp0906 Jul 3 at 20:55
  • @Toto Why did you post a second answer that's the same as the first? Different regex values doesn't equal a new answer. – JW0914 Jul 5 at 13:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.