Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Here's an example of a link I found on YouTube in the comments section of a video.

gnu.org/distros/free-distros.h­tml

This is the way it shows up in the comment.

If I highlight this link and copy to clipboard (ctrl+c), then go to a new browser tab and paste it (ctrl+v) in the address bar, then this is how it shows up.

gnu.org/distros/free-distros.h­tml

It looks the same, right? But if I hit Enter I get an error.

404 - Page Not Found

The page you were looking for could not be found on the GNU web server.

If you followed a link that turned out to be broken, and the page with the broken link mentions an explicit address to which to report bugs, please use that address.

The URL also changes to the following.

http://www.gnu.org/distros/free-distros.h%C2%ADtml%EF%BB%BF

If I remove %C2%ADtml%EF%BB%BF and type in tml so that I get back the address http://www.gnu.org/distros/free-distros.html and then hit Enter, well now it works, and the page loads.

I thought to myself that this is very strange so I tried pasting the same text from clipboard to a plain text editor (notepad) and this is what I got.

gnu.org/distros/free-distros.h­-tml

How was the dash between h and tml introduced? This is why I was getting the 404 error. But the URL appears correctly when pasted to the address bar. Is this some kind of hidden character perhaps?

Also, if I go back to YouTube and highlight the link, I can see that there is a bump on the last three letters. The highlighting is taller around "tml". You can see that in the screen capture below.

screen1

screen2

Why is this happening? What's going on? Could it be that Google is somehow intentionally salting the link?

Update

If I paste into Notepad++ (version 6.3) I get following.

gnu.org/distros/free-distros.h­tml?

If I try to paste into the address bar of the Google Chrome browser, there appears to be some kind of hidden character at the end of the URL. See scree capture below.

screen3

That's not a white space. It's something else... something alien! Something from planet X?

Note: The vertical line at the end is not the one I mean, that's just the text input cursor blinking.

Update 2

Inspecting the html code in Firefox by using the element inspection tool.

screen4

Why is there a square within the opening wbr tag?

Update 3

The "square" appears to be the soft hyphen character entity. Here follows the actual source code of this particular line.

<p>gnu.org/distros/free-distros.h<wbr>&shy;tml</p>

The soft hyphen is the &shy; you see here. HTML tags, such as or i.e. for bold text, are not selectable. When you highlight a text of a web page in a browser, you are not selecting the HTML tags. Nothing within <> is shown.

So it seems that soft hyphen is the root cause of the copy and paste issue. It is not displayed on the web page, but it is selected when you highlight the text.

Update 4

This is what it looks like when I paste the URL into Microsoft Word 2010 and view hidden characters.

screen5

To move the text cursor from .|html to .ht|ml requires pressing the arrow key three times. You can tell by the image above why that is. It's because of this hidden character. With the cursor in front of that strange looking character, pressing Alt+X shows 0068. With the cursor behind that character, and in front of the letter T reveals nothing at all. The 0068 is just the Unicode code page for the letter H.

share|improve this question
    
Is it possible to have a link to this Youtube page ? –  Levans Aug 7 '13 at 7:09
    
I am using Firefox 22 on Windows Vista 64-bit SP2. But I just tried pasting into Google Chrome and I still get the 404 error. –  sammyg Aug 7 '13 at 7:10
    
@Levans It's "Richard Stallman Talks About Ubuntu" by Muktware. –  sammyg Aug 7 '13 at 7:11
    
@Levans youtu.be/CP8CNp-vksc –  sammyg Aug 7 '13 at 7:12
    
Lesson learned: soft hyphens are nasty! =) –  sammyg Aug 7 '13 at 9:45
add comment

2 Answers

up vote 2 down vote accepted

Yes it is a nuisance.

There are two hiphens The normal one \u2D, and the funny one. The funny one is used sometimes within youtube comments. \u00AD and comes up as hidden.

Paste into notepad(to remove formatting) and also, notepad shows it, and then into MS Word(or just in Ms Word do paste special..unformatted unicode), put your cursor to the right of the hiphen, or any character, and press ALT-x and you see the ASCII or unicode code for it.

This may seem strange. Be aware that there are a few characters with two different types. A type you use usually which is within the 0-7F range, and a type people tend to not use much or at all, which is >7F. The two types of spaces(a normal one and another called the non-breaking space, ascii code 160 \uA0 which can be of use). There two types of pipes 7C and A6 The A6 one is just asking for problems as it causes failures on the command line. And two types of hiphens, the second one you see, behaves funny too, as youtube comments sometimes use it and hide it and don't display it as a hiphen.

Another funny character I see which is used by youtube in comments is \uFEFF You can run notepad2(download it), choose file..encoding..UTF-8 then paste the text in, and search for \uFEFF replacing with nothing, (check the box that says transform backslashes).

Similarly you can open notepad2, search for \u00AD (that funny hiphen) and replace it with a regular hiphen. Editpad free might be able to do it, though I use the pro version for its regex support.

I'd note that charmap doesn't copy the funny hiphen correctly. (So if you want to experiment and you choose copy and paste it into a piece of software and it displays funny, blame charmap), but it copies fine(as in with the character) from your link in my browser(chrome). Better if the character wasn't there though, it is a nuisance! But you can see the ascii code of it in Ms Word, and you can search and remove it in notepad2

You see from charmap it(\u00AD) is called the "soft Hiphen" (i'm just glad they didn't hiphenate that title!)

In the pic I used Ms Word and did ALT-x

enter image description here

share|improve this answer
    
I look at the source code now and I see <p>gnu.org/distros/free-distros.h<wbr>&shy;tml</p>. So the reason we get this problem is because of the soft hyphen and not so much because of the wbr tag? –  sammyg Aug 7 '13 at 8:37
    
ascii.cl/htmlcodes.htm Hex AD, &shy It's the &shy; that is the soft hiphen(the weird hiphen!) which is the issue. And &shy; is right there in the html you quoted there –  barlop Aug 7 '13 at 8:50
    
If you look at the source in chrome for your question, where you included a failing link, there rather than an &shy; it literally has the soft hiphen there within the letters html but displays nothing for it e.g. paste it into the URL bar so in edit type mode, and move your cursor through it. And if you move your cursor through it(with the arrow keys) you see there's a funny character between h and t of html. I experimented with these things once, you can fit tons of these characters in there consecutively, which show up in one program but in another occupy no space. –  barlop Aug 7 '13 at 9:04
    
You have lost me. Define "edit type mode". I did try pasting into MS Word 2010 and I see that I have to press the arrow key three times to move the text cursor from .|html to .ht|ml. It should be enough to press it two times to move the cursor two steps. This is because there is a hidden character there. –  sammyg Aug 7 '13 at 9:19
    
Also, when the cursor is in front of the t (.h|tml) the Alt+X doesn't show any ASCII code in MS Word. But I can see it by viewing hidden characters in Word (see screen capture above). –  sammyg Aug 7 '13 at 9:30
show 2 more comments

Looking at the source code of this portion of page, I see this :

<p>gnu.org/distros/free-distros.h<wbr>­tml</p>

It seems that Youtube automatically inserted a <wbr> tag. It's a word-break opportunity, it tells the browser that if needed, the word may be broken to insert a newline.

On UTF-8 encoded pages, this is displayed as a ZERO-WIDTH SPACE, not showing anything, but allowing a newline. That's what causes your encoding issue.

It looks like Youtube has an algorithm to auto-insert <wbr>into long words at good places (not cutting a syllabe in two) , but as the http:// was missing at the beginning of the URL, the algorithm didn't recognize it as such, and thus assumed it was a word that could be broken.

share|improve this answer
    
But there is no line break? The dash is not seen on YouTube? Only when copy and pasted? –  sammyg Aug 7 '13 at 7:41
    
There is no line break because it is not needed to display the content, yet the invisible character is still here. The dash on copy-paste is probably the result of poor encoding translation from UTF-8 to the one used by Windows, while the URL is translated to URL encoding, with poor results as well. And I missed something, <wbr> is not supposed to insert a -. I'll correct. –  Levans Aug 7 '13 at 7:46
    
Why is there a dotted square within the opening wbr tag? See the screen capture above. Shouldn't it just say "<wbr>" and nothing else? I would describe the above as "<wbr?>" where the ? marks the position of this strange looking, square like character. That's what I see when I inspect the element inside Firefox. I mean wbr alone should not cause this problem, right? –  sammyg Aug 7 '13 at 8:00
    
Right, "the <wbr> element does not introduce a hyphen at the line break point." –  sammyg Aug 7 '13 at 8:05
    
@Sammy Indeed, I looked a little more, and it seems this URL's encoding is quite screwed, and that is probably what caused Youtube to insert a <wbr>. Probably the one who posted it had an encoding issue with his own computer, and omitting the http:// caused Youtube algorithm to act strangely. –  Levans Aug 7 '13 at 8:09
show 9 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.