Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I want to parse some information from the User-Agent: HTTP header. The problem is that I'm getting two User-Agent: HTTP headers in the same HTTP Request:

Proxy-Connection: keep-alive
User-Agent: Mozilla/5.0 ( AppleWebKit/536.6 (KHTML, like Gecko) Chrome/20.0.1092.
Proxy-Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.6 (KHTML, like Gecko) Chrome/20.0.1092.

I want the regex to match the non-http portion e.g Windows NT 6.1; WOW64. The flow analyzer software I'm using java regex engine.

My attempts


Its matching both; I want to skip http portion of it.

share|improve this question
Why do you want a regex when a string function is 10x faster in a case like this? – Ramhound Sep 26 '13 at 13:04
I cannot use string function its not programmable im using closed source solution. – asadz Sep 26 '13 at 13:28
Reg expressions are not universal. This sounds like a Stackoverflow question. I would post the criteria then post what you have tried. The syntax required to do this is simple but every programming language has its own way of doing something like this. – Ramhound Sep 26 '13 at 13:36
@Ramhound I have updated the question. – asadz Sep 26 '13 at 17:49
What have you tried? What version of java regex engine are we talking about? – Ramhound Sep 26 '13 at 18:43
up vote 4 down vote accepted

There are several problems with this question:

  • You haven't told us what programming language you are using, or at least which flavor of regular expressions. POSIX? PCRE? Something unusual?

  • In what context are you using this? Are you running an HTTP server, and clients are submitting requests like these? Or are you running a proxy that is accepting these requests and analyzing them? Which proxy are you using -- is it something you've written by hand (in what language?) or something you downloaded?

  • It's not clear from your post that you're getting two User-Agent: headers specified in a single request. In your post, you seem to have some line break problems (which I've edited), but it appears that you are getting three separate HTTP requests, and it's perfectly okay to have different User-Agent: in each HTTP request. If you are actually getting input from a client like the data you were seeing in your original post (with NO linebreak after the User-Agent: and before the next CONNECT), then someone is trying to fool your proxy on purpose by omitting the linebreak, and attempting to do HTTP pipelining through your proxy. A standards-compliant implementation will have linebreaks, even if the HTTP Request Body is empty.

  • Trying to parse things using regular expressions is dangerous. At the very least, you need to be prepared to handle arbitrary input. There is no universally agreed-upon standard for the formatting of the User-Agent: string, and many public Internet hosts that connect to a public web server will attempt to "fuzz" the server by sending it malformed requests. It is perfectly possible for a client to submit headers such as the following, even all in a single request if they want:

    User-Agent: Lol your regex won't match this User-Agent: Mozzarella/0.-0 ((Let's )()confuse your reg((ex))((()) ApplePie/-NaN (FUZZ, like Police) 3.14159\EvilAttacker ) User-Agent: Mozilla/5.0 (Windows LOL 973.1; LOL64) ...

  • Trying to understand a client's user-agent and then glean useful information from that is stupid. Because the client can set it to whatever they like, you should not consider a given user-agent to mean anything at all. It's like if I tell you "I have a million dollars!" -- do you have any particular reason to believe me? I'm simply alleging that fact. User-Agent: is no different. A Windows 95 box could allege to be Windows 8.1 running IE 11. A Mac OS X 10.8 box could allege to be Red Hat Enterprise Linux 5 running Firefox 2.0. Someone deliberately trying to fool with you could allege that their operating system is "AsadzOS version 0.1".

There are very few reasonable uses of the User-Agent: header, and in each of those use cases, it is essential that you handle all possible matching failures, false positive, false negatives, etc. very carefully, and prevent any unintended behavior in your parser by giving up if you can't make sense of the input.

On top of all this, this is a question for StackOverflow, as it is completely off-topic on SuperUser to be discussing programming techniques. I just needed to post this long-form answer to explain to you why this is a bad question.

share|improve this answer
It feel like you have overstated or over explained a very simple question,and not a bad question, I have submitted multiple time questions regarding regex and I got excellent help. I mentioned in my question , that these packets, payloads or header information I'm getting is from flow analyzer box. Its closed source. I don't have api interface to program it, I can just write regex which are using java engine at backend. I don't why overstretch it so much.:O – asadz Sep 26 '13 at 17:54
I'm NOT writing a white paper ; infact for every single point you mentioned i can write 100 or 1000 lines of counter argument why i need the regex and things about user-agent tampering is not applicable to my use-case; but then thats another fight that I can do on emails or chat. – asadz Sep 26 '13 at 18:03
If you have access to a "flow analyzer" box (presumably a tap into raw ethernet or IP packets for a large network), why are you asking for such a simple regex? Try using Regex Pal. Learn the concepts behind regular expressions, and you won't have to ask a question every time you need to develop a regular expression. Furthermore, this question is still bad, because you didn't even give us a regex you tried to write to capture the OS information from the User-Agent:. I'd be more inclined to help if you gave a non-working regex and explain how it fails. – allquixotic Sep 26 '13 at 18:30
@Ramhound neither Have i written more then 10 lines comment for any regex question I have ever posted on this site; but then I have been lucky also as others have not stretched every single details to atomic levels. – asadz Sep 26 '13 at 18:53
@asadz - This question requires more then 10 lines to explain. I can't judge your previous questions because I have not seen them, and if they were as poorly explained as this one, you don't want me to go voting on those questions. – Ramhound Sep 26 '13 at 18:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .