Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

At my work, http://twitter.com is blocked, but https://twitter is not. Is there any way to have all programs/browsers access this URL using the HTTPS protocol instead of HTTP?

This specifically relates to third-part clients which need to access the site, but are usually hard coded to HTTP with no option to change them.

UPDATE: I tried to create a fiddler script as suggested below, but I was unable to get it running correctly. An answer with the script would be most helpful, or another method consisting entirely of accessing.

share|improve this question
    
Remember, not all websites will work with https (although they should) –  David Pearce Jul 15 '09 at 13:29
    
@joshhunt - good to keep in mind. This would apply specifically to sites that I know the https is working, but the http is not. –  dancramer Jul 16 '09 at 15:30
    
Do you have administrative access on your work computer? And what version of Windows are you using? –  Arjan Sep 18 '09 at 22:19
    
Win XP with admin access –  dancramer Sep 21 '09 at 17:57
    
Ok, then changing the hosts file should be possible, though it feels a bit awkward. If you currently don't need to use some company proxy (and don't mind paying some bugs), then using Charles Web Debugging Proxy like Matthew suggested should be fine as well, if your Twitter applications indeed honour such proxy settings. (And it might be preferred as it would probably not break the certificate chain when explicitly using https, though DeleGate could probably also be run as both a man-in-the-middle HTTP-to-HTTPS gateway, and a normal proxy for applications that use it.) –  Arjan Sep 21 '09 at 18:13
add comment

9 Answers

up vote 6 down vote accepted
+100

I wonder if all third-party applications will actually take proxy settings into account. Or if changing system-wide proxy settings might interfere with other applications. (Like when the company's proxy is required for internet access, so cannot be changed just to redirect traffic for twitter.com.) Luckily, when the application does not use HTTPS, then it cannot find a man-in-the-middle. So: set up a man-in-the-middle for twitter.com on port 80, using DeleGate.

The following steps have been tested on Mac OS X 10.6 and Windows XP, using accounts with full administrative rights.

  1. Download DeleGate. Don't let the 1990's homepage fool you: the program is still maintained.

  2. Tell DeleGate to forward all local requests on port 80 (and 443) to the HTTPS server, based on the value of Host header in the HTTP request. Like for a Mac on Intel (where sudo is required to use privileged ports below 1024):

sudo ./macosxi-dg -v -P80,443 \
      SERVER=https \
      RELAY=vhost \
      RESOLV=cache,dns \
      STLS=-fcl,fsv \
      ADMIN=a@b.c

For Windows, if unzipped to c:\:

cd c:\dg9_9_4\bin
dg9_9_4.exe -v -P80,443 SERVER=https RELAY=vhost RESOLV=cache,dns STLS=-fcl,fsv ADMIN=a@b.c

If you're required to use your company's proxy for internet access, then DeleGate will happily use that if you add something like PROXY=proxy.example.com:8080 to the command line.

  1. In your /etc/hosts file (c:\windows\system32\drivers\etc\hosts on Windows XP; see Wikipedia for locations on other OS's), add the following line to direct all requests for twitter.com to your own computer. Note that the mapping from domain name to IP address does not take the protocol into account. So: this will not only be used for HTTP, but also for HTTPS (and everything else, such as commands like ping).

    127.0.0.1 twitter.com
  2. Ensure your browser is not set to use a proxy server, or add twitter.com as an exception. Just in case your browser has cached Twitter's IP address, you might want to restart it.

  3. Now, http://twitter.com actually gets you (and all your applications) https://twitter.com.


The output shows that indeed the HTTPS site is requested from https://twitter.com:

REQUEST - GET / HTTP/1.1
REQUEST = https://twitter.com:443/ GET / HTTP/1.1
[..]
ConnectToServer connected [16] {168.143.161.20:443 <- 192.168.1.68:57067}
## SSLway -- TLSxSNI: sent ru=0 ty=0 nm=localhost
## SSLway ## 0.459622 connected/accepted
## SSLway server's cert. = 
  **subject /
  C=US/
  O=twitter.com/
  OU=GT09721236/
  OU=See www.rapidssl.com/resources/cps (c)09/
  OU=Domain Control Validated - RapidSSL(R)/
  CN=twitter.com
  **issuer /
  C=US/
  O=Equifax Secure Inc./
  CN=Equifax Secure Global eBusiness CA-1


When using MOUNT="/* https://twitter.com/*" instead of RELAY=vhost then even http://localhost would give one https://twitter.com:

Twitter through DeleGate


When explicitly requesting HTTPS using https://twitter.com, then the trusted certificate chain is broken: a HTTPS-aware applicate will discover the man-in-the-middle attack, and will fail if it cannot ask you for your permission to continue:

Man-in-the-middle


After testing, to run as a service on Windows, simply remove the -v parameter. This will install the program as a service. It will then run in the background, and ask you if you want to run it on startup:

Trying to start as a service [DeleGate Server -P80,443] ...
Set Automatic Start on System Startup ? [y] / n :

After running the above command without the -v parameter: see Control Panel » Administrative Tools » Services to manually start or stop DeleGate. Note that this service will refer to the location from which you initially started the dg9_9_4.exe program. So, you should not delete or move that program; be sure to unzip the download to, for example, c:\dg9_9_4 to avoid a reference to some Downloads directory that you might delete in the future.

To remove the service, just ensure to specify the same value for the -P parameter:

dg9_9_4.exe -P80,443 ADMIN=a@b.c
[..]
The service `DeleGate Server -P80,443' exists.  Delete it ? [y] / n : y
OK. DELETEd the previous service.
Create a new service ? [y] / n : n


Finally, one may wonder how DeleGate knows the IP address of twitter.com (as we've mapped that to 127.0.0.1 in the hosts file). DeleGate actually retrieves that itself, because of RESOLV=cache,dns:

MOUNT[5]X[2] /* https://twitter.com/*  
{R} SOA got [162.143.168.in-addr.arpa][ns1.dn.net]
  [dnsadmin.enterprise.verio.net] 2008121001 10800 3600 604800 86400
share|improve this answer
    
Awesome answer, get examples and it worked like a charm. Thanks Arjan! –  dancramer Sep 21 '09 at 18:07
add comment

You need some kind of proxy server with a rewrite rule capability. I suspect that, in the absence of a better answer being offered, you could write a modifier rule for Fiddler to do this. Fiddler rules are written in JavaScript and there are several examples online.

Edit: Please see, and vote up, Eric Law's answer on this page.

share|improve this answer
    
I think this is going to be the way to go, I'll down load Fiddler and see how hard it would be to write the redirect script –  dancramer Jul 16 '09 at 15:26
1  
Ok, I made a good faith effort to get this up and running with no luck. I'm going to need an example of the script to really get this going. –  dancramer Sep 9 '09 at 19:53
1  
And the proxy server has to be outside the firewall, if the blocking isn't happening at a per-application level on your own machine. –  reinierpost Sep 17 '09 at 10:23
1  
@reinierpost, I'd say the (additional) proxy server needs to be inside the firewall (so: on one's workstation) as it needs to rewrite HTTP to HTTPS prior to trying to pass that firewall. –  Arjan Sep 20 '09 at 8:50
1  
The Fiddler script is trivial, but the comment system here reformats the code in a comment, so I've added an additional answer with the script. –  EricLaw Nov 6 '09 at 5:42
add comment

The Fiddler Rule is pretty simple. Inside OnBeforeRequest, add:

if (oSession.fullUrl.StartsWith("http://twitter.com/"))
{
    oSession.oRequest.headers.UriScheme = "https";
}
share|improve this answer
    
Nice. (Hopefully all third-party applications do honour the system wide proxy settings. If one also needs to use some company proxy to connect to the internet, then apparently Fiddler also supports "Chain to upstream gateway proxy" in its settings.) –  Arjan Nov 6 '09 at 7:52
add comment

You could escape all censure by using a VPN and tunneling out of your work network.

VPN tunneling involves establishing and maintaining a logical network connection (that may contain intermediate hops). On this connection, packets constructed in a specific VPN protocol format are encapsulated within some other base or carrier protocol, then transmitted between VPN client and server, and finally de-encapsulated on the receiving side.

What it means is that all your packets are sent via an intermediary on the internet. This intermediary will normally not be blocked by your company's firewall, and it forwards all your packets to their destination, and then returns the received answers to your computer. In addition, the VPN connection ensures that all your packets both ways are encoded and secure from prying eyes.

You will get the best service from commercial products, but some are free and not too bad. For example, HTTP-Tunnel Client is a very good product.

share|improve this answer
add comment

It's not free, but the Charles (a web debugging proxy) has a Map Remote Settings function which could be told to map http://twitter.com to https://twitter.com easily using a GUI. You can download a trial version from the site to see if it meets your needs.

alt text

share|improve this answer
add comment

Under Firefox, it seems that the add-on Force-TLS answers your problem.
Quote:

Here's how it works:

  1. A site x.com served via HTTPS provides a header X-Force-TLS in its response. The header contains a max-age value (how long to remember the forced TLS) and optionally an includeSubDomains flag.
  2. The browser recieves this header and adds it to a Force TLS database.
  3. In the future, any requests to x.com are modified to be via HTTPS if they are attempted through HTTP before the request hits the network.
  4. If any subdomains *.x.com are requested via HTTP and the includeSubDomains flag was set, they are also forced to be HTTPS.
share|improve this answer
    
A solution for all browsers is quite complex. I believe that the OP prefers a simple solution. –  harrymc Sep 17 '09 at 12:27
1  
The OP doesn't need a solution for any browser (one can easily manually type https in a browser). It's about software that only connects to HTTP, and in which the URL cannot be changed. –  Arjan Sep 17 '09 at 12:57
    
I believe he rather connects through https to a page containing http links that he needs to convert to https, since http to that site is blocked. –  harrymc Sep 17 '09 at 13:06
1  
Well, he does write This specifically relates to third part[y] clients which need to access the site, but are usually hard coded to http with no option to change them. ... –  Arjan Sep 17 '09 at 13:37
    
You might be right - I've added another answer that addresses all programs on the computer, using VPN tunneling. –  harrymc Sep 17 '09 at 14:11
add comment

You could write a Greasemonkey script with Firefox. Here's one for FriendFeed.

Edit: Script for Twitter.

share|improve this answer
1  
Doesn't help him with external clients. –  crb Jul 15 '09 at 13:23
    
Ah right. Well if anyone wants to do it locally you can get the script here userscripts.org/scripts/show/53744 –  Stephen Jul 15 '09 at 13:47
add comment

I think there is not =/

Somebody can show up a way to do it, but seeing this as a software developer: many applications have http://twitter.com hard-coded, and since not only the address, but the protocol is different, I don't see a way to do it automagically, without changing the application. Some application may work with both, so you can configure it.

Note: web applications are more hacky than desktop ones.

EDIT: good, crb. A proxy server, depending on the way Twitter is being blocked, could help him.

share|improve this answer
1  
That's where a proxy server comes in handy :) –  crb Jul 15 '09 at 13:24
add comment

A different solution, maybe, would be to use a client, like DestroyTwitter or TweetDeck. These don't use twitter.com directly, but tweet and read through an API which is probably not blocked.

share|improve this answer
1  
The API uses HTTP (and the twitter.com domain name) just like a browser would. Like twitter.com/statuses/public_timeline.xml or twitter.com/statuses/user_timeline/codinghorror.xml?count=5 –  Arjan Sep 17 '09 at 16:12
    
it depends on how they block, if they block twitter.com it will work, if they block it by keyword or by twitter.com/* it will fail, it've gotten both in various places –  alpha1 Sep 19 '09 at 22:19
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.