7

At my work, http://twitter.com is blocked, but https://twitter is not. Is there any way to have all programs/browsers access this URL using the HTTPS protocol instead of HTTP?

This specifically relates to third-part clients which need to access the site, but are usually hard coded to HTTP with no option to change them.

UPDATE: I tried to create a fiddler script as suggested below, but I was unable to get it running correctly. An answer with the script would be most helpful, or another method consisting entirely of accessing.

5
  • Remember, not all websites will work with https (although they should)
    – Josh Hunt
    Jul 15, 2009 at 13:29
  • @joshhunt - good to keep in mind. This would apply specifically to sites that I know the https is working, but the http is not.
    – dancramer
    Jul 16, 2009 at 15:30
  • Do you have administrative access on your work computer? And what version of Windows are you using?
    – Arjan
    Sep 18, 2009 at 22:19
  • Win XP with admin access
    – dancramer
    Sep 21, 2009 at 17:57
  • Ok, then changing the hosts file should be possible, though it feels a bit awkward. If you currently don't need to use some company proxy (and don't mind paying some bugs), then using Charles Web Debugging Proxy like Matthew suggested should be fine as well, if your Twitter applications indeed honour such proxy settings. (And it might be preferred as it would probably not break the certificate chain when explicitly using https, though DeleGate could probably also be run as both a man-in-the-middle HTTP-to-HTTPS gateway, and a normal proxy for applications that use it.)
    – Arjan
    Sep 21, 2009 at 18:13

9 Answers 9

8
+100

I wonder if all third-party applications will actually take proxy settings into account. Or if changing system-wide proxy settings might interfere with other applications. (Like when the company's proxy is required for internet access, so cannot be changed just to redirect traffic for twitter.com.) Luckily, when the application does not use HTTPS, then it cannot find a man-in-the-middle. So: set up a man-in-the-middle for twitter.com on port 80, using DeleGate.

The following steps have been tested on Mac OS X 10.6 and Windows XP, using accounts with full administrative rights.

  1. Download DeleGate. Don't let the 1990's homepage fool you: the program is still maintained.

  2. Tell DeleGate to forward all local requests on port 80 (and 443) to the HTTPS server, based on the value of Host header in the HTTP request. Like for a Mac on Intel (where sudo is required to use privileged ports below 1024):

    sudo ./macosxi-dg -v -P80,443 \
    SERVER=https \
    RELAY=vhost \
    RESOLV=cache,dns \
    STLS=-fcl,fsv \
    [email protected]
    

    For Windows, if unzipped to c:\:

    cd c:\dg9_9_4\bin
    dg9_9_4.exe -v -P80,443 SERVER=https RELAY=vhost RESOLV=cache,dns STLS=-fcl,fsv [email protected]
    

    If you're required to use your company's proxy for internet access, then DeleGate will happily use that if you add something like PROXY=proxy.example.com:8080 to the command line.

  3. In your /etc/hosts file (c:\windows\system32\drivers\etc\hosts on Windows XP; see Wikipedia for locations on other OS's), add the following line to direct all requests for twitter.com to your own computer. Note that the mapping from domain name to IP address does not take the protocol into account. So: this will not only be used for HTTP, but also for HTTPS (and everything else, such as commands like ping).

    127.0.0.1 twitter.com
    
  4. Ensure your browser is not set to use a proxy server, or add twitter.com as an exception. Just in case your browser has cached Twitter's IP address, you might want to restart it.

  5. Now, http://twitter.com actually gets you (and all your applications) https://twitter.com.


The output shows that indeed the HTTPS site is requested from https://twitter.com:

REQUEST - GET / HTTP/1.1
REQUEST = https://twitter.com:443/ GET / HTTP/1.1
[..]
ConnectToServer connected [16] {168.143.161.20:443 <- 192.168.1.68:57067}
## SSLway -- TLSxSNI: sent ru=0 ty=0 nm=localhost
## SSLway ## 0.459622 connected/accepted
## SSLway server's cert. = 
  **subject /
  C=US/
  O=twitter.com/
  OU=GT09721236/
  OU=See www.rapidssl.com/resources/cps (c)09/
  OU=Domain Control Validated - RapidSSL(R)/
  CN=twitter.com
  **issuer /
  C=US/
  O=Equifax Secure Inc./
  CN=Equifax Secure Global eBusiness CA-1


When using MOUNT="/* https://twitter.com/*" instead of RELAY=vhost then even http://localhost would give one https://twitter.com:

Twitter through DeleGate


When explicitly requesting HTTPS using https://twitter.com, then the trusted certificate chain is broken: a HTTPS-aware applicate will discover the man-in-the-middle attack, and will fail if it cannot ask you for your permission to continue:

Man-in-the-middle


After testing, to run as a service on Windows, simply remove the -v parameter. This will install the program as a service. It will then run in the background, and ask you if you want to run it on startup:

Trying to start as a service [DeleGate Server -P80,443] ...
Set Automatic Start on System Startup ? [y] / n :

After running the above command without the -v parameter: see Control Panel » Administrative Tools » Services to manually start or stop DeleGate. Note that this service will refer to the location from which you initially started the dg9_9_4.exe program. So, you should not delete or move that program; be sure to unzip the download to, for example, c:\dg9_9_4 to avoid a reference to some Downloads directory that you might delete in the future.

To remove the service, just ensure to specify the same value for the -P parameter:

dg9_9_4.exe -P80,443 [email protected]
[..]
The service `DeleGate Server -P80,443' exists.  Delete it ? [y] / n : y
OK. DELETEd the previous service.
Create a new service ? [y] / n : n


Finally, one may wonder how DeleGate knows the IP address of twitter.com (as we've mapped that to 127.0.0.1 in the hosts file). DeleGate actually retrieves that itself, because of RESOLV=cache,dns:

MOUNT[5]X[2] /* https://twitter.com/*  
{R} SOA got [162.143.168.in-addr.arpa][ns1.dn.net]
  [dnsadmin.enterprise.verio.net] 2008121001 10800 3600 604800 86400
1
  • Awesome answer, get examples and it worked like a charm. Thanks Arjan!
    – dancramer
    Sep 21, 2009 at 18:07
5

You need some kind of proxy server with a rewrite rule capability. I suspect that, in the absence of a better answer being offered, you could write a modifier rule for Fiddler to do this. Fiddler rules are written in JavaScript and there are several examples online.

Edit: Please see, and vote up, Eric Law's answer on this page.

5
  • I think this is going to be the way to go, I'll down load Fiddler and see how hard it would be to write the redirect script
    – dancramer
    Jul 16, 2009 at 15:26
  • 1
    Ok, I made a good faith effort to get this up and running with no luck. I'm going to need an example of the script to really get this going.
    – dancramer
    Sep 9, 2009 at 19:53
  • 1
    And the proxy server has to be outside the firewall, if the blocking isn't happening at a per-application level on your own machine. Sep 17, 2009 at 10:23
  • 1
    @reinierpost, I'd say the (additional) proxy server needs to be inside the firewall (so: on one's workstation) as it needs to rewrite HTTP to HTTPS prior to trying to pass that firewall.
    – Arjan
    Sep 20, 2009 at 8:50
  • 1
    The Fiddler script is trivial, but the comment system here reformats the code in a comment, so I've added an additional answer with the script.
    – EricLaw
    Nov 6, 2009 at 5:42
3

The Fiddler Rule is pretty simple. Inside OnBeforeRequest, add:

if (oSession.fullUrl.StartsWith("http://twitter.com/"))
{
    oSession.oRequest.headers.UriScheme = "https";
}
1
  • Nice. (Hopefully all third-party applications do honour the system wide proxy settings. If one also needs to use some company proxy to connect to the internet, then apparently Fiddler also supports "Chain to upstream gateway proxy" in its settings.)
    – Arjan
    Nov 6, 2009 at 7:52
2

You could escape all censure by using a VPN and tunneling out of your work network.

VPN tunneling involves establishing and maintaining a logical network connection (that may contain intermediate hops). On this connection, packets constructed in a specific VPN protocol format are encapsulated within some other base or carrier protocol, then transmitted between VPN client and server, and finally de-encapsulated on the receiving side.

What it means is that all your packets are sent via an intermediary on the internet. This intermediary will normally not be blocked by your company's firewall, and it forwards all your packets to their destination, and then returns the received answers to your computer. In addition, the VPN connection ensures that all your packets both ways are encoded and secure from prying eyes.

You will get the best service from commercial products, but some are free and not too bad. For example, HTTP-Tunnel Client is a very good product.

1

It's not free, but the Charles (a web debugging proxy) has a Map Remote Settings function which could be told to map http://twitter.com to https://twitter.com easily using a GUI. You can download a trial version from the site to see if it meets your needs.

alt text

0

Under Firefox, it seems that the add-on Force-TLS answers your problem.
Quote:

Here's how it works:

  1. A site x.com served via HTTPS provides a header X-Force-TLS in its response. The header contains a max-age value (how long to remember the forced TLS) and optionally an includeSubDomains flag.
  2. The browser recieves this header and adds it to a Force TLS database.
  3. In the future, any requests to x.com are modified to be via HTTPS if they are attempted through HTTP before the request hits the network.
  4. If any subdomains *.x.com are requested via HTTP and the includeSubDomains flag was set, they are also forced to be HTTPS.
5
  • A solution for all browsers is quite complex. I believe that the OP prefers a simple solution.
    – harrymc
    Sep 17, 2009 at 12:27
  • 1
    The OP doesn't need a solution for any browser (one can easily manually type https in a browser). It's about software that only connects to HTTP, and in which the URL cannot be changed.
    – Arjan
    Sep 17, 2009 at 12:57
  • I believe he rather connects through https to a page containing http links that he needs to convert to https, since http to that site is blocked.
    – harrymc
    Sep 17, 2009 at 13:06
  • 1
    Well, he does write This specifically relates to third part[y] clients which need to access the site, but are usually hard coded to http with no option to change them. ...
    – Arjan
    Sep 17, 2009 at 13:37
  • You might be right - I've added another answer that addresses all programs on the computer, using VPN tunneling.
    – harrymc
    Sep 17, 2009 at 14:11
0

You could write a Greasemonkey script with Firefox. Here's one for FriendFeed.

Edit: Script for Twitter.

2
  • 1
    Doesn't help him with external clients.
    – crb
    Jul 15, 2009 at 13:23
  • Ah right. Well if anyone wants to do it locally you can get the script here userscripts.org/scripts/show/53744
    – Stephen
    Jul 15, 2009 at 13:47
-1

I think there is not =/

Somebody can show up a way to do it, but seeing this as a software developer: many applications have http://twitter.com hard-coded, and since not only the address, but the protocol is different, I don't see a way to do it automagically, without changing the application. Some application may work with both, so you can configure it.

Note: web applications are more hacky than desktop ones.

EDIT: good, crb. A proxy server, depending on the way Twitter is being blocked, could help him.

1
  • 1
    That's where a proxy server comes in handy :)
    – crb
    Jul 15, 2009 at 13:24
-1

A different solution, maybe, would be to use a client, like DestroyTwitter or TweetDeck. These don't use twitter.com directly, but tweet and read through an API which is probably not blocked.

2

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .