15

I am developing a website using a local server, but I have remote dependencies. I would like to fake requests for files from remote servers. For instance, when the browser makes a request for fooCDN.com/bloatedLib.js the contents of /Users/name/Desktop/bloatedLib.js will be returned.

I use a mac and a solution that works at the system or browser level would be fine. If it works at the browser level, only a Firefox or Chrome solution will be alright. I cannot use a windows computer.

13
  • Are you able to use a Windows computer?
    – geek1011
    Aug 9 '15 at 13:21
  • @geek1011 No, that would not be possible
    – Daniel F
    Aug 9 '15 at 13:22
  • 6
    Not sure if I understand you correctly, but are you trying to be able to access a website even when you have no connection to the internet? and if so, access a local copy? Like a proxy server does?
    – LPChip
    Aug 9 '15 at 13:33
  • @LPChip I need external dependencies like jQuery to be loaded even when I am offline. This is someone else's repository, so I can't just make external dependencies local. If it could just load the version in the browser cache when it is offline ti would be perfect. Most of the website is in a local server.
    – Daniel F
    Aug 9 '15 at 13:35
  • You could use HTML5 "offline web apps"
    – Michael
    Aug 9 '15 at 23:35
12

I hacked together this little proxy server to only download missing files. Just set up your /etc/hosts file to point sites you want to cache at 127.0.0.1 and those you want to block at 0.0.0.0

#!/bin/sh 
nc -ll -p 80 -e sh -c ' 
while read A B DUMMY 
do 
   case "$A" in 
      [Gg][Ee][Tt]) 
         FULL=$B #full path of the file
         F=${FULL##*/}
         F=${F%%\?*} #file name only
         #if we have it cat it back to browser
         [ -f "$F" ] && cat "$F" && break 
      ;; 
      [Hh][Oo][Ss][Tt]*) 
         [ -f "$F" ] && break #file already exists
         HOST=${B:0:$((${#B}-1))} #the host name
         #resolve by DNS first so we can cache it
         sed -i "s/hosts:\t\tfiles /hosts:\t\t/g" /etc/nsswitch.conf 
         wget -t 0 -q --no-dns-cache $HOST$FULL
         #got it now revert to checking host file 1st
         sed -i "s/hosts:\t\t/hosts:\t\tfiles /g" /etc/nsswitch.conf
         #cat the file because I didn't think to wget through tee
         cat "$F" 
         break 
      ;; 
   esac 
done 
'

Note that it puts all files in one directory, so it may cause version conflicts. (I did this intentionally so I wouldn't have 500 copies of jquery)

5
  • Can you comment your code so that I can understand it better. I like this answer a lot because I don't need to install a lot of software
    – Daniel F
    Aug 10 '15 at 12:17
  • 1
    @DanielF I added some inline comments. Let me know if it makes sense, it's probably one of the hackiest hacks I've written. I only wrote it out of necessity because I kept getting browser stalls waiting for one busy resource. I should mention I was using puppy Linux when I wrote it, so some lines may need to be run sudo. Aug 10 '15 at 12:38
  • Note: some browsers may need an actual http header instead of just "catting" the file (though none of mine did). If you have a browser that doesn't accept/fix this bad behavior, you can use stat to get the file size and use it to printf a proper header. Aug 12 '15 at 10:55
  • @technosaurus: I'm trying to implement something similar, however I'm getting an error: illegal option -- e, can you help with this? Sep 16 '15 at 11:25
  • @AnuragPeshne there are several varied implementations of netcat; depending on which implementation you are using, you may be able to use the -c parameter for running a script (same as -e without the sh -c part) Sep 17 '15 at 7:45
24

It sounds like you're running a local web server. Excellent.

In your Mac's filesystem, there is a file called /etc/hosts. You can redirect all requests for fooCDN.com to your local machine by adding this line in /etc/hosts:

127.0.0.1   foocdn.com www.foocdn.com

You will need root (super user) permissions to edit /etc/hosts.

The above line means that fooCDN.com will load from your own computer, where a web server is listening.

You didn't specify what web server you're running locally, though. Following the web server's documentation, you should create a virtual host that points the document root of fooCDN.com to /Users/name/Desktop/.

This is a sample configuration (I haven't tested it myself) you can try to use with Apache:

<VirtualHost 127.0.0.1:80>
    ServerName foocdn.com
    ServerAlias www.foocdn.com
    DocumentRoot /Users/name/Desktop
</VirtualHost>

Here's a sample configuration for Nginx (also untested):

server {
    listen 80;
    root /Users/name/Desktop;
    server_name foocdn.com;
}

Don't forget to restart the you web server service or reload the new configuration file.

3
  • Your answer was good, but I have accepted the answer that didn't require my installing anything
    – Daniel F
    Aug 10 '15 at 12:45
  • @DanielF: Oh, so you're not running a local web server?
    – Deltik
    Aug 10 '15 at 12:47
  • 1
    @Deltik Correct. He wants to be able to ask fooCDN.com for a file when he has no Internet connection.
    – BenjiWiebe
    Aug 10 '15 at 12:58
2

You could use proxy server software that supports URL rewriting to accomplish the task. Many proxy server applications support URL rewriting. E.g., the Charles Web Debugging Proxy Application for Windows, Mac OS, and Linux supports URL rewriting. You could install it on your Mac system and then configure the browsers on the system to use the proxy server.

Alternatively, Apache, which is free and open source, has mod_proxy and mod_rewrite modules.

Mitmproxy is free and will also run on a Mac OS X system.

If you need to pull items out of your browser's cache to make them available via the proxy server, you can use techniques provided at Viewing Chrome cache (the easy way). E.g., in Google Chrome you can put chrome:\\cache in the browser's address bar and then locate the relevant items in the Chrome cache and copy them elsewhere.

4
  • Do you know of any software other than Charles, especially free software?
    – Daniel F
    Aug 9 '15 at 14:50
  • Your answer is good, but I accepted the other answer that doesn't require non free software.
    – Daniel F
    Aug 9 '15 at 14:59
  • That's fine; Deltik's solution is a good one for your situation. I added a couple of free alternatives to Charles.
    – moonpoint
    Aug 9 '15 at 15:13
  • 1
    @DanielF Another debugging proxy is Fiddler, which has an alpha OS X build and supports file responses and replaying specific responses. It's more fine-grained than redirecting the whole domain too.
    – Bob
    Aug 10 '15 at 6:01
2

Sounds like good old wwwoffle should suite your needs. It's a proxy server where you can select resources for offline usage.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.