3

I set up the following pipeline. This allows me to see the raw HTTP requests and responses that pass through the pipeline.

Note: I use BSD nc below (this is what comes with macOS and AmazonLinux), but I've also tried nmap's ncat, which can take the same arguments, and socat 1.7.3.2 The behavior is the same.
socat takes different arguments, but the pipeline is the same. I'll add the socat one-liner only, as the explanation would be the same.

One-liner:

mkfifo res; nc -kl 8888 < res | tee /dev/stderr | nc google.com 80 | tee res

OR

mkfifo res; socat tcp-listen:8888,reuseaddr,fork - < res | tee /dev/stderr | socat - tcp:google.com:80 | tee res

Explanation:

mkfifo res; \             <-- Make a named pipe file with the name 'res'.
nc -kl 8888 < res \       <-- Open a socket listening on all interfaces at TCP port 8888. Use the 'res' pipe as input.
    | tee /dev/stderr \   <-- Copy the request to stderr, so it's emitted by the terminal. stdout will be used as input for the next netcat.
    | nc google.com 80 \  <-- Resolve google.com, connect to it on TCP port 80, and send what was received on stdin. The response goes to stdout.
    | tee res             <-- Copy the response to the 'res' pipe.

I run this pipeline in one terminal (the pipline terminal), and use curl to make the request in another terminal (the curl terminal):

curl --resolve google.com:8888:127.0.0.1 http://google.com:8888

On both macOS and Linux:

The curl terminal correctly receives and displays the response. The pipeline terminal displays the request AND response.

On macOS (macOS Mojave 10.14.3):

The pipeline continues running. I can send another request with curl, and can see the request in the pipeline terminal and the response in both terminals.

On Linux (AmazonLinux 4.1.13):

The pipeline continues running. curl hangs if I send another request, and I can't see the request or response anywhere.

Versions:

macOS

$ uname -a
Darwin ch007837.na.webmd.net 18.2.0 Darwin Kernel Version 18.2.0: Thu Dec 20 20:46:53 PST 2018; root:xnu-4903.241.1~1/RELEASE_X86_64 x86_64 i386 MacBookPro15,1 Darwin

$ curl --version
curl 7.64.1 (x86_64-apple-darwin18.2.0) libcurl/7.64.1 SecureTransport zlib/1.2.11
Release-Date: 2019-03-27
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile libz NTLM NTLM_WB SSL UnixSockets

$ bash --version | head -1
GNU bash, version 5.0.3(1)-release (x86_64-apple-darwin18.2.0)

$ ncat --version
Ncat: Version 7.70 ( https://nmap.org/ncat )

I'm not sure about nc since it doesn't have a --version flag, but it came with the OS. There's this string in the binary though:

$ strings "$(which nc)" | tail -1
@(#)PROGRAM:nc  PROJECT:netcat-42.200.1

Linux

$ uname -a
Linux ip-10-200-38-72 4.1.13-19.31.amzn1.x86_64 #1 SMP Wed Jan 20 00:25:47 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

$ curl --version
curl 7.40.0 (x86_64-redhat-linux-gnu) libcurl/7.40.0 NSS/3.19.1 Basic ECC zlib/1.2.8 libidn/1.18 libssh2/1.4.2
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz UnixSockets

$ bash --version | head -1
GNU bash, version 4.2.46(1)-release (x86_64-redhat-linux-gnu)

$ ncat --version
Ncat: Version 5.51 ( http://nmap.org/ncat )

$ yum list nc | tail -1 | sed 's/ \+/ /g'
nc.x86_64 1.84-24.8.amzn1 @amzn-main
3
  • Note: on my Kubuntu 18.04.2 LTS the code seems to work like in your "On macOS" case. Apr 2, 2019 at 17:49
  • Thanks. I'm trimming down the list of variables. How are you running Kubuntu? AWS/Virtualbox/Bare metal? Do you know which nc? Apr 3, 2019 at 2:01
  • Kubuntu is directly installed on my laptop. Currently my nc is from netcat-openbsd 1.187-1ubuntu0.1 (it doesn't seem to support --version or such). I get the same "On macOS" behavior with ncat instead of nc. ncat is from nmap 7.60-1ubuntu5, ncat --version yields Ncat: Version 7.60 ( https://nmap.org/ncat ). Apr 3, 2019 at 4:36

0

You must log in to answer this question.

Browse other questions tagged .