VPN works on one Mac but not another

Question

I'm trying to set up a Cisco VPN on two macs using the native mac VPN. I followed exactly the same steps. One one machine, it works, and the system.log shows something like

Apr 26 10:18:25 ... racoon[29960]: Connecting.
Apr 26 10:18:25 ... racoon[29960]: IKE Packet: transmit success. (Initiator, Aggressive-Mode message 1).
Apr 26 10:18:25 ... racoon[29960]: IKEv1 Phase1 AUTH: success. (Initiator, Aggressive-Mode Message 2).
Apr 26 10:18:25 ... racoon[29960]: IKE Packet: receive success. (Initiator, Aggressive-Mode message 2).
Apr 26 10:18:25 ... racoon[29960]: IKEv1 Phase1 Initiator: success. (Initiator, Aggressive-Mode).
Apr 26 10:18:25 ... racoon[29960]: IKE Packet: transmit success. (Initiator, Aggressive-Mode message 3).
etc.

On the other machine however, I get a message saying "a config error occurred. verify your settings and try reconnecting" and system.log shows:

Apr 26 11:06:00 ... configd[13]: SCNCController: Disconnecting. (Connection tried to negotiate for, 0 seconds).
Apr 26 11:06:00 ... [0x0-0x3e63e6].com.google.Chrome[7358]: 2011-04-26 11:06:00.348 Google Chrome Helper[10521:107] __CFServiceControllerBeginPBSLoadForLocalizations received error 1100 from bootstrap_look_up2
Apr 26 11:06:00 ... [0x0-0x3e63e6].com.google.Chrome[7358]: 2011-04-26 11:06:00.348 Google Chrome Helper[7506:107] __CFServiceControllerBeginPBSLoadForLocalizations received error 1100 from bootstrap_look_up2

Any idea what is going on?

Answer 1

This sometimes happens with me as well... Most of the times, the solution is found by restarting a process called racoon.

First, shut down racoon by opening Terminal and using following commands:

$ ps -ef | grep racoon
    0  3385     1   0   0:00.12 ??         0:00.22 /usr/sbin/racoon
  501  3968  3964   0   0:00.00 ttys000    0:00.01 grep racoon
$ sudo kill 3385
$ ps -ax | grep racoon

Be sure racoon is shut down. If it doesn't shut down, try sudo kill -9 3385 instead

Next, restart racoon:

$ sudo /usr/sbin/racoon
$ ps -ef | grep racoon
    0  3995     1   0   0:00.00 ??         0:00.01 /usr/sbin/racoon -x
  501  3998  3964   0   0:00.00 ttys000    0:00.00 grep racoon

Now try again to connect with the VPN client.

Answer 2

I had a similar issue where VPN creds were working on Windows but not Mac after an upgrade to Mac OSX 10.8.2 (Mountain Lion), had the following error in system.log "IKE Packet: transmit failed",

...
Jun  7 08:23:38 mac1.local racoon[418]: Connecting.
Jun  7 08:23:38 mac1.local racoon[418]: IPSec Phase1 started (Initiated by me).
Jun  7 08:23:38 mac1.local racoon[418]: IKE Packet: transmit failed. (Initiator, Aggressive-Mode Message 1).
...

The quick fix for me was to identify what was bound to IKED then close them down, try again, and it now works,

sudo lsof -i
...
iked        63           root   15u  IPv4 0x830bba8d64129e4b      0t0    UDP *:isakmp
iked        63           root   16u  IPv4 0x830bba8d64129cc3      0t0    UDP *:ipsec-msft
...

sudo kill 63

tail system.log
...
Jun  7 10:27:09 mac1 kernel[0]: utun_ctl_connect: creating interface utun1
Jun  7 10:27:09 mac1.local racoon[1066]: IKE Packet: transmit success. (Initiator, Quick-Mode message 1).
Jun  7 10:27:09 mac1.local racoon[1066]: IKE Packet: receive success. (Initiator, Quick-Mode message 2).
Jun  7 10:27:09 mac1.local racoon[1066]: IKE Packet: transmit success. (Initiator, Quick-Mode message 3).
Jun  7 10:27:09 mac1.local racoon[1066]: IKEv1 Phase2 Initiator: success. (Initiator, Quick-Mode).
Jun  7 10:27:09 mac1.local racoon[1066]: IPSec Phase2 established (Initiated by me).
Jun  7 10:27:09 mac1.local configd[16]: IPSec Phase2 established.

Answer 3

Are these two computers on the same LAN behind a NAT router?

Assuming as such:

Often SOHO NAT routers will only allow one VPN out to an external network at a time. Try disconnecting the working computer's VPN connection, reboot the router (or wait a while for it's routing tables clear) and try the second computer's VPN connection.

Answer 4

I had a similar problem. Here is what seemed to help in my case.

Before you connect the VPN, start a ping to some machine that is on the other side of the VPN. Of course, each ping attempt will fail at first.

Then, connect your VPN.

Doing this caused the VPN to connect successfully (and immediately).

Before, I was getting the dreaded Disconnecting. (Connection tried to negotiate for, 23.376395 seconds) message after several seconds of trying to complete the connection.