I need some help with our VPN solution, if you would be so kind :)

The full error message is

Remote Access error 789 - The L2TP connection attempt failed because the security layer encountered a processing error during initial negotiations with the remote computer

This only happens on one single users PC, and only when he is on his home network. No other clients is having issues.

I've tried to change the network profile from Public to Private. Did not help. I've also tried to change it to Domain network, and that didn't help either.

We've got Symantec Antivirus on all machines, but again, there hasn't been any troubles like this on ANY other client.

The firewall SEEMS to be turned off when on Private and Domain network, but not on Public. But this is controlled by the antivirus.

I haven't got immediate access to his router, and would prefer if this could be leaved "untouched"

The connection is a L2TP/IPsec, requires encryption, with a Pre-shared key. The clients use a batch, which is using rasdial to connect.

The server itself, is a Meraki firewall.

Please, ask any questions if you have any.

  • I assume the client is on Windows - which version? Error 789 is a generic error thrown when IPSec negotiation fails. The reasons can be a bad key, firewall blocking ports 500 or 4500, or in Control Panel > Administrative Tools > Services the service named "IKE and AuthIP IPsec Keying Modules" is not started - in that case start it and if that's the problem change its Startup type to "Automatic". Same for the "IPsec Policy Agent" service.
    – harrymc
    Nov 9 '16 at 10:01
  • It is Windows 7 (pro i believe). The key is good, i've double-triple checked that. I will check the firewall settings, but that is controlled by the Antivirus. Which every other client has the same version of. I will check the services also. Thank you! Nov 9 '16 at 10:49

It can be a protocol incompatibility (AES/3DES, etc).

  • Check the firewall settings
  • UDP port 500 and 4500 should be NATed and 1701 forwarded
  • Run services.msc as administrator.
  • Find “IKE and AuthIP IPsec Keying Modules” and “IPsec Policy Agent” Check the status, right click to “restart” if it states “started” If the “started” option is disabled, enable it.
  • Right click and scroll down and click on Properties Select the “Startup type” and change it to “Automatic” and save Restart your VPN and it should work smoothly now as the protocol settings should reset to default
  • If that does not work, you will have to manually set the encryption method for both server and clients so they are compatible.
  • I will try that. Thank you! But doesn't it seem odd, when he can connect via VPN, if he is anywhere but on his home network? Nov 9 '16 at 10:46
  • Default type of protocol encryption can be changed by various apps so I never exclude this issue when debugging VPN connections.
    – Overmind
    Nov 11 '16 at 9:34
  • Didnt seem to help. The client still can't connect.. Have you got any more up your sleve? Nov 14 '16 at 8:22
  • On the problem-client, go to the IPsec Settings tab. IPsec defaults, enter Customize. Under Data protection, select Advanced, and then Customize. Under Data integrity and encryption, select an algorithm that is compatible with the destination. If that is good, use an external port checker to see if the ports are open.
    – Overmind
    Nov 14 '16 at 10:43
  • I will check that, thank you :) Also i told the guy to send me his home network public ip. The i can check the ports. Port 500, 4500 and 1701 should be open? Or am i getting that wrong? Nov 14 '16 at 12:20

From: Fix Windows 10 VPN error 789 connection failed due to security issues:

Remote Access error 789 pops up when your system is not properly set up to connect to an L2TP server, thus the connection attempt fails even before you establish a connection with the server.

It is also linked to incorrect configuration of your operating system like Windows 10 in this case. This generic error is thrown when the IPSec negotiation fails for the L2TP/IPSec connections.

Other possible causes include:

  • L2TP based VPN client (or VPN server) is behind NAT.
  • Wrong certificate or pre-shared key is set on the VPN server or client.
  • Machine certificate or trusted root machine certificate is not present on the VPN server.
  • Machine Certificate on VPN Server does not have ‘Server Authentication’ as the EKU.

Here are solutions you can use to fix Windows 10 VPN error 789 on your computer.

  1. Reset network adapter
  2. Check the certificate
  3. Re-enable IPSec on your computer

Before trying any of these solutions, ensure that L2TP and IPSec pass-through options are enabled from your router. If you configured your VPN service manually, then make sure you use the preshared key.

  1. Reset network adapter

    • Right-click Start and select Device Manager.

      Right-click Start and select Device Manager

    • Find Network adapters and click to expand the list

      Find Network adapters and click to expand the list

    • Identify your network adapter and right-click on it, then select Uninstall.

    • Click OK.
    • Restart your computer. The device will reinstall and should reset it to default settings.

    If this doesn’t fix error 789, try the next solution.

  2. Check the certificate

    Ensure the correct certificate is used both on the client and the server side. In case Pre Shared Key (PSK) is used, ensure that the same PSK is configured on the client side, and the VPN server machine.

  3. Re-enable IPSec on your computer

    • Right-click Start and select Run.

      Right-click Start and select Run

    • Type services.msc

    • Find 'IKE and AuthIP IPSec Keying Modules'.

      Find 'IKE and AuthIP IPSec Keying Modules'

    • Find 'IPSec Policy Agent'.

      Find 'IPSec Policy Agent'

    • Check the status. If it says 'started' click to restart. If the 'started' option is disabled, enable it.

    • Double-click on each of the two.
    • Select Startup type.

      Select Startup type

    • Change it to Automatic.

      Change it to Automatic

    • Save the changes.

    • Restart your VPN service.

Once you have done all the steps above carefully, the VPN should work smoothly as the protocol settings have been reset to default. If, however, it doesn’t work, you have to manually set the encryption method both for the server and the client side, in order for them to be compatible.

In case you have a user-specific issue on your computer yet you still get error 789 after trying any of the above solutions, you can also contact the customer care or tech support team for your specific VPN provider and share the details for further assistance.

  • 2
    "LT2P based VPN client is behind NAT" - is this requirement for real? Practically, wouldn't almost all VPN clients be behind NAT? Mar 18 '20 at 21:40

In my case, the network administrator changed the vpn server configuration, so I got error 789, after changing my vpn config from VPN Type L2TP to the correct PPTP or Automatic, it worked.


Though this solution is implied by a number of other answers, the only thing I had to do is described by the third option on this blog. I copy it here in case the link goes bad:

  • IKE and AuthIP IPsec Keying Modules disabled: Solution: This occurs most often when 3rd party VPN software has been installed and disables the IKEEXT service. This can be re-enabled by navigating in Windows to Control Panel > Administrative Tools > Services. Find the service named “IKE and AuthIP IPsec Keying Modules” and open it. Change the Startup type to “Automatic”. it may be necessary to remove the 3rd party VPN software.

In my case, I didn't have to uninstall any 3rd party VPN software. I happened to be running Windows 10 (1803) at the time.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.