Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[volox]

I'm driving myself up a wall trying to get this to work; so any help would be greatly appreciated.

I'm trying to get L2TP/IPSec to work between my Vista laptop and the DIR-330 router.  When I try to connect the laptop to the VPN it just bails out with:

Error 789: The L2TP connection attempt failed because the security layer encountered a processing error during initial negotiations with the remote computer.

Here is my setup:

I've got the DIR-330 router configured with a CA certificate from my internal certificate server (windows certificate server).  I generated a certificate for the router from the certificate server with the IPSec certificate setting and then used OpenSSL to convert it to a text based private key and certificate which I then uploaded to the router.  I also generated a certificate for my computer and installed it.  So at this point the certificates are deployed like this:

Router:
    CA public certificiate installed
    Router Cert installed as local identity with private and public
    My cert public installed as peer identity

Laptop:
    CA public certificate installed
    My cert public and private installed

All certificates are showing as valid in the router's config screen.

I've got an IPSec tunnel setup with Site to Site setup with 0.0.0.0/0 and the certificates selected as the authentication.  I've got Perfect Forward Secrecy turned off and I've tried it with NAT-T turned off and turned on.

I've got the L2TP over IPSec setup with MSCHAP v2 and using the certificates.

Doesn't seem to matter what I do with the Vista settings, I can't get it to connect successfully.  I've dug around all over the internet and haven't found a set of instructions that seem to explain how to get this working. 

Maybe the problem is with my router setup (since I haven't been able to find instructions on setting this up on DIR-330 and have just been figuring it out from instructions for other routers). 

Maybe the problem is with the way I've generated the certificates since D-Link doesn't seem to provide any information on what the certificate requirements are for the router.  In fact it took me a while just to figure out how to get from the private key export that windows provides to a key format that the router would take).

Maybe the problem is I'm not hitting the magical checkbox somewhere in Vista.

Whatever the issue is I'm about to go mad trying to get this to work and since VPN is one of the reasons that I bought this router, I'm going to be upset if I can't get it to work.

Thanks in advance for any help or leads.

[Fatman]

Lets take it down a notch for testing purposes.  Does your VPN work using PSK?

[volox]

Nope.  PSK does work either; but my understanding of the built-in Vista client is that it wouldn't support L2TP over IPSec with PSK that it would only support it with certificates; hence why I was trying to use certificates.

[Fatman]

No, the built in Vista client works real well with PSK.  You did manually select the tunnel type on the PC's VPN profile right?

[volox]

I'll review my settings in Vista (for those of you following along  )

General Tab
- Has the DNS address that translates to the public IP address on my router

Options
   X Display progresss...
   X Prompt for name and password...
   _  Include Windows...  (unchecked)
   
   Redail options (left defaults

   PPP Settings
       X Enable LCP extensions
       X Enable software compression
       _ Negotiate multi-link for single-link connections

Security
   X Typical
       Verify my identity:
           Require secured password
       _ Automatically use my Windows...
       X Require data encryption

Neworking
   Type of VPN
      L2TP IPSec VPN
   IPSec Settings
      X Use preshared key: (key value entered here to match key entered in router settings)
   Items selected for use (left defaults and deselected IPv6 since I don't use it)

Sharing
    (Left defaults - not shared - second and third boxes are grayed out)


So there is it, those are my Vista settings.  Let me know where I went wrong.

[volox]

Anyone...

[Fatman]

I got your PM, I am sorry if I have been an absentee in this discussion, but there isn't much discussion left

We have FAQs listed for making a L2TP tunnel on the DIR and for connecting from Vista.  Assuming that you have read the docs correctly, I know this process is real simple, and I have replicated it dozens of times.

The bottom line in these cases is usually (from my experience) the intermediary path between that host and the DIR.  Usually the router closest to the L2TP over IPsec client.

[volox]

Fatman,
I'm probably just not looking in the right place, but I wasn't able to find FAQ for an L2TP/IPSec VPN setup between a computer and the DIR-330.  (I looked in the support resources on D-Link website and all the L2TP is router to router)  Could you perhaps post a link to the FAQ you are referencing?

And if I do have everything setup properly and there is something between the endpoints causing the blockage, do you know how I might go about determining where the blockage is?  If it is either of my ISPs I may be able to get the block changed so that this will work, but I need to know which end / hop is the problem.

Thanks.

[Fatman]

You appear to be right, the docs I could have sworn were up are nowhere to be found.  I will look into that.

As for the point of contention, as I said previously in my experience it is usually the router the L2TP client is behind.

[volox]

Let me know when the docs get re-posted (or post them here please).  When I get my hands on them I'm going to step through my entire setup to make sure it matches the recommendations.