I was thinking about the problem and had an idea. I then read
http://forums.dlink.com/index.php?topic=46615.0 which reinforced what I was thinking, namely:
Perhaps the DNS IPs are inserted into resolv.conf after they are obtained via L2TP after all, but when the kernel tries to use these to resolve names for requests issued by the router itself, the connection to the DNS server is routed directly via the WAN interface rather than the LAN interface (192.168.0.1). As a result the DNS packet bypasses the L2TP stack and is therefore not routed to the ISP! The only way the DNS server can be reached is via L2TP as it is outside of the cable company's network.
What I will try is to
enable DNS relay, then set the
secondary DNS to 192.168.0.1 (overriding the value obtained via L2TP). Client DNS requests will be forwarded to the primary DNS as before. The router's own request (via the kernel) to the primary DNS will fail, so the kernel will try the secondary DNS of 192.168.0.1 and connect to the router's own DNS daemon (relay), which in turn will resolve via the primary DNS which should succeed.
Will try it this weekend. I'll also attach a sniffer to the WAN port to try confirm if this is what is happening.
The only thing bugging me about this theory is that if the router tries to connect to various external IPs (such as to ping an IP, update via NTP, or email via SMTP), that works just fine, so connections to these IPs are definitely going via the L2TP encapsulation. In which case why not the same for DNS? Either a route is being added just for DNS (as suggested by Gamification in his post), bypassing L2TP, or these other programs all force the use of the LAN interface guaranteeing connection via L2TP (something the kernel can't do for name resolution - that requires adding a route definition).