With my D-Link DIR-655 (hardware version A3, firmware version 1.31EU) I could turn on/off a setting called 'Advanced DNS Service'. I read about this in another forum post, but could not find the feature in the Advanced or Tools sections.
With my version of firmware I go to http://192.168.0.1/Basic/WAN.shtml (you might need to change the IP-address to your routers IP- address), or "Setup > Manual Internet Connection Setup > Advanced DNS Service".
This option fills in the fields for Primary DNS Server and Secondary DNS Server, disabling Advanced DNS Service fills in a default 0.0.0.0 IP address which results in using you internet service providers DNS's.
One can also use a custom Public DNS like Verizon's 4.2.2.2/4.2.2.6, fill IP-addresses like 4.2.2.2 and 4.2.2.3 in on your computer's IP settings page as your Primary and Secondary DNS's. Since these are used by many people, setting 4.2.2.5 and 4.2.2.6 might realize faster lookups.
I have not tried using a Public DNS as the routers DNS's, but think this might work as well (in case you want to override you ISP's error handling or so).
Cheers, Laurens