I had this problem, and D-link had me RMA the router, so i sent it back, got the new one, upgraded to the newest firmware,. and blammo, exact same problem,
i turned off wirless, and its been running fine for hours, after rebooting every 5-10 minutes.
Unfortunately this is not really a solution, as i bought this unit specifically for the wireless N.
From reading on broadbandreports forums, this is happening to many people, so its obviously buggy firmware.