Nicolas,
You should still be able to access the routers config page without having internet connection. As long as your PC is connected to a LAN port of the router you can access the config page from a web browser by typing 192.168.0.1 in the address field.