Yes, use the Local Only option. This keeps the router from doing any IPv6 with the ISP modem if your ISP and the ISP modems doesn't support it natively.
Further, if you want to disable IPv6 in Windows, you can un-check the IPv6 protocol under the network adapter settings as well.