To find what you had missed, show your settings as screenshots, all what you changed, including Status > Routing.
1) Interfaces > wan1
Check "enable DHCP client"
wan1 - wan1_ip - wan1net - wan1_gw
Advance Tab:
Check On - Automatically add a route for this interface using the given network.
Check Off - Automatically add a default route for this interface using the given default gateway.
Route metric = 100
Do the same for wan2
wan2 - wan2_ip - wan2net - wan2_gw
except for Route metric = 101
2) Intrefaces > Interface groups
Make group "wans" with wan1 and wan2
3) Rules > IP rules > lan_to_wan1
Change in all rules destination interface from wan1 to wans
4) Routing > Routing tables > main
Add routes (keep "local IP address" blank and enable host monitoring with DNS servers or some external stable host)
Monitor for Route Failover Tab:
Check On - Monitor
Host Monitor
Check On - Enable Host Monitoring.
wan1 all-nets wan1_gw none 100
wan2 all-nets wan2_gw none 101
Host Monitor Tab: no configuration
5) Routing > Route load balancind > Instances
Add new instance on table "main" with algorithm "destination"
Network:
isp1: modem - dsa-3100 private lan - wan1
isp2: modem - wan2
Routing table contents (max 100 entries)
Flags Network Interface Gateway Local IP Metric
192.168.2.0/24 wan1 100
172.17.100.0/24 dmz 100
192.168.0.0/24 lan 100
112.202.0.0/17 wan2 101
M 0.0.0.0/0 wan1 192.168.2.1 100
M 0.0.0.0/0 wan2 112.202.0.1 101
Routing table:
main
# Type Interface Network Gateway Local IP address Metric Monitor this route Comments
1 Route wan1 all-nets wan1_gw 100 Yes
2 Route wan2 all-nets wan2_gw 101 Yes
3 Route wan1 wan1net 100 No
4 Route wan2 wan2net 101 No
5 Route dmz dmznet 100 No
6 Route lan lannet 100 No