To find what you had missed, show your settings as screenshots, all what you changed, including Status > Routing.
1) Interfaces > wan1
Check "enable DHCP client"
wan1 - wan1_ip - wan1net - wan1_gw
Advance Tab:
Check On - Automatically add a route for this interface using the given network.
Check Off - Automatically add a default route for this interface using the given default gateway.
Route metric = 100
Do the same for wan2
wan2 - wan2_ip - wan2net - wan2_gw
except for Route metric = 101
2) Intrefaces > Interface groups
Make group "wans" with wan1 and wan2
3) Rules > IP rules > lan_to_wan1
Change in all rules destination interface from wan1 to wans
4) Routing > Routing tables > main
Add routes (keep "local IP address" blank and enable host monitoring with DNS servers or some external stable host)
Monitor for Route Failover Tab:
Check On - Monitor
Host Monitor
Check On - Enable Host Monitoring.
wan1 all-nets wan1_gw none 100
wan2 all-nets wan2_gw none 101
Host Monitor Tab: no configuration
5) Routing > Route load balancind > Instances
Add new instance on table "main" with algorithm "destination"
Network:
isp1: modem - dsa-3100 private lan - wan1
isp2: modem - wan2
Routing table contents (max 100 entries)
     Flags             Network             Interface     Gateway     Local IP             Metric    
           192.168.2.0/24       wan1                                      100
           172.17.100.0/24       dmz                                      100
           192.168.0.0/24       lan                                      100
           112.202.0.0/17       wan2                                      101
    M        0.0.0.0/0               wan1        192.168.2.1               100
    M        0.0.0.0/0               wan2        112.202.0.1               101
Routing table:
main  
#  Type  Interface     Network     Gateway     Local IP address     Metric Monitor this route  Comments 
1     Route     wan1     all-nets     wan1_gw         100    Yes     
2     Route     wan2     all-nets     wan2_gw         101    Yes     
3     Route     wan1     wan1net                      100    No    
4     Route     wan2     wan2net                      101    No    
5     Route     dmz             dmznet                      100    No         
6       Route     lan             lannet                      100    No