Ping router with dual internet gateway, reply from other gateway??

I built a 4 NIC Fedora 33 box which is using iproute2 and nftables. I have 2 LAN gateways, a cable modem gateway and a DSL gateway. When I ping the DSL gateway from the internet, the reply comes out the cable gateway (which is set by the kernel as default). Pinging the cable gateway works as expected. Switching the default to the DSL gateway will enable pinging it and disable pinging the cable gateway. I need both gateways to work as expected, at the same time.

I have tried all the recommended multi gateway routing tricks I coud find. Most of them describe creating custom routing tables, but none of them work. The documentation for doing what I want to do is really incomplete, even from the official sources. I suspect the issue is with needing to mark the packets using an nftables rule, but I haven't found any examples using nftables+iproute2. Does anyone know of a decent book or web site which can give me what I need to get this relatively simple configuration to work? Even if it starts off as generalized advanced routing in iproute2+nftables it would be a step in the right direction...