ce L3 switch design question

I have a design question I need some guidance on.

​

Here's basically a quick diagram of what I want to achieve: https://imgur.com/a/89V47Cl

​

We provide internet service to a specialized market sector and I am in the process of adding a second customer edge switch. I want it to uplink to both of our cores, which uplink to a redundant chasis pe router.

​

I had designed it so that each ce would connect to each core over a 20gig LAG and then I connected the ce's together over a 40gig QSFP link. They are all on one publicy address /29 on a shared VLAN. When I added the second ce switch and tagged the communication VLAN between core-1 and ce switches, I believe it created a loop because I was seeing a ton of packet loss. When I disabled the QSFP link between the two CE's, the packet loss stopped. I didn't enable the link between the ce's and core-2 because I was afraid the same issue would happen.

​

I was thinking that the best way to achieve this redundancy and not have any loops is to enable spanning tree. it's currently not enabled (anywhere in the network... I inherited this).

​

Is enabling spanning tree the best way to solve this? Is there a better way? Maybe I need them to be on different VLANs, not all on a shared one? Maybe I only need ce-1 to connect to core-1 and ce-2 to connect to core-2?

​

How would you design this?

​

Thanks!