10G SFP+ LACP between compute node and 2 shared storage nodes *help*

Hey everyone,

Here is would I would like to achieve:

I have a Proxmox node with a 2-port SFP+ PCI card and 2 OmniOS CE boxes with 2-port SFP+ PCI cards attached. My goal is to connect the Proxmox node to each storage node, and for the 2 SFP+ ports on each node to use one logical interface. I want to avoid having 2 separate 10G IPs on each node, and I also want to avoid buying a 10G switch. Eventually, I will have another Proxmox node with 4 10G SFP+ ports, and I will add another 2 to the original Proxmox host in order to create a Proxmox cluster. At that time, I can add 2 links to the aggregation group in my original Proxmox node and still have the same logical interface with one IP.

​

Here is what I have tried:

I configured the Proxmox node for LACP L3+L4 with both 10G SFP+ nics as the slaves, and I did the same for my 2 storage nodes. The results were strange as Proxmox could see 1 or the other but not both. I have just 2 cables. 1 goes from Proxmox SFP+ nic1 to storage node 1's nic1, and the other goes from Proxmox SFP+ nic1 to storage node 2's nic1. Yes, I have configured LACP on the 2 storage node even though I am only using one physical link at the moment. This should not matter, though right?

​

What happens in this configuration is that Proxmox can see only 1 at a time. I can turn off 1 storage node and then reboot the other, and the other is now communicating with the Proxmox node.

​

Am I missing something? I have played with Active/Passive stuff, but I have not gotten better results. Can anyone make sense of this enough to assist me? I am very new to LACP stuff.