Wednesday, September 9, 2020

Cheap switch for 15GB/s NAS going to 33 Computers

In my office I have 33 rented renderfarm servers, with 23 servers in room A and 10 severs in room B, each running windows on an SSD and 1GbE networking. Every single computers pulls data and pushes lots of data into one single NAS drive, often at the same time, for many hours a day.

The NAS is a 15GB/s NVME raid array running windows sever with dual 10GbE (Mellanox CX-3), but is currently connected with only one single SFP+ link into a Mikortik CRS328-24P-4S+RM(24x GBe + 4x SFP+). This mikrotik is then used to link into the 23 servers in Room A, with the last GbE port going into a unmanaged netgear 24-port GbE switch in ROOM B.

I'm having performance issues because im using the single gigabit link between Room A and Room B, from the managed mikrotik into the unmanaged netgear switch. I am just using windows file sharing(should be on SMB3.0, but I have not checked.)

Can I just get a second switch that has with 10GbE, and connect that into the mikrotik? Is there any additional setup steps that I would have to take? What if I wanted to connect two of the 10GbE ports from switch to switch? I'm trying to avoid setting up LACP because I have read that it does not actually aggregate the throughput of the network, instead SMB3.0 supposedly supports multichannel mode naively, and it seems to work if I plug in both connections from the 10GbE nic into the switch, but I don't know how it works from switch to switch. I don't have a very good grasp on this.

1) Does anyone have any quiet 48 port w/ SFP+ switch recommendations for under $1000? I am looking at netgear or ubiquiti. The nice mikrotik 48 port switch seems to run RouterOS only, which is probably a pain to setup. I am just looking for a simple layer 2 switch I think. I guess I could also get two 24 port switches, but the 48 port would be more flexible.

2) At some point, 33-48+ computers all pulling/pushing data from that single 10GbE connection to the NAS server is going to be the bottleneck, and SMB3.0 seems to just aggregate the amount of bandwidth i have. So, I can just use two SFP+ 10GbE links from the NAS into the Mikrotik. Is this wrong, and do I actually need to setup LACP or something else?

3) Should I look into getting a better router to route this many computers or am I okay? Internet speed is not something I am worried about for this setup. I have a D-Link DIR-882 running DD-WRT and it seems to be doing fine now. Is there any point getting a professional grade router at this point if I am on a budget? I won't have the servers forever, so I don't think I don't want to spend too much money on something that wont be useful in a month or two, but it would be great to be rid of these current bottlenecks.



No comments:

Post a Comment