Patch panels: Where should and shouldn't you put them?

Hi all,

I'm far from experienced in enterprise networking, however I'm 18 and currently going to school where we've started to move on to this topic. There's a question I'd like an answer to regarding patch panels and where it'd be wise to put them in a network of about 7 servers, a switch, about 30 outlets i.a.:

Would you recommend a patch panel between the server(s) and the switch, in addition to between the switch and the various outlets? (Servers > Patch Panel > Switch > Patch panel > TO). Or would to connect the servers directly to the switch? (Servers > Switch > Patch panel > TO). The switch would be placed in the same rack as the servers or in a rack just next to them, does this make the patch panel unnecessary due to the short length of the cable? What about a router/firewal, how would you patch this?

Thank you for your answers, and for understanding my level of knowledge and experience!