Have you tried swapping the port the 360 is connected to on the 5 port switch? Seems like it has something to do with the Ethernet since it will work connected directly.
Maybe test it direct to a unit one more time without changing anything else.

Sounds like you have a completely new setup....is the 5 port switch new also?