Depending on how hard the wind is blowing baits one side of the boat maybe moving slower or faster then the other, like making a turn when long lining. Could also be the wind is putting one side of the baits more under the boat , which may be spooking fish. Could try and go in the opposite direction see if the bite still stays on the same side of the boat.