Have you tried counting down after the jig hits the water to control depth? For instance. If fish are 10-15 feed deep, start at a 5 count , then add a couple counts until you find the number that matches the depth the fish prefer. Then your retrieve is repeatable at the depth the fish prefer. Like db4d pointed out keeping your rod tip low in windy conditions can help. Keep slack out the best you can and retrieve slowly. Good luck.