Not sure if this may improve your success, but I tie directly to the lure or hook without a swivel. This will usually give the lure/bait a more natural presentation without the added weight and reduced action from the swivel.

It has worked for quite a few of us from bream to crappie to various saltwater species.