Ok I have developed an interest in this topic. It seems that we take vision for granted to such an extent that the basic processes are not at first well understood. I have been doing a bit of reading :-) Although the following relates to motion not color I feel it is somehow a good starting point to explore fish vision.

In addition, medakas proved to be able to discriminate different kinds of biological motion, preferring the motion pattern of conspecifics to human motion and being particularly sensitive to the smoothness and the speed of the movement. This is particularly relevant since, also in our species, the speed of movement can drastically alter the perception of biological motion, with abnormal speeds giving the impression of unnatural. Moreover, both humans and the fish tested by Nakayasu and Watanabe (2014) seem to be more affected if the movement sequences were slowed down than if velocity was increased.

In this study zebrafish spontaneously chose to associate with a “natural” movie of swimming conspecifics rather than with a backward version of the movie, while they did not react to another violation that also created an unfamiliar visual scene (movie presented upside down). In the backward movie, movement and shape information were both still present and virtually unaltered, but were inconsistent with each other. To recognize the original movie from the backward one fish needed to integrate form and motion

This result was then replicated in the same study (Neri, 2012) with computer generated stimuli that were more controlled, even though less natural: an image representing a side view of a zebrafish was moved along a linear trajectory, which could be either consistent or inconsistent with the orientation of the image of the zebrafish (the direction toward which it was facing). As long as a sufficient number of individuals was depicted in this artificial animation, zebrafish were able to direct their response on the basis of the conjunction of motion direction and shape orientation, even when stimuli were constructed using images of another species (needlefish, Xenentodon) or when only the frontal part of a zebrafish image was visible.
Frontiers | What can fish brains tell us about visual perception? | Frontiers in Neural Circuits

I see two possible ramifications of this research.

First that a naturally moving lure should be more attractive in most cases. By natural we apparently mean smooth and slow. Experience would however tell us that fish will also attack objects in what has become known as triggered strikes. The later feeding pattern would be more typical of aggressive fish such as bass that tend to be less discriminating.

The second take away would be that biological motion may be discriminated by fish more readily by multiple lures "swimming" in unison.

A third interesting observation from the few minutes of reading that I did is that motion can be ambiguous.

The motion direction of a contour is ambiguous, because the motion component parallel to the line cannot be inferred based on the visual input. This means that a variety of contours of different orientations moving at different speeds can cause identical responses in a motion sensitive neuron in the visual system.



The aperture problem. The grating appears to be moving down and to the right, perpendicular to the orientation of the bars. But it could be moving in many other directions, such as only down, or only to the right. It is impossible to determine unless the ends of the bars become visible in the aperture.
https://en.wikipedia.org/wiki/Motion...ion_perception

The above would suggest that a lure should have a marked beginning and end. This idea is supported by the fact that only humans have been shown to have what is described as "global" vision. Fish likely see trees not forests. They will only see the trees however if the end points of the "trees" are clearly indicated because of non global neural processing and the aperture effect of objects moving in water. This is of course my own interpretation and would require experimentation to confirm.