I've always "heard" that Crappie tend to school by year class, and (+/-) size. This hasn't always played out, when fishing for a considerable amount of time in one place ... but, the original school of fish, that I was catching, may have been replaced (or joined) by a school of different sized fish.
It's quite possible that the school of smaller fish DID have some larger ones in it .... but, the smaller ones were more aggressive.
I've caught Crappie out of a spot ... and they looked like they were all from the same mold :D (almost identical size/weight). Other times, I've caught dinks & slabs from the same general spot (though, usually there are 10:1 dinks to slabs :p )
Personally, I think I would have moved ... more probably to a deeper location ... after about 30mins of the littlen's. But, don't take my word for it ... I've only Crappie fished the Ky River, a few times & in a couple of places ... and that was years ago. I've been in the Highbridge area, once, and caught some nice fish on the outside branches of some flooded, live, shoreline trees. Most of my river fishing has been at Clays Ferry ... catching Crappie around timber washed against the bridge pilons.
I Crappie fish, more for the fun of it than for the meat ... I love to trick them into biting my artificials But, I do (eventually) run out of filets, and have to "restock" ... so, I do make the occasional trip for that purpose.

I'm not sure that using different bait would have made any difference. Sounds like you got into a school of hungry young'uns, and the thrill of the action kept you occupied (if not satisfied). You had fun, and that counts for something.
Maybe next time you will find a school of hungry bigguns ...... luck2ya ... cp