There's always gonna be some like that....especially if they see you catch a fish...

The lake is for public use IMO.....not just for some...

The only thing I may have done different is fish a spot within eyeshot of the "secret sweet spot" you were wantin to fish until the other guy had already passed....

Chances are that he didn't know about your sweet spot and would have passed right by it anyway....

Catchin a fish (or multiple fish) off a sweet spot while another fisherman is watchin just gives away the "secrecy" of the secret sweet spot...that's why I don't use marker buoys while other fisherman are around....