Depends on how smooth the primer coat is, but most of the time I would lightly wet sand the primer with 800 grit or finer paper. The sanding will show up minor imperfections prior to finish painting. Wash off sanding residue, blow dry with air hose and then top coat.
Good luck..