I've gotten some Termite spray insecticide that is also for carpenter bees. Spectracide is the brand and get at WM, Lowes, HD, etc. Comes with a small spray tube and you insert the tube into the bored holes and spray. have also heard that the bees will eat chewing gum put into the holes and it will kill them somehow, but IDK about that. The spray works, but you have to spray the insecticide fairly deep into the bored holes as I've read they bore deep and have several routes or tunnels in the wood.
GO BIG ORANGE !
I meant to behave, but there were just way too many other options available at the time.