Just tell him his pile was so small that you put yours on top of it so it is now your pile, but that he can fish it when you're finished!