Advice please

When we arrived home it appears my three birdhouses from dad, that are mounted on the courtyard fence have new residents.  I think they are wasps not honeybees.  Husband and I were thinking that we should buy wasp spray and spray it in late in the evening from a distance.  Any advice is welcome, I am afraid if we do this no birds will want to go in the houses in the future.  I think there is probably not much choice but any suggestions are appreciated.