Taking over

I thought the Buddleia was coming into bud early - but discovered it is in fact a bramble coming right up through both the Buddleia and the Lilac. There have been odd shoots coming through the fence every year since I've lived here but never as many or as vigorous as this year. The flowers are pretty, the fruit may be nice but the thorns are vicious. I don't know how to get rid of it without harming the flowering shrubs.