Barnstaple . . .

 . . .  used to have a castle; it fell into disrepair and was demolished.  However, at the entrance to Castle Green, the area where the castle stood, this small turreted house proudly sits. I assume it was a sort of gate house originally.