In about 1300 Sir John de Broughton built his manor house in a sheltered site at the junction of three streams and surrounded it with a substantial moat. In 1406 it was "crenellated and embattled’, allowing the house to be called a castle. Over the years the castle was taken by the Royalists during the Civil War after a brief seige, and went through a series of ups and downs due to the squandering of family fortunes.