This magnificent C12th castle was repeatedly improved and remodelled over the generations – including by Paxton and Pugin. There’s a C17th walled garden and many other delights.