The estate of Dun was bought by Robert Erskine of Erskine in Renfrewshire in 1375. The remains of the original tower house can be traced in the large walled garden to the west of the present house. A large stone gateway marks the former entrance to the court. The castle having fallen into disrepair, a new house was commissioned from William Adam in 1727 by David, the 12th Laird, a judge in Edinburgh who took the title Lord Dun. He also commissioned designs from Alexander McGill, and designs for the garden were drawn up by the Earl of Mar (SRO.GD. 123/120). The Earl of Mar's proposals refer to the old garden by the castle being a bowling green and cherry garden, and suggest putting in a bridge across the Den. Two beech-lined drives were to sweep gracefully from the Brechin Road from the east and the west to the house. Contemporary accounts refer to existing old trees of chestnut, oak and beech in the park.
Further major improvements to the house were carried out in the time of John Kennedy-Erskine from the 1820s onwards. His mother, Margaret Erskine, the 17th Laird, succeeded her sister Alice. Margaret married Lord Kennedy of Culzean and her second son John inherited Dun. He married Lady Augusta Fitzclarence, natural daughter of King William IV, (who gave her for one of her wedding presents, a yellow coach). John died in Pisa four years later and Lady Augusta returned to London where her child was born; she later remarried and did not move back to Dun until c.1840.
In Alice Erskine's time the walled garden to the east of the house had been used as an exercising ground for her horses and it had since been turned into a drying green. Lady Augusta had it laid out with ribbon borders surrounding a lawn, with rose bowers in the centre. A sunk fence was put in across the south park to take in a greater area of garden, which was then laid out in the terraces which remain today. Stone steps and pillars led up to the south windows and yew hedges were planted along the terraces. She also put in the walk along the narrow gorge of the Den of Dun up to the walled garden and to the burial ground where her first husband was buried. She created a large rockery along the walk on the east side of the burn, driving around the countryside and coast in her yellow coach in search of plants and white quartz for her rockery. Popular stories of the time refer to her very fat coachman having to climb down from the coach to collect huge rocks from the shore.
Lady Augusta's granddaughter, Violet Erskine, was born in 1863. She is better known as the writer Violet Jacob, and she was brought up at the House of Dun which formed the background for many of her works including 'The Lairds of Dun' which recounted the history of the Erskine family. No further changes to the designed landscape were made in the period up to World War II, when the house was occupied by the Army. Since the war it has been let as an hotel up until 1985. In 1980, Mrs A.A. Lovett of the Erskine family bequeathed the estate to the National Trust for Scotland; the house and the 45 acres of policies are inalienable.