The Carnegie family have lived at Kinnaird since 1401, when Duthac de Carnegie by purchase and marriage acquired the estate and built the original castle in 1402. Duthac de Carnegie died in 1411 and his son Walter was involved in the Battle of Brechin after which the Earl of Crawford burnt Kinnaird Castle in 1452 in revenge for Walter having sided with the Earl of Huntly. In 1567 extensive alterations to the Castle were made by the 5th laird Walter's greatgreat-grandson Robert, who was Ambassador to France; this castle entertained James VI, Charles I and II and the Chevalier. In 1616 David, the grandson of Sir Robert, was created Lord Carnegie of Kinnaird and in 1633 he was raised to the Earl of Southesk. By 1685 Robert, the 3rd Earl, was reputed to have a great house, with gardens and parks with fallow deer. The 4th Earl, Charles, in the 1690s commissioned plans for the improvement of the house and grounds from William Bruce and Alexander Edward and these plans are preserved today at Kinnaird Castle, although it is doubtful if much, if any, of the designs were carried out as the 4th Earl died in 1698 when his son James was still in his minority. There are references though to account receipts from Alexander Edward for the importation of plants which would suggest that some work was carried out, perhaps the avenue plantings which appear on General Roy's map of 1750.
After the 1715 rising the 5th Earl was exiled and his estates and titles attainted, and the estate was purchased by the York Building Company. By 1718 the house was reputedly in a 'decaying condition' and although it was tenanted from then until 1763 little was done to keep it up. On the insolvency of the York Building Company, a large part of the estate was repurchased by the last Earl's third cousin, Sir James Carnegie of Pittarrow, Bt. He died before l the completion and it was left to his successor, Sir David Carnegie, to carry out repairs. In 1790 he made extensive alterations, commissioning Playfair to rebuild the house and Thomas White Snr to lay out the grounds. Sir James, the 6th Baronet and the present Earl's grandfather, succeeded in 1849. In 1855 the Earldom was restored with original precedence and in 1869 he was also created Baron Balinhard of Farnell. He commissioned David Bryce to carry out further alterations to the house between 1854 - 1862, and the policies were extended to the north and west at this time, large woods and a 20 acre loch being added to the grounds. He also planted a great many clumps of hardwood trees and roundels in the Park about 1860.
Bryce added the front porch, stairs, balconies and terraces to the house. The terraced gardens along the south-west and south-east fronts of the house were constructed involving considerable earthmoving to create a flat area where previously there had existed a steep slope. By 1885 the estate contained 22,525 acres and resembled 'an ancient French chateau with many lofty steep-roofed towers and turrets, long stone balconies and balustraded terrace walls. The park, three-fourths of which are occupied by the deer park, with four hundred fallow deer, comprises between 1300 and 1400 acres... most of the trees were planted towards the close of the last century, but there are several 170-400 years old'. (OS Gazetteer)
In 1921 a fire gutted all the south-east end and the west and central south-facing part of the Castle. The 10th Earl rebuilt the west and central parts but decided to just put a roof over the east end.