The earliest known owner of Pitmuies was Patrick Guthrie of Guthrie to whom the property belonged in 1599. Pitmuies was given as dowry on his daughter's marriage to David Ogilvy, younger son of Lord Ogilvy. Their joint coat of arms, dated 1643, is carved on the north wall of the doocot. Records of 1730 refer to their successor, Mr Ogilvy, having 'built himself a fine new house at Pitmuies'. Following his death, his widow married Mr Carnegy of Lochlair in 1744 and her son later inherited Pitmules. He moved abroad and died bankrupt, which resulted in the sale of the estate in 1760 to Mr Robert Pierson. He lived for only two years at Pitmuies before his death in 1763; his initials R.P. are carved on the sundial in the garden.
James Mudie purchased the property in 1768 and he appears to have been responsible for the structure of the designed landscape which remains today as there is no indication of a designed landscape shown on General Roy's map of c.1750. Two single-storey bow wings were added to the west-facing house which dates from c.1730 and has the 17th century south-facing annexe behind it. The farm quadrangle, bridges and gate pillars were also built at this time. A court action was taken to prevent people using the drive to the west front of the house as a right of way and Mudie subsequently built a new road to the east of the house. The legal papers of the case are held by the present owners. He and the two successive generations of the family were also responsible for planting hardwood trees, some of which remain today.
In 1876 the last of the Mudie family left the estate to Sir Leonard Lyell, later 1st Baron. He remained at his home, Kinnordy, in Kirriemuir, and Pitmuies was let to a successive number of tenants until 1919 when it was sold to Major Crombie who was responsible for recreating the garden. The Delphiniums in the herbaceous borders were planted by him; a species he showed at national exhibitions and which won him awards. His son sold Pitmuies to Major & Mrs Douglas Ogilvie in 1945. They planted many trees and shrubs throughout the policies and their improvement work was continued by their late son, Mr Farquhar Ogilvie, who inherited Pitmuies in 1966. His wife and family remain at Pitmuies today. They have been particularly responsible for the restoration of the farm and other buildings on the estate. Mrs Ogilvie has a particular interest in the garden and some of the plants came from her family home, Castlewellan, County Down.
Pitmuies House, listed category B, is a three-storey mansion dating from 1730 which incorporates the earlier 17th century house behind. The wings and portico were added in the late 18th or early 19th century. An octagonal glasshouse on the south side has been added since 1966 to the design of Schomberg Scott. The Home Farm, listed category B, stands to the north of the house and also dates from c.1770. There are date stones of 1770, 1775 and 1820 as well as a weathervane inscribed 'I.M. 1773'. The Doocot, listed category B, incorporates in its north wall a stone with a carved coat of arms of the Ogilvy/Guthrie families and the date 1643. The Gothic features are thought to have been added in the early 19th century. The Wash-house, listed B, was built in the Gothic style in the early 19th century to the south of the walled garden and features on an estate map dated 1814. The walled garden, sundial, pillar, seat and gatepiers are listed category B and are thought to date from the early 19th century. The bridge carrying the road above the woodland garden was built by Mudie in 1819 and is listed category B. Further down this road, the bridge over the Vinny Water is listed category C(S). The west and east gates are both early 19th century and both listed category C(S). A stone cross slab bearing Pictish symbols stands in the wood to the north of the west drive; it is a Scheduled Monument and listed category B. A stone statue stands in the centre of the lily pool in the formal garden.
There are two areas of parkland at Pitmuies, one lying to the west of the house and the other to the south of the Vinny Water. Oak trees line the southern boundary of the latter park which is grazed. The larger park to the west of the house is farmed. Reference to the 1st edition OS map of 1868 indicates that the park once swept nearer to the house than it does at present. The ha-ha wall which now separates the park from the lawn in front of the house was built in recent years from stone acquired after the demolition of Guthrie Station. Comparison of the 1st and 2nd edition OS maps shows that historically, the park was never well stocked with trees. The oldest trees, including sweet chestnut and copper beeches, are around 200 years old or more and are now incorporated within the front lawn. Within the park, some elm trees remain which have been affected by Dutch Elm disease. There is also a hornbeam walk. There is a lochan to the north of the park, formed since General Roy's plan of 1750, and a nature walk is proposed around the water features and their associated wetland habitats.
The woodland garden lies to the north of the house, on either side of the drives, and to the south of the house, along the banks of the Vinny Water. The area to the north of the house is beautiful open woodland, with predominantly beech trees which date from the 19th century. Other ornamental trees, including conifers, have been established around the loch. The low lying area by the east drive is being developed by Mrs Ogilvie with a range of plants including Magnolias, Eucryphias, species Rhododendron and Primulas. To the south of the house, a gate leads from the walled garden into the old bleaching ground adjacent to the wash-house, where numerous snowdrops and crocuses have been established forming a thick, colourful carpet in spring. A walk leads along the southern boundary through a mature beech and lime canopy of trees planted in the 19th century. At the west end, specimen trees such as Liriodendron tulipifera, Ginkgo biloba and a particularly fine Acer griseum have been planted in more recent years amid Victorian conifers (including one of the tallest Monkey puzzles in Scotland) and Rhododendrons. The Liriodrendron is thought to be one of the largest and most northerly known in Scotland.
The formal garden is situated adjacent to the south front of the house. Its outline is shown on the map of 1783 held at Pitmuies but the original design is unknown. The 1st edition OS map only shows three parallel paths running in an approximate north/south direction dividing the garden into three areas. The garden is still divided into three areas but in a different way now. It is enclosed on three sides by walls and on the west side by a yew hedge. This hedge forms a backdrop to blue, yellow and grey-coloured double herbaceous borders which consist of a variety of species but include many Delphiniums established by Major Crombie between 1919-45. A stone seat is situated at the south end of the borders, looking back towards the house.
The Kitchen Garden lies to the east of the house and on the north side of the walled garden. It is well stocked with vegetables and fruit for the house and is also used for propagating material from the garden for replanting or sale. A new display greenhouse has been constructed and a new herb garden has just been planted out in a paved area next to the east wall.