The church was built of brick faced with stone over the period 1824-33. It was designed by Sir Robert Smirke and commissioned by the 4th Duke of Newcastle as a tomb for the Duchess, who died in 1822 after giving birth to twins. Other members of the Newcastle family are buried here.
The building comprises a rectangular nave with a raised sanctuary area, a portico at the eastern end, a rotunda, north and south transepts and an octagonal lantern of two stages, with Greek Doric columns, containing two bells.
The nave of has a plain west entrance whereas the mausoleum has a four-column pedimented portico entrance to the east. The portico leads to a vestibule (with vestries on the north and south) and beyond is the rotunda with transepts on the north and south. The transepts were originally designed to be tomb chambers but the Duke changed his mind and decided they would instead house family monuments.
When the mausoleum opened it became the parish church of West Markham (renamed Markham Clinton) instead of medieval church of All Saints, which stands half a mile away, so the latter was neglected. The roles were reversed in 1949 when the parishioners agreed to use the old parish church so the mausoleum started to decay and was vandalised. A recent restoration has revived interest in this individual building after the Redundant Churches Fund (now the Churches Conservation Trust) took over in 1972.
There is a medieval-style memorial for the 4th Duke (d1851) and a plain mural tablet for Joseph Denman (d1863), but the main attraction is the marble monument by Sir Richard Westmacott for Georgina Elizabeth, Duchess of Newcastle and her twin babies.