This house is said to have been built by Thomas Gunter, a leading local Roman Catholic, supposedly in about 1600. but inspection shows that it is evidently of two dates and it also seems plain that it never included No. 40 (qv and see Interior) although this has often been associated with it. It is unlikely that this building was a hall-and-cross-wings house, dating to the early C17, as the Gunter Mansion consists of the hall range and right-hand projecting wing only, what is now Nos. 37-39 (consec) Cross Street. In fact Nos. 37 and 38 are only the ground floor shops, the whole of the upper floor is a part of No. 39 which contains the main staircase.
The original house is the long two storey four room range parallel with the street which is probably later C16, the oldest visible features being the Tudor door frames and the moulded timber ceilings (see Interior|). This house was then given a major improvement in the mid C17 when all three projecting gables, one at the front and two at the rear, one of these containing the main stair), were added, and the decorative scheme was enriched by the very fine plaster ceiling on the first floor. It is this work that was probably done by Thomas Gunter, a determined recusant who was known for his support for the Catholic faith during the years following 1649. This period ended with the capture and execution of two priests after the so-called Popish Plot invented by Titus Oates and Dr. Tonge, and they are commemorated in the Church of Our Lady and St. Michael, Pen-y-pound Road (qv). They were Father David Lewis and Father Philip Evans, who were both executed at Usk prison in 1679. The C18 and early C19 saw minor Georgianisation of the house with surviving joinery details like the cornice along the street front (though this could be Edwardian) and possibly the cross-framed windows at the rear. A late C19 photograph of the street front shows the property divided into three tenements and with Georgian features. None of these now survive, but many of the openings are the same. The centre part became the Parrot Inn in 1864 and was then the Cardiff Arms in 1898-1907 when it was sold, but this is not evident in this photograph. Another photograph, which is dated to 1913, shows the shopfronts (or pub front), the pedimented door and the upper floor sashes as today, but the other windows have been changed since then and the roofline and chimneys altered, including one of the main ones just to the left of the gable being completely removed. The next major repair followed the purchase of the property in 1907 by the local builder and developer Thomas Foster who, together with his architect brother Edwin, repaired and improved the property. The principal visible feature datable to them is the range of fancy sash windows on the first floor front while the pedimented main doorcase and the public house frontages may also be theirs but could be late C19. The Fosters discovered the evidence for Gunter's secret Catholic chapel in the attic, the Adoration mural from which is now shown in Abergavenny Museum, (qv). Since the Fosters' repairs in 1907-8 the building has been little altered. The only major C20 alterations to the building have been the removal of one of the main chimneys (present in 1913) and the single storey extension along the whole of the rear wall. The original main door of the Gunter Mansion is displayed in Abergavenny Museum.