The tower was built shortly after 1805 for the owner of Middleton Hall, Sir William Paxton. The architect was Samuel Pepys Cockerell (1753-1827), who had completed the Hall about ten years before. The tower commemorates Nelson's victories, notably Trafalgar, and provides the vantage point for an almost unsurpassed view of the Towy valley. An 1833 description refers to a sumptuous banqueting room, but hospitality may have been impractical as there does not appear to have been any kitchen.
There were inscriptions on the three external faces in Lord Nelson's honour in English, Latin and Welsh respectively, and the window facing Carmarthen is said to have been painted with a portrait of the Admiral, none of which survive.
Horatio Nelson (1758-1805) is regarded as the greatest figure in the history of the British Royal Navy. In his personal correspondence Nelson described himself as a defender of “our present colonial system”, which in the West Indies was based on slavery. In his early career Nelson fought against the Indian Kingdom of Mysore and the American colonies in the Revolutionary War. His reputation though is based on his role in the Napoleonic Wars, in which notable victories included the Battles of the Nile in 1798, Copenhagen in 1801 and Trafalgar in 1805, in which Nelson's fleet defeated the combined French and Spanish fleets. Nelson lost his life in the fighting at Trafalgar, but the victory ended the threat of an invasion and secured Britain's dominance at sea, as well as Nelson's status as a national hero.
Paxton was a friend of Nelson and had benefitted from the latter's decision to rebuild the British fleet at Milford Haven. The inscription read: "To the invincible Commander, Viscount Nelson, in commemoration of the deeds before the walls of Copenhagen, and on the shores of Spain; of the empire everywhere maintained by him over the Seas; and of the death which in the fulness of his own glory, though ultimately for his own country and for Europe, conquering, he died; this tower was erected by William Paxton."
It is also said that Sir William, displeased at having lost the County election of 1802 after an expenditure of some £15,000, built the tower with money he had offered his electors during the campaign for a free bridge over the Towy. It was therefore conspicuously sited as a reminder to the electorate of their folly, and not only for the sake of the view.
The tower is now in the care of the National Trust, who have restored the roof and the floor of the main room and have installed a steel staircase.