Early Years
Lee was born at Stratford Hall on May 28, 1787, the third child of Henry Lee III and Matilda Lee. His mother died on August 16, 1790, but not before putting Stratford Hall in trust to the younger Henry. The following year his father took office as the governor of Virginia and the family moved to Richmond. In 1793, Henry Lee III married Anne Hill Carter. Over the next two decades Lee IV would gain six half-siblings, including the future Confederate commander Robert E. Lee.
A gifted student, Henry Lee IV attended Washington Academy (later Washington and Lee University) and, from 1807 to 1808, the College of William and Mary. In 1808 he became the master of Stratford Hall after his father, having made some bad investments, fled the dilapidated estate and his creditors. (The former governor landed in debtor’s prison the following year.)
Like his father, Lee had political and military ambitions. He represented Westmoreland County in the House of Delegates for three terms (1810–1813) before entering military service against Great Britain during the War of 1812. He was appointed a major in the 36th Regiment of the U.S. Infantry and saw action along the Canadian border, serving as an aide-de-camp to General James Wilkinson and later to General George Izard of South Carolina.
After the war, Lee returned to Stratford. With the help of his uncle Richard Bland Lee, he sought a military appointment. President James Madison eventually offered him the position of assistant inspector general for the southern division, which Lee declined. In 1816 he ran as a Federalist to represent Virginia’s Ninth Congressional District—the same seat his father had held from 1799 to 1801—but was defeated by William Lee Ball. One of three candidates, Lee managed to get only 14.5 percent of the vote.
Marriage and Scandal
After his congressional defeat, Lee remained active in social circles in Washington, D.C., and New York City. He was known more for his intellect than his looks, and for his tendency toward sarcasm and invective. No known image of Lee exists, but Governor Henry A. Wise once wrote of him, “He was not handsome as his half brother, General Robert E. Lee; but rather ugly in face, a mouth without a line of the bow of Diana about it, and nose not cut clean and classic, but rather meaty.” Yet, he added, Lee was “one of the most attractive men in conversation we ever listened to.” Indeed, he courted a number of women—notably Serena Livingston, one of the wealthiest young women in New York.
On March 29, 1817, Lee married Anne Robinson McCarty, his distant cousin and heiress with her younger sister to Pope’s Creek, a plantation adjoining Stratford. McCarty’s land holdings, combined with the 3,000-acre Stratford property, amounted to roughly 5,000 acres—a considerable bit of property. The marriage brought Lee not only land, but additional slaves and cash. He also became the guardian of his wife’s younger sister, Elizabeth “Betsy” McCarty, and the manager of her estate.
In 1818, Anne McCarty Lee gave birth to a daughter, Margaret. But two years later, the child fell down a stairway at Stratford Hall and died. The Lees never had another child, and a distraught Anne Lee turned to opium to alleviate her grief.
With the house in mourning, Henry Lee and Betsy McCarty had an affair. When rumor spread of an illicit pregnancy, the ensuing scandal rocked the Lee family. (The pregnancy rumor remains unconfirmed.) According to Virginia law, sex between siblings-in-law was considered incest; furthermore, Lee had embezzled money from his young ward’s estate. A contrite McCarty left Stratford, cut short her red hair, and began wearing a veil in public. She reinstated as her guardian her stepfather, Richard Henry Stuart, who then sued Lee for mismanaging McCarty’s estate. In 1822, a chancery court ruled that Lee owed McCarty one-half of the earnings from her Pope’s Creek estate from November 1817 to May 1821—a total of $11,568.97.For his part, Lee was remorseless about the affair. He later wrote in a letter that a man who “associates with the sister of his wife, with all the unguarded intimacy which he would observe toward his own sister and there being no blood connection, no barrier of instinct between them, is liable to be surprised into adultery.” He did not face criminal charges of incest, but he was forced to sell Stratford Hall to cover his debt to McCarty. Lee’s friend William C. Somerville bought the house in June 1822 for $25,000. After Somerville’s unexpected death in 1826, the Westmoreland County Court seized the property and auctioned it to pay Somerville’s debts. The buyer was Henry Storke—who in 1826 had married Betsy McCarty. Elizabeth McCarty Storke was the mistress of Stratford until her death in 1879.
Later Years
Lee’s marriage survived the scandal, but his political career did not. Once considered one of the most promising men in the country, Lee could not escape his notoriety. Lee’s political enemies began to refer to him as “Black Harry” or “Black Horse Harry” —the latter a play on his father’s heroic Revolutionary War moniker, “Light-Horse Harry.” In 1825, with the help of John C. Calhoun and John Marshall, Lee secured a position in the post office—though it was at a salary, he is reported to have said, that would “hardly bribe a slave.” He hired out his family’s slaves to supplement his income and turned his attention to writing. He authored a history of his father’s military exploits, The Campaign of 1781 in the Carolinas (1824), and reissued his father’s memoirs. Lee then began writing a history of the War of 1812 and traveled to New York and Canada to conduct research. In 1826 he visited Monticello to speak with Thomas Jefferson, but Jefferson died before Lee could meet him.
In 1826, Lee began corresponding with Andrew Jackson to get further information for his book. Henry and Anne Lee developed close ties with Jackson and his wife, Rachel, when Anne received treatment for her opium addiction at a health resort near the Jacksons’ Tennessee residence. Henry Lee wrote articles supporting Jackson and his 1828 presidential campaign and, at the request of Duff Green, an influential Democrat, began writing a campaign biography of the general. Although Lee never finished the work, his manuscript was used in later sources. Lee became Jackson’s speechwriter and publicist, drafting Jackson’s inaugural address and attending the new president’s swearing-in ceremony in March 1829.
Lee hoped Jackson would appoint him to a lucrative public office; in April 1829, he learned the president had appointed him consul to Algiers. Algiers was considered very dangerous, but Lee seemed to relish it. Upon moving there with his wife later that year, he claimed to keep six pistols, six muskets, and a Tennessee rifle in his house, and walked the streets armed. But his stay in Algiers was short-lived. When senators discovered the appointment in March 1830, they voted unanimously to have Lee recalled.
Henry and Anne Lee never returned to the United States. After leaving Algiers, they toured Italy and then settled in Paris, France. Still short of money, Lee begged his half-brother Charles Carter Lee for funds and threw himself into his writing. In 1832, he published Observations on the Writings of Thomas Jefferson, a response to Thomas Jefferson Randolph‘s four-volume book on the man. Randolph’s book had included sharp criticisms of Light-Horse Harry Lee; in his Observations, Lee unleashed a barrage of invective against Jefferson. In 1834, Lee published Life of the Emperor Napoleon, the first of two planned volumes on the French military leader. The book was reviewed positively, but sold poorly. Lee never overcame his financial problems. In 1835, he sold his wife’s slaves for $9,000, but it was not enough to pay his debts.
On January 30, 1837, Lee died of the flu. He is buried in an unmarked grave in Montmartre. His death left Anne Lee alone in Paris, where she remained until her death in 1840. In a letter to a friend she wrote, “All my habitudes are of this country I have been here so long, and the idea of returning there [America] without him is horrible to me. I can have no satisfaction in association with those who disliked and persecuted by dear husband while living and who may probably rejoice that he is no more.”
Major Works
- The Campaign of 1781 in the Carolinas (1824)
- Observations on the Life of Thomas Jefferson (1832)
- Life of the Emperor Napoleon (1834)
