Henry Lee (1787–1837)


Henry Lee, also known as Henry Lee IV, was a writer, politician, diplomat, army officer, and the last member of the Lee family to own Stratford Hall. The son of Henry “Light-Horse Harry” Lee, a governor and hero of the American Revolution (1775–1783), Lee was educated at Washington Academy in Lexington and then the College of William and Mary. He represented Westmoreland County in the House of Delegates from 1810 to 1813 and served as a major in the U.S. Army during the War of 1812. In 1817 he married Anne McCarty, a distant cousin, and became ward of her younger sister, Elizabeth “Betsy” McCarty. The Lees had a daughter who died in a fall, and in the wake of their mourning, Henry Lee began an adulterous and legally incestuous affair with Betsy McCarty, embezzling profits from her estate, which was under his management. The resulting scandal earned him the nicknames “Black Harry” and “Black-Horse Harry” and thwarted his political career. He was forced to sell Stratford Hall. Despite his notoriety, Lee wrote campaign materials for Andrew Jackson and briefly served as consul to Algiers. In 1831 he moved with his wife to Paris, France, where he later died of influenza.

Early Years

Lee was born at Stratford Hall on May 28, 1787, the third child of Henry Lee III and Matilda Lee. His mother died on August 16, 1790, but not before putting Stratford Hall in trust to the younger Henry. The following year his father took office as the governor of Virginia and the family moved to Richmond. In 1793, Henry Lee III married Anne Hill Carter. Over the next two decades Lee IV would gain six half-siblings, including the future Confederate commander Robert E. Lee.

A gifted student, Henry Lee IV attended Washington Academy (later Washington and Lee University) and, from 1807 to 1808, the College of William and Mary. In 1808 he became the master of Stratford Hall after his father, having made some bad investments, fled the dilapidated estate and his creditors. (The former governor landed in debtor’s prison the following year.)

General James Wilkinson

Like his father, Lee had political and military ambitions. He represented Westmoreland County in the House of Delegates for three terms (1810–1813) before entering military service against Great Britain during the War of 1812. He was appointed a major in the 36th Regiment of the U.S. Infantry and saw action along the Canadian border, serving as an aide-de-camp to General James Wilkinson and later to General George Izard of South Carolina.

After the war, Lee returned to Stratford. With the help of his uncle Richard Bland Lee, he sought a military appointment. President James Madison eventually offered him the position of assistant inspector general for the southern division, which Lee declined. In 1816 he ran as a Federalist to represent Virginia’s Ninth Congressional District—the same seat his father had held from 1799 to 1801—but was defeated by William Lee Ball. One of three candidates, Lee managed to get only 14.5 percent of the vote.

Marriage and Scandal

Robert E. Lee and Son

After his congressional defeat, Lee remained active in social circles in Washington, D.C., and New York City. He was known more for his intellect than his looks, and for his tendency toward sarcasm and invective. No known image of Lee exists, but Governor Henry A. Wise once wrote of him, “He was not handsome as his half brother, General Robert E. Lee; but rather ugly in face, a mouth without a line of the bow of Diana about it, and nose not cut clean and classic, but rather meaty.” Yet, he added, Lee was “one of the most attractive men in conversation we ever listened to.” Indeed, he courted a number of women—notably Serena Livingston, one of the wealthiest young women in New York.

On March 29, 1817, Lee married Anne Robinson McCarty, his distant cousin and heiress with her younger sister to Pope’s Creek, a plantation adjoining Stratford. McCarty’s land holdings, combined with the 3,000-acre Stratford property, amounted to roughly 5,000 acres—a considerable bit of property. The marriage brought Lee not only land, but additional slaves and cash. He also became the guardian of his wife’s younger sister, Elizabeth “Betsy” McCarty, and the manager of her estate.

In 1818, Anne McCarty Lee gave birth to a daughter, Margaret. But two years later, the child fell down a stairway at Stratford Hall and died. The Lees never had another child, and a distraught Anne Lee turned to opium to alleviate her grief.

With the house in mourning, Henry Lee and Betsy McCarty had an affair. When rumor spread of an illicit pregnancy, the ensuing scandal rocked the Lee family. (The pregnancy rumor remains unconfirmed.) According to Virginia law, sex between siblings-in-law was considered incest; furthermore, Lee had embezzled money from his young ward’s estate. A contrite McCarty left Stratford, cut short her red hair, and began wearing a veil in public. She reinstated as her guardian her stepfather, Richard Henry Stuart, who then sued Lee for mismanaging McCarty’s estate. In 1822, a chancery court ruled that Lee owed McCarty one-half of the earnings from her Pope’s Creek estate from November 1817 to May 1821—a total of $11,568.97.

For his part, Lee was remorseless about the affair. He later wrote in a letter that a man who “associates with the sister of his wife, with all the unguarded intimacy which he would observe toward his own sister and there being no blood connection, no barrier of instinct between them, is liable to be surprised into adultery.” He did not face criminal charges of incest, but he was forced to sell Stratford Hall to cover his debt to McCarty. Lee’s friend William C. Somerville bought the house in June 1822 for $25,000. After Somerville’s unexpected death in 1826, the Westmoreland County Court seized the property and auctioned it to pay Somerville’s debts. The buyer was Henry Storke—who in 1826 had married Betsy McCarty. Elizabeth McCarty Storke was the mistress of Stratford until her death in 1879.

Later Years


Lee’s marriage survived the scandal, but his political career did not. Once considered one of the most promising men in the country, Lee could not escape his notoriety. Lee’s political enemies began to refer to him as “Black Harry” or “Black Horse Harry” —the latter a play on his father’s heroic Revolutionary War moniker, “Light-Horse Harry.” In 1825, with the help of John C. Calhoun and John Marshall, Lee secured a position in the post office—though it was at a salary, he is reported to have said, that would “hardly bribe a slave.” He hired out his family’s slaves to supplement his income and turned his attention to writing. He authored a history of his father’s military exploits, The Campaign of 1781 in the Carolinas (1824), and reissued his father’s memoirs. Lee then began writing a history of the War of 1812 and traveled to New York and Canada to conduct research. In 1826 he visited Monticello to speak with Thomas Jefferson, but Jefferson died before Lee could meet him.

Andrew Jackson

In 1826, Lee began corresponding with Andrew Jackson to get further information for his book. Henry and Anne Lee developed close ties with Jackson and his wife, Rachel, when Anne received treatment for her opium addiction at a health resort near the Jacksons’ Tennessee residence. Henry Lee wrote articles supporting Jackson and his 1828 presidential campaign and, at the request of Duff Green, an influential Democrat, began writing a campaign biography of the general. Although Lee never finished the work, his manuscript was used in later sources. Lee became Jackson’s speechwriter and publicist, drafting Jackson’s inaugural address and attending the new president’s swearing-in ceremony in March 1829.

Lee hoped Jackson would appoint him to a lucrative public office; in April 1829, he learned the president had appointed him consul to Algiers. Algiers was considered very dangerous, but Lee seemed to relish it. Upon moving there with his wife later that year, he claimed to keep six pistols, six muskets, and a Tennessee rifle in his house, and walked the streets armed. But his stay in Algiers was short-lived. When senators discovered the appointment in March 1830, they voted unanimously to have Lee recalled.

Observations on the Writings of Thomas Jefferson

Henry and Anne Lee never returned to the United States. After leaving Algiers, they toured Italy and then settled in Paris, France. Still short of money, Lee begged his half-brother Charles Carter Lee for funds and threw himself into his writing. In 1832, he published Observations on the Writings of Thomas Jefferson, a response to Thomas Jefferson Randolph‘s four-volume book on the man. Randolph’s book had included sharp criticisms of Light-Horse Harry Lee; in his Observations, Lee unleashed a barrage of invective against Jefferson. In 1834, Lee published Life of the Emperor Napoleon, the first of two planned volumes on the French military leader. The book was reviewed positively, but sold poorly. Lee never overcame his financial problems. In 1835, he sold his wife’s slaves for $9,000, but it was not enough to pay his debts.

On January 30, 1837, Lee died of the flu. He is buried in an unmarked grave in Montmartre. His death left Anne Lee alone in Paris, where she remained until her death in 1840. In a letter to a friend she wrote, “All my habitudes are of this country I have been here so long, and the idea of returning there [America] without him is horrible to me. I can have no satisfaction in association with those who disliked and persecuted by dear husband while living and who may probably rejoice that he is no more.”

Major Works

  • The Campaign of 1781 in the Carolinas (1824)
  • Observations on the Life of Thomas Jefferson (1832)
  • Life of the Emperor Napoleon (1834)

May 28, 1787
Henry Lee IV is born at Stratford Hall in Westmoreland County to Henry Lee III and Matilda Lee.
August 16, 1790
Matilda Lee, wife of Henry Lee III and mistress of Stratford Hall, dies. Before she dies, she places Stratford Hall in trust to her three children.
Henry Lee IV attends the College of William and Mary.
Henry Lee III flees Stratford Hall to escape his creditors. His son, Henry Lee IV, becomes the master of the estate.
Henry Lee IV represents Westmoreland County in the House of Delegates.
William Lee Ball wins election to the U.S. House of Representatives from Virginia's Ninth Congressional District, defeating Henry Lee IV.
March 29, 1817
Henry Lee IV marries Anne Robinson McCarty.
Margaret Lee is born to Henry Lee IV and Anne McCarty Lee.
Margaret Lee, the daughter of Henry Lee IV and Anne McCarty Lee, falls to her death down the exterior staircase at Stratford Hall.
A chancery court rules that Henry Lee IV owes his ward, Elizabeth "Betsy" McCarty, one-half of the earnings from her Pope's Creek estate from November 1917 to May 1821—a total of $11,568.97.
June 1822
William C. Somerville buys Stratford Hall from Henry Lee IV for $25,000.
After William C. Somerville's death, Westmoreland County Court seizes Stratford Hall and auctions the property. The buyer is Henry Storke.
President Andrew Jackson appoints Henry Lee IV consul to Algiers.
The U.S. Senate votes to recall Henry Lee IV as consul to Algiers.
January 30, 1837
Henry Lee IV dies of influenza in Paris, France.
  • Armes, Ethel. Stratford Hall: The Great House of the Lees. Richmond, Virginia: Garrett and Massie, 1936.
  • Nagel, Paul C. The Lees of Virginia: Seven Generations of an American Family. New York: Oxford University Press, 1990.
  • Pryor, Elizabeth Brown. Reading the Man: A Portrait of Robert E. Lee through His Private Letters. New York: Viking, 2007.
  • Royster, Charles. Light-Horse Harry Lee and the Legacy of the American Revolution. New York: Knopf, 1981.
APA Citation:
Woodward, Colin. Henry Lee (1787–1837). (2020, December 07). In Encyclopedia Virginia. https://encyclopediavirginia.org/entries/lee-henry-1787-1837.
MLA Citation:
Woodward, Colin. "Henry Lee (1787–1837)" Encyclopedia Virginia. Virginia Humanities, (07 Dec. 2020). Web. 21 Jul. 2024
Last updated: 2021, December 22
  • This field is for validation purposes and should be left unchanged.