Carter was the son of John Carter and Elizabeth Hill Carter and grandson of. He was born in 1732 probably at Shirley, the Charles City County plantation where his parents lived while his father was secretary of the and a member of the . Carter’s father died in July 1742, and several years later his mother married Bowler Cocke, who moved from Henrico County to Shirley plantation. Carter was educated at the Gloucester County grammar school of William Yates, a clergyman who later briefly served as president of the College of William and Mary, and at the college. He then assumed management of Corotoman, the Lancaster County plantation that he had inherited from his father. Carter resided there for two decades and supervised that and his other properties on the Northern Neck and elsewhere. He purchased Nanzatico, a 2,200-acre King George County plantation, from his financially strapped cousin . Following the deaths of his mother and Bowler Cocke in 1771, Carter made extensive repairs and renovations to the main house at Shirley and moved there permanently about 1775.
Carter’s success as a planter and entrepreneur made him one of the wealthiest men in Virginia. At the time of his death in 1806, his will enumerated more than 13,000 acres in thirteen Virginia counties and at least 710 slaves. In addition to his extensive landholdings Carter operated a large mill near Corotoman, lent substantial sums of money to other planters, and provided in his will for investing some assets in bank stocks or similar securities. He was also a sometime administrator and trustee of the complicated estate left by his brother-in-law, the councillor(1728–1777). Although Carter’s uncle, , characterized him as an inefficient plantation manager, the uncle’s pessimistic frame of mind and negative assessments of nearly everyone suggest that the criticism was overstated.
Sometime in the mid-1750s Carter married his first cousin Mary Walker Carter, daughter of, who represented King George County in the House of Burgesses for many years. They had two daughters and six sons (including one set of twins) before her death on January 30, 1770. Late in November of that year he married Ann Butler Moore. Of their eight daughters and seven sons, four children died in infancy, and one was stillborn. As had been the case in his generation and the generations of his parents and grandparents, nearly all of the Carter children who lived to adulthood married into . The best-known example was the marriage in 1793 of his and Ann Butler Moore Carter’s eldest daughter, Ann Hill Carter, to .
Carter remained active in the affairs of the church and attended annual Episcopal Church conferences in Virginia from 1789 to 1793 as a lay delegate from Westover Parish. Carter died at Shirley on June 28, 1806. His will directed that he be buried near the bodies of his parents (probably at Shirley or in the graveyard of the Westover Parish church) “without any funeral pomp and nothing but the burial service [to] be read over my grave by the parson of the parish (should we be so fortunate as to have one among us).”