Early Years
Willoughby was born on April 3, 1830, in Groton, Tompkins County, New York, and was the son of Franklin Willoughby and Kesia Delano Willoughby. Educated at academies in Groton and Homer, in adjacent Cortland County, Willoughby graduated in 1854 from Hamilton College in Clinton, New York, earning his way by teaching and working as a carpenter. In his senior year he received an award for an essay on “The Imagination: Its Nature and Relations to Oratory,” and at graduation he had the highest academic ranking in his class and spoke on “The Characteristics of Genius.” Willoughby then studied law while teaching at academies in Aurora and Moravia, both in nearby Cayuga County. In May 1857 he was admitted to the New York bar at Owego. Willoughby then practiced law in Groton, where on May 10, 1859, he married Jennie Rebecca Woodbury, a schoolteacher. They had twin sons and one daughter.
Willoughby helped recruit solders for the 137th New York Volunteer Regiment. His November 1, 1862, commission dated from September 5, when his three-year enlistment presumably began. Willoughby was severely wounded by artillery fire at the Battle of Chancellorsville on May 3, 1863, and resigned as a result of disability in September of that year. He settled in Alexandria and resumed the practice of law. On July 5, 1864, Willoughby became the commonwealth’s attorney of Alexandria County (later Arlington County). During the trial of a burglary case in May 1866, Judge H. W. Thomas refused to permit an African American woman to testify against a white man, citing a state law prohibiting such testimony still in effect at that time. Willoughby argued that a federal statute voided the prohibition, and he was apparently so incensed by the judge’s behavior that he went to Richmond and obtained an indictment of Thomas for violating the witness’s civil rights. Later in his career Willoughby reportedly succeeded in seating African Americans on juries. He attested to his progressive views in a different way on May 6, 1870, when he and his wife attended the organizational meeting of the Virginia State Woman Suffrage Association in Richmond and were among those who accepted election as vice presidents.
Judicial Career
In 1869 the commanding general then in charge of Virginia removed most of the judges of the state’s circuit courts to comply with a federal statute requiring dismissal of Virginians with Confederate backgrounds. On March 22, 1869, he appointed Willoughby to replace Thomas as judge of the Ninth Circuit, which encompassed Alexandria, Fairfax, Fauquier, Loudoun, Prince William, Rappahannock, and Stafford counties. Willoughby took his qualifying oaths the same day but held the post for only three months. As part of the same sacking of former Confederates, the commanding general also removed the members of the Virginia Supreme Court of Appeals, the state’s highest tribunal. On June 12, 1869, he named Willoughby one of the three replacements, all of whom were active or former Union officers. Willoughby took the qualifying oaths three days later.The reconstituted court met from June 22 to 28, 1869, again on October 12, 1869, and finally from January 11 to February 25, 1870. Willoughby was the least senior judge, but he made his mark, writing six of the opinions in the eight cases the judges decided and that were reported in the official volumes of the Virginia Reports. He was never in the minority and cast the decisive vote in perhaps the court’s most important ruling, a decision allowing officials for an Alexandria railroad who had sided with the Confederacy to contest as fraudulent the sale of the line’s assets during the Civil War. Willoughby’s tenure in office ended after the constitution ratified in 1869 took effect early in 1870, but unlike Horace B. Burnham, the court’s president, the General Assembly permitted him to serve to the conclusion of the February session.
Law Practice and Political Career
For the remainder of his life Willoughby practiced law in Alexandria and Washington, D.C., to which he moved in about 1887. Much of his work involved real estate, and he lectured on the law of real property in 1889 and 1890 at National University School of Law (later part of the law school of the George Washington University). In 1871 Willoughby and Charles Whittlesey, recently the state’s attorney general, formed a partnership specializing in seeking reimbursement from the recently created Southern Claims Commission for Union supporters who had had property seized by Northern armies during the Civil War. Five years later Willoughby participated in a failed effort to return Alexandria to federal control by contesting the validity of the 1846 retrocession of part of the District of Columbia to Virginia.
The federal government occasionally employed Willoughby as one of its attorneys to defend seizures of real estate owned by Confederate sympathizers, generally accomplished by wartime auctions to recover unpaid taxes. After the war ended the original owners or their heirs sued to recover their property, arguing that the tax commissioners had unjustly refused to allow the taxes to be paid by anyone but the owners themselves, who were then behind Confederate lines. Willoughby participated in a series of cases culminating in the lawsuit George Washington Custis Lee filed to obtain compensation for his family’s Arlington estate. Willoughby and his co-counsel maintained that under the doctrine of sovereign immunity the courts lacked jurisdiction in such instances. In a ruling that attracted national attention, a sharply divided U.S. Supreme Court rejected Willoughby’s position and ruled in 1882 that even the descendants of the leading Confederate general were entitled to compensation.
Willoughby was active in the Republican Party as early as May 1868, when he attended a Radical meeting in Alexandria. He subsequently lost bids for the state senate in 1871 and city council in 1873, spoke at and chaired local political meetings attended by both African Americans and whites, and was nominated for attorney general of Virginia in 1881 at the Lynchburg convention of the Republican Party. The convention’s delegates, called Straightout Republicans, were unwilling to join with other Republicans and support the Readjuster Party‘s program of paying for public education by repudiating some of the state’s prewar debts. Willoughby was apparently not at the convention and quickly withdrew his name from consideration, as did the other nominees. The Straightouts then concluded they could best defeat the Readjusters by running no slate that might draw support away from the Readjusters’ principal opponents, the Conservative Party. In 1884 Willoughby attended a Straightout convention in Richmond, and later that year he spoke in support of the presidential candidacy of Republican presidential nominee James G. Blaine.
Later Years
Willoughby supplemented his legal career with involvement in a number of corporate ventures. He was president of the Virginia Peat Company in 1870, a director of the East Fairmont Gas, Coal and Coke Company of West Virginia in 1876, vice president of the Arlington-Breckenridge Mining Company in 1884, and president of the Columbia Dredging and Construction Company in 1897. He was also active in the Presbyterian Church, the Young Men’s Christian Association, and the alumni association of Hamilton College. Willoughby died at his Washington residence on December 21, 1897, and was buried two days later at Arlington National Cemetery.