The United States may have been founded on the idea that all men are created equal, but during the late 18th and early 19th centuries, slaveholding was common among the statesmen who served as president. All told, at least 12 chief executives—over a quarter of all American presidents—enslaved people during their lifetimes. Of these, eight held enslaved people while in office.

The “peculiar institution” loomed large over the first few decades of American presidential history. Not only did enslaved laborers help build the White House all of the earliest presidents (except for John Adams and his son John Quincy Adams) owned enslaved people. George Washington kept some 300 bondsmen at his Mount Vernon plantation. Thomas Jefferson—despite once calling slavery an “assemblage of horrors”—owned at least 175 enslaved people at one time. James Madison, James Monroe and Andrew Jackson each kept several dozen enslaved people, and Martin Van Buren owned one during his early career.

William Henry Harrison owned several inherited enslaved people before becoming president in 1841, while John Tyler and James K. Polk were both enslavers during their stints in office. Zachary Taylor, who served from 1849-1850, was the last chief executive to keep enslaved people while living in the White House. He owned some 150 enslaved people on plantations in Kentucky, Mississippi and Louisiana.

Perhaps surprisingly, the last two presidents to own enslaved people were both men closely associated with Abraham Lincoln, who led the nation during a civil war caused in large part by the divisions sowed by slavery, and later signed the Emancipation Proclamation and championed passage of the 13th Amendment ending slavery. Andrew Johnson, who served as Lincoln’s vice president before becoming president in 1865, had owned at least half a dozen enslaved people in his native Tennessee and even lobbied for Lincoln to exclude the state from the Emancipation Proclamation.

The last president to personally own enslaved people was Ulysses S. Grant, who served two terms between 1869 and 1877. The former commanding general of the Union Army had kept a lone Black enslaved man named William Jones in the years before the Civil War, but gave him his freedom in 1859. Grant would later sum up his evolving views on slavery in 1878, when he was quoted as saying that it was “a stain to the Union” that people had once been “bought and sold like cattle.”