If a Mount Rushmore for America’s most unpopular presidents is ever created, John Tyler would be a leading candidate to have his likeness carved into stone.

“Popularity, I have always thought, may aptly be compared to a coquette—the more you woo her, the more apt is she to elude your embrace,” said America’s 10th president. Playing hard to get, though, also failed to garner Tyler popular affection. The maverick president’s fierce independent streak succeeded only in alienating politicians on both sides of the aisle.

Six years after Tyler left the Democratic Party over differences with President Andrew Jackson, the rival Whig party nominated the former congressman, senator and Virginia governor in 1840 as William Henry Harrison’s running mate. After the victory of their “Tippecanoe and Tyler too” ticket, the 68-year-old Harrison became the oldest president in the country’s short history. Tyler, deeming the vice president’s duties largely irrelevant, returned home to his Virginia plantation.

Just 31 days after the inauguration, however, Tyler was stirred from his sleep by a rap on the door and given the news that Harrison had become the first American commander-in-chief to die in office. Upon returning to the nation’s capital, Tyler took the presidential oath, angering strict constructionists who argued that the Constitution only specified that, when a president died, the vice president would inherit presidential “powers and duties”—not the office itself. Former president John Quincy Adams wrote that Tyler was “in direct violation both of the grammar and context of the Constitution,” and eight senators voted against a resolution recognizing Tyler as the new president.

Those questioning Tyler’s legitimacy nicknamed the president “His Accidency.” Fellow Whigs would soon call him much worse.

The new president scoffed at his first cabinet meeting when Secretary of State Daniel Webster informed him that Harrison had agreed to abide by the majority decision of the cabinet on any policy matter—even if he was personally opposed. “I can never consent to being dictated to,” Tyler informed his cabinet. “I am the president, and I shall be responsible for my administration.” He made it clear he would neither serve as an interim “acting president” nor carry out all of his predecessor’s agenda, which included re-establishment of a national bank and protective tariffs.

This infuriated Whig leaders, in particular Senator Henry Clay. After Tyler twice vetoed Clay’s bill to re-establish a national bank, supporters of the senator forced open the White House gates, hurled stones at the presidential mansion and shouted, “Groans for the traitor!” They hanged the president’s effigy—and then burned it on the White House porch for good measure.

Clay engineered a mass resignation of the cabinet with only Webster, who was in the midst of a treaty negotiation, remaining. The Whigs excommunicated the president from the party and tried to evict him from the White House altogether after he vetoed yet another one of their bills. In July 1842, Representative John Botts of Virginia introduced the first impeachment resolution against a president in American history, accusing him of being “utterly unworthy and unfit to have the destinies of this nation in his hands.” The House approved an investigative committee’s report that condemned Tyler for “gross abuse of constitutional power” but declined to further pursue impeachment proceedings.

Expelled by the Whigs, then rebuffed in his attempts to return to the Democrats, Tyler became a president without a party. After his efforts to form a third party failed, he was forced to drop out of the 1844 presidential election.

Tragedy also seemed to stalk Tyler during his presidency. His wife, Letitia, died in 1842, and he was on board the USS Princeton on February 28, 1844, for a sail on the Potomac River when one of its cannons exploded during a ceremonial firing, killing six people including Secretary of State Abel Upshur, Secretary of the Navy Thomas Walker Gilmer and the president’s enslaved valet, Armistead.

On Tyler’s last full day as president, Congress gave him one final rebuke by passing the first override of a presidential veto in American history, on a bill that required legislative approval of any appropriation of federal money to build revenue cutter ships, predecessors to the U.S. Coast Guard.

Upon leaving the White House, America’s 10th president returned to his Virginia plantation where he owned dozens of slaves. Clay expressed his pleasure at Tyler’s departure and said the Whig political outlaw could return, like Robin Hood, to his Sherwood Forest. Embracing the gibe, Tyler changed the name of his plantation from Walnut Grove to Sherwood Forest.

In his post-presidential years, Tyler opposed limitations on the expansion of slavery and after the 1860 election of Abraham Lincoln wrote, “The day of doom for the great model republic is at hand.” As southern states began to secede, Tyler in early 1861 chaired an unsuccessful peace conference in a last-ditch attempt to preserve the Union. Once the Civil War began, however, Tyler voted for Virginia to leave the nation over which he once presided. He led the committee negotiating the terms of Virginia’s admission into the Confederacy and won election to the Confederate House of Representatives. He died, however, on January 18, 1862, before taking his seat.

A Confederate flag draped Tyler’s coffin as it was brought for burial to a Richmond, Virginia, cemetery. While bells tolled and flags were lowered to half-staff in the Confederate capital, silence greeted the news of Tyler’s death in the country he betrayed. Lincoln did not issue the customary official proclamation to observe Tyler’s passing, while the New York Times obituary noted that he had left the presidency as “the most unpopular public man that had ever held any office in the United States.”

Some of Tyler’s successors didn’t think very highly of him either. Harry Truman called him “one of the presidents we could have done without.” “He has been called a mediocre man; but this is unwarranted flattery,” said Theodore Roosevelt. “He was a politician of monumental littleness.”

Tyler hasn’t rated highly in the eyes of historians, either. He was ranked in the bottom five presidents in C-SPAN’s 2017 Presidential Historians Survey, along with Warren Harding, Franklin Pierce, Andrew Johnson and James Buchanan. What may have saved Tyler from the ranking’s bottom spot were his foreign policy achievements, including the signing of the Webster-Ashburton Treaty that formalized the U.S.-Canada border, negotiation of the first U.S.-China treaty, and securing congressional approval of the admission of Texas to the Union.

His most enduring mark on the presidency, however, was the “Tyler Precedent” that the vice president automatically assumes the office of the presidency after the death of a president. Seven subsequent vice presidents assumed the presidency following the demises of their predecessors until presidential succession was finally codified in the 25th Amendment, which was ratified in 1967.