The justices who sit on the Supreme Court of the United States hold a unique governing power, making their selection extremely fraught. Senators might oppose a nominee because of his or her record, or even because of disagreements senators have with the president who made the nomination. This tension has been present since the court’s beginning.

WATCH: The Supreme Court

As the first president, George Washington successfully appointed 10 justices to the Supreme Court. The only time the Senate rejected a Supreme Court nomination from Washington was when he tried to make former associate justice John Rutledge the chief justice in 1795.

Although the Senate was controlled by the Federalists, the party with which Washington was unofficially aligned, it rejected Rutledge over a speech he’d given opposing the Jay Treaty, which the Senate had recently approved.

Though this may not sound very dramatic today, at the time, Rutledge’s speech was considered such a faux pas that some actually questioned his mental stability. It was the first truly contentious attempt to nominate a Supreme Court justice.

The first president to have real difficulty filling Supreme Court vacancies was John Tyler, who was so unpopular his party kicked him out while he was still in the White House.

Critics dubbed Tyler “His Accidency” because he was the first vice president to assume the office after a president’s death. Tyler had been William Henry Harrison’s running mate on the 1840 Whig party ticket; and when Harrison died after only 31 days in office, the Whigs weren’t happy to be stuck with Tyler. Within a year, all but one of Tyler’s cabinet members resigned, and the Whigs expelled him from the party.

When it came time to fill Supreme Court vacancies, the Whig-controlled Senate only confirmed one out of the five men Tyler nominated. The Senate rejected, voted to postpone or just ignored his other nominations, marking the first time the Senate had so forcefully opposed such a large portion of a president’s Supreme Court selections.

Ulysses S. Grant was much more successful than Tyler at filling vacancies on the Supreme Court. This was during Reconstruction, when Black men were exercising their right to vote and winning seats in Congress for the first time. During Grant’s two terms in office from 1869 to 1877, the Republican-controlled Senate confirmed five of the Republican president’s eight nominees.

READ MORE: The First Black Man Elected to Congress Was Nearly Blocked From Taking His Seat

One of the Grant nominees the Senate didn’t confirm was 73-year-old Caleb Cushing, a former attorney general who had criticized abolitionists and written an introductory letter to Jefferson Davis while he was president of the Confederacy. Cushing had also supported the Supreme Court’s 1857 Dred Scott decision that Black Americans weren’t U.S. citizens—a decision Congress had since invalidated with the 1866 Civil Rights Act and the 14th Amendment.

The Senate strongly opposed Grant’s 1874 nomination of Cushing for chief justice, leading Cushing to ask Grant to withdraw his name from consideration before the Senate could even vote on him.

After Reconstruction ended in 1877, southern white men seized political power again by preventing Black men from voting, and in the process pushed out Black Republicans who had won seats in Congress during Reconstruction. In December 1887, when Democratic president Grover Cleveland nominated Democrat Lucius Quintus Cincinnatus Lamar—the first former Confederate nominated to the Supreme Court—Republicans held only a narrow majority in the Senate.

Lamar was a former member of the House of Representatives who’d resigned in 1860 in order to help Mississippi secede, and then became a colonel in the Confederate Army. He went back to Congress between 1873 and 1885, first retaking his old House seat and then joining the Senate. Even though his seditious actions and ties to railroad companies made 62-year-old Lamar an unpopular choice for the Supreme Court, the Senate confirmed its former member by 32 to 28 (16 senators didn’t vote). Lamar served on the court until his death in 1893.

Nearly a century after a former Confederate took the bench, Republican candidate Richard Nixon ran for president using a “Southern strategy,” as well as an emphasis on law-and-order politics, to woo white southern Democrats who were upset with Lyndon B. Johnson’s support for civil rights legislation. After winning the presidency in 1968, Nixon nominated two men for the Supreme Court who each had a history of supporting segregation.

READ MORE: How the ‘Party of Lincoln’ Won Over the Once Democratic South

The first one, Clement Haynsworth Jr., had supported a decision by Prince Edward County, Virginia—the county in which one of the five class-action suits incorporated into Brown v. Board of Education was filed—to close schools rather than integrate them. Civil rights and labor activists criticized his record, and pointed out that he had a financial conflict of interest in one of the cases he’d decided. This was especially troubling for the Democrat-controlled Senate because, if confirmed, Haynsworth would replace Abe Fortas, who himself had resigned from the Supreme Court over financial conflicts of interest.

The Senate rejected Haynsworth in 1969, and the next year Nixon nominated G. Harrold Carswell to replace Fortas. Some critics claimed Carswell’s career was mediocre, leading one Republican senator to mount this defense: “Even if he is mediocre, there are a lot of mediocre judges and people and lawyers, and they are entitled to a little representation, aren't they?”

Far more damning than this “endorsement” was a statement Carswell had made when he was running for the Georgia legislature in 1948: “I yield to no man as a fellow candidate or as a fellow citizen in the firm, vigorous belief in the principles of white supremacy, and I shall always be so governed.” Civil rights groups highlighted this statement and his support for segregation, and the Senate rejected him as a Supreme Court justice.