At the time of the Constitutional Convention, the majority of the new nation’s citizens lived in cities like Philadelphia or Boston, while the Southern states were more rural and sparsely populated. African American enslaved people made up a full 40 percent of the South’s population, and Southern delegates wanted them to be counted along with white citizens when it came to calculating how many representatives their states would receive in Congress. Northerners, on the other hand, argued that slaves were property, and didn’t require representation.

This ugly debate was resolved with the so-called “three-fifths compromise,” by which each Black person would count as three-fifths of a person when determining congressional representation for each state by population. As each state’s number of electors in the Electoral College was equal to its number of representatives in Congress, this compromise also affected how the country elected its executive branch.

READ MORE: What Is the Electoral College and Why Was It Created?

The framers put a great deal of thought into establishing the Electoral College, but were still uncertain exactly how it would work in practice. The first serious problem emerged in the election of 1800, when Thomas Jefferson, the Democratic-Republican candidate for president received the same number of electoral votes as his chosen running mate, Aaron Burr.

The election went to the House of Representatives, dominated by the rival Federalist Party; some of the party’s members saw Burr as less objectionable than Jefferson, and wanted him to become president. After a chaotic process, including no fewer than 36 votes, Jefferson was elected president and Burr V.P. The whole debacle led to the adoption of the 12th Amendment, which mandated that electors specify their choices for president and vice president.

Despite that crucial improvement to the Electoral System, there was still a major issue looming: What if the winner of the popular vote, the candidate chosen by the majority of the nation’s citizens, failed to win the Electoral College, and didn’t become president? This happened for the first time in 1824, to Andrew Jackson, who had risen from the backwoods of the Carolinas, with little formal education, to become a successful slaveholding lawyer in Tennessee and the celebrated hero of the War of 1812.

READ MORE: 5 Presidents Who Lost the Popular Vote But Won the Election

All four leading candidates in the 1824 election were from the same party, the Democratic-Republicans. Though Jackson got the most electoral votes (99) and held a slight lead (38,149 votes) in the popular vote, no candidate received a majority in the Electoral College, and the decision again fell to the House of Representatives, who would decide between the top three finishers, Jackson, John Quincy Adams (who received 84 electoral votes) and Secretary of State William Crawford.

Speaker of the House Henry Clay, who had come in fourth in the presidential race and hated Jackson, suddenly had tremendous influence over the election results. The Kentuckian forged an alliance between electors in Ohio and New England (including states that had voted decisively for Jackson) in order to hand the White House to Adams, who later appointed Clay as his secretary of state.

The so-called “corrupt bargain” haunted Adams’ presidency and allowed Jackson to effectively market himself as the common man’s champion against an elitist, corrupt system. “Old Hickory” won the White House in 1828, and in his first annual message to Congress he recommended eliminating the Electoral College.

Since then, four other U.S. presidential elections (1876, 1888, 2000 and 2016) have ended with a candidate winning the Electoral College but losing the popular vote. This apparent disconnect between the democratic ideal of the people playing a direct role in government and the reality of the system in practice has fueled the continuing debate over the Electoral College, and led many people over the years to echo Jackson’s call to abolish it.

READ MORE: How Are Electoral College Electors Chosen?

Critics of the Electoral College system call it a relic of the 18th century—when only three-fifths of a black person was counted, and black men, women and white men who didn’t own property couldn’t vote—and argue that it doesn’t fairly represent our nation as it exists today. In recent elections, national campaigns have increasingly focused on a small handful of “battleground” states whose electoral votes are up for grabs, effectively depriving millions of citizens (as many as four out of every five Americans, according to some analysts) of their voice in the electoral process.

Another potential weakness in the Electoral College system is the existence of so-called “faithless electors,” who for whatever reason choose to vote against their state’s chosen candidate. There have been 157 such faithless electors over the years, starting way back in 1796, when a Pennsylvania elector changed his vote from John Adams to Thomas Jefferson, for reasons that remain unclear. Today, many states require electors to take a formal pledge to uphold their state’s choice, but 21 states have no such requirement.

On the other hand, supporters of the Electoral College see it as an integral part of our federal union, and a necessary way to give the states, as well as the people, a role in choosing the leadership of the government. Now, as in 1787, it’s seen as a critical way of preventing large urban centers from dominating the political process at the expense of rural areas.

READ MORE: How the Great Compromise Affects Politics Today

In any case, eliminating the Electoral College would not be an easy process. Congress, or a national constitutional convention, would have to propose an amendment, which would then have to be ratified by either three-quarters of state legislatures (currently 38) or by state ratifying conventions in three-quarters of the states.

Those who argue in favor of the current system can point to the fact that in more than 200 years, the electoral and popular votes have split just five times (nearly 10 percent of presidential elections), and faithless electors have never changed the result of an election. Given that statistic, perhaps many would agree with Alexander Hamilton, who in No. 68 of the Federalist Papers wrote of the new Electoral College system that “if the manner of it be not perfect, it is at least excellent.”