Every U.S. president has been a member of a church, except for one: Abraham Lincoln.

Famously opaque on the subject of religion, Lincoln’s personal faith was something even his closest friends said they couldn’t figure out. Lincoln rented pews for his family at First Presbyterian Church in Springfield, Illinois, and New York Avenue Presbyterian Church in Washington, D.C., but never formally joined either one. And though he became more interested in religious questions toward the end of his life, “Honest Abe” never directly identified himself as a Christian—even after he realized it could hurt him politically.

“He once spoke of how not having any kind of noticeable religious profile had levied what he called a tax on his popularity with the voters,” says Allen Guelzo, a professor of Civil War-era studies at Gettysburg College and author of Abraham Lincoln: Redeemer President. “It was something that he was aware of, something he tried to cope with, and yet he wouldn’t go the distance of trying to pretend that he was something that he wasn’t.”

Lincoln’s religious views shifted throughout his life, as most people’s do. He grew up in a Baptist household but was never baptized as a child or an adult, and in his early 20s he was outspoken about his religious skepticism.

“He would actually be aggressive on the subject of unbelief,” Guelzo says. “More than one observer who knew him from those days said that Lincoln could shock people.” For example, he might say the Bible was just an ordinary book, or that Jesus Christ was an illegitimate child. “By the time he moves into his late 20s and early 30s, he has started to temper that because he realizes that doesn’t get him very far politically.”

During his failed campaign to be a Whig nominee for the U.S. House of Representatives in 1843, Lincoln observed that his absence of religious affiliation hurt him. “It was everywhere contended that no Christian ought to go for me, because I belonged to no church,” he wrote. Three years later, after he secured the Whig nomination for the House, he faced more accusations about his faith from his opponent, a revivalist preacher named Peter Cartwright. By then Lincoln had learned not to flout his skepticism, and knew he needed to address his critics.

“That I am not a member of any Christian Church, is true,” he responded in a handbill; “but I have never denied the truth of the Scriptures; and I have never spoken with intentional disrespect of religion in general, or of any denomination of Christians in particular.”

Still, Lincoln didn’t actually say whether he believed in the Christian faith. Instead, “he vigorously denies accusations that were not actually leveled at him,” Guelzo says. “He just deflects.”

Lincoln won that election and continued to be tight-lipped about his personal faith into his 50s. Yet a series of traumatic events—the death of his son Edward Baker in 1850, the outbreak of the Civil War in 1861 and the death of his son William Wallace in 1862—pushed him into a period of religious reflection, in which he thought seriously about what a hypothetical God might want for the U.S. and the institution of slavery.

“In the present civil war it is quite possible that God's purpose is something different from the purpose of either party,” Lincoln wrote in his personal papers in September 1862. “[God] could have either saved or destroyed the Union without a human contest. Yet the contest began. And, having begun He could give the final victory to either side any day. Yet the contest proceeds.”

Lincoln told his friend, Senator Orville Hickman Browning, that he thought God wouldn’t favor the Union’s cause unless it sought to end slavery—something he had not initially sought to do when the war began. He only mentioned his idea that God wanted to end slavery to a few people, and they were usually shocked to hear him say it since he didn’t regularly attend church, pray or speak about his faith. One of these instances was during a cabinet meeting at which he said he wanted to issue the Emancipation Proclamation.

“When he’s asked why he’s doing this,” Guelzo says, “the answer he says is: I made a vow, a covenant, with my maker, that if the Union army defeated the Confederate army in Maryland—which it did at the Battle of Antietam—I would send a proclamation after them… And it was so astonishing to his cabinet that one member of the cabinet actually asked him to repeat himself.”

After Lincoln was assassinated in 1865, Mary Todd Lincoln defended her late husband from questions about his faith while also giving conflicting accounts of what he actually believed, or when he started to believe it. Despite his later assertion that it was God’s will to end slavery, some of his close friends admitted that they had no idea what Lincoln’s broader religious views were because he was so secretive about them.

Over 150 years later, it’s even harder for us to tell what Lincoln believed. Still, his Bibles are among the most famous of any president’s. Both Barack Obama and Donald Trump used the so-called “Lincoln Bible” from his inauguration at their own swearing-in ceremonies. Ironically, Lincoln didn’t actually acquire that Bible until he showed up for his first inauguration in the middle of the night. A Supreme Court clerk brought him a Bible to swear on, and that’s the one most associated with him today.