Reading about the president’s sex life is still a pretty new phenomenon. Americans first experienced it in the 1990s, when the cover-up of Bill Clinton’s affair with White House intern Monica Lewinsky—and Clinton’s subsequent impeachment trial—became a huge news story.

“Until Bill Clinton … there had not been coverage of any infidelity on the part of a sitting president in the mainstream press,” says Alison Dagnes, a political science professor at Shippensburg University and editor of Sex Scandals in American Politics.

Before then, journalists didn’t report on presidential affairs because they didn’t consider them newsworthy. Even if a president’s affairs were well-known to his friends, staffers, and journalists, Dagnes says the public didn’t learn about them until “after the president was out of office, most of the time after the president was long dead.”

Following the death of Warren G. Harding, a president who only served two years in office before dying of a heart attack in 1923, his mistress Nan Britton released a juicy tell-all book. This is when the public learned of the president’s supposed predilection for having sex in a White House coat closet.

Britton, who was 31 years Harding’s junior, had met him in his home state of Ohio when she was a teenager and he was running for U.S. Senate. In her book, The President’s Daughter, Britton revealed that she’d secretly had Harding’s daughter, Elizabeth Ann, and also that they’d probably conceived on Harding’s Senate office couch. Harding’s relatives disputed this all the way up until 2015, when a DNA test showed that Harding’s grandniece and grandnephew were second cousins with Britton’s great-grandson—proving that Elizabeth Ann was indeed his daughter (but still leaving open the question of whether it happened on the couch).

Still, that isn’t even the most surprising detail about Harding’s sex life to surface in the 21st century. In 2014, a series of racy love letters between Harding and another mistress, Carrie Fulton Phillips, were finally unsealed. His family had fought to keep them private for decades, presumably because they were embarrassed by his codename for his penis (It was “Jerry.” Example: “Wish I could take you to Mount Jerry.”)

Like Harding, Franklin D. Roosevelt had a long-time mistress (and possibly others) that the public didn’t know about until after his death. FDR started his affair with Lucy Mercer, Eleanor Roosevelt’s social secretary, more than a decade before he became president. When Eleanor discovered some of their love letters in September 1918, she confronted her husband about them.

The couple considered a divorce, but stayed together in order to protect their social standing and FDR’s career. Even though the episode had deeply hurt Eleanor, historian Doris Kearns Goodwin writes in No Ordinary Time that it ultimately made her “free to define a new and different partnership with her husband, free to seek new avenues of fulfillment.”

FDR would maintain a relationship with Mercer until his death in 1945. In turn, Eleanor pursued relationships with women, finding a particularly significant partner in journalist Lorena Hickok.

By coming to a mutually respectful arrangement, the Roosevelts were an anomaly among presidential couples. Lyndon B. Johnson, on the other hand, seemed crudely unconcerned with how his long-term affair with Alice Glass might affect his wife, Lady Bird Johnson.

LBJ maintained his on-and-off relationship with Glass between 1939 and the early years of his presidency. It was doubly risky for him because she was also romantically involved with one of his big donors. According to LBJ biographer Robert Caro, she eventually broke up with him because of her opposition to the Vietnam War.

It seems pretty clear that Lady Bird knew about Glass and the other women he slept with. When Lady Bird died in 2007, The Guardian noted in her obituary: “[Lyndon] Johnson was so casual in his affairs with Alice Glass and his congressional colleague Helen Gahagan Douglas that Lady Bird … was openly humiliated.” This isn’t surprisingly considering that LBJ was known for exposing his genitals and bragging that he’d had sex with more women than John F. Kennedy—whose exploits were well-known in D.C.

JFK’s affairs stayed out of the press not necessarily because journalists wanted to protect him, but because they just didn’t consider such things to be news. At one point during JFK’s presidency, the Secret Service tackled CBS journalist Marvin Kalb so he wouldn’t get a good look at a woman entering the president’s hotel room.

Even though he’d been manhandled, Kalb later wrote: “never for one moment did I even consider pursuing and reporting what I had seen and experienced that evening.”

But despite the fact that no one wrote about it at the time, most Americans are by now aware that JFK slept around. Many even suspect JFK slept with Marilyn Monroe, the actress who famously sang him a breathy version of “Happy Birthday,” though there’s still no evidence of that. He did, however, have verifiable affairs with many other known women, including a 19-year-old intern named Mimi Beardsley and his wife’s friend Mary Pinchot Meyer, whose 1964 murder is still unsolved.

During the early 1960s, he had a relationship with Judith Campbell Exner, who had previously dated Frank Sinatra and would later have an affair with mobster Sam Giancana. Fittingly, the details of her relationship with the president became public in a swirl of intrigue. Her story came out in 1975, when the Senate was investigating whether JFK had tried to use the Mafia to oust Fidel Castro in Cuba.

Over time, historians have come to understand a couple of these supposedly “consensual” presidential affairs to be coercive.

Though Sally Hemings was once known as Thomas Jefferson’s “mistress,” today’s scholars emphasize that the enslaved Hemings had no legal right to consent to or decline sex with him. Similarly, Grover Cleveland’s illegitimate child with a woman named Maria Halpin wasn’t widely understood as rape until recently (in an affidavit, she said he’d impregnated her “[b]y use of force and violence and without my consent”). And today, some liberal writers have called on Democrats to reevaluate the claims made against Bill Clinton.

During his campaign and presidency, Donald Trump has been accused of sexual harassment or assault by at least 16 women, and separately linked to two consensual affairs. In particular, adult film actress Stormy Daniels has received a lot of media coverage for her assertion that Trump tried to cover up their consensual tryst with her with a non-disclosure agreement (NDA), then disregarded its terms.

As with Clinton, it’s not the affair itself, but rather the cover-up, that could get Trump in legal trouble. However, Dagnes is doubtful that the Stormy Daniels legal challenge will end up impacting Trump or his fans’ opinion of him in a major way. As an illustration, she points to a 1994 article by Tony Kornheiser in The Washington Post: “if there were a story tomorrow involving Bill Clinton, a vat of lime Jell-O, two orangutans and the June Taylor Dancers, almost everybody in America would: a. believe it; and, b. not care.”

To be sure, there are plenty of people who do care about the claims made about Trump. But as he once said of his supporters, “I could stand in the middle of 5th Avenue and shoot somebody and I wouldn’t lose voters.”