On February 14, when we share chocolates, special dinners or doily cards with our loved ones, we do it in the name of Saint Valentine. But who was this saint of romance?

Search the internet, and you can find plenty of stories about him—or them. One Saint Valentine was supposedly a Roman priest who performed secret weddings against the wishes of the authorities in the third century. Imprisoned in the home of a noble, he healed his captor’s blind daughter, causing the whole household to convert to Christianity and sealing his fate. Before being tortured and decapitated on February 14, he sent the girl a note signed “Your Valentine.”

Some accounts say another saint named Valentine during the same period was the Bishop of Terni, also credited with secret weddings and martyrdom via beheading on February 14.

Unfortunately for anyone hoping for a tidy, romantic backstory to the holiday, scholars who have studied its origins say there’s very little basis for these accounts. In fact, Valentine’s Day only became associated with love in the late Middle Ages, thanks to the English poet Geoffrey Chaucer.

“The two stories that everybody talks about, the bishop and the priest, they’re so similar that it makes me suspicious,” says Bruce Forbes, a professor of religious studies at Morningside College in Iowa.

Valentine was a popular name in ancient Rome, and there are at least 50 stories of different saints by that name. But Forbes said the earliest surviving accounts of the two February 14 Valentines, written starting in the 500s, have a whole lot in common. Both were said to have healed a child while imprisoned, leading to a household-wide religious conversion, and they were executed on the same day of the year and buried along the same highway.

The historical evidence is so sketchy that it’s not clear whether the story started with one saint who then became two or if biographers of one man borrowed details from the other—or if either ever existed at all.

Perhaps more disappointing for the romantics among us, the early accounts of the two Valentines are typical martyrdom stories, stressing the saints’ miracles and gruesome deaths—but containing not a word about romance.

“They’re both mythical to begin with, and the connection with love is even more mythical,” says Henry Kelly, a scholar of medieval and renaissance literature and history at UCLA.

Saint Valentine’s Day has also been associated with a Christian effort to replace the older holiday of Lupercalia, which Romans celebrated on February 15. Some modern stories paint Lupercalia as a particularly sexy holiday, when women wrote their names on clay tablets which men then drew from a jar, pairing up random couples.

But, again, early accounts don’t support this. The closest parallel between Lupercalia and modern Valentine’s Day traditions seems to be that the Roman festival involved two nearly naked young men slapping everyone around them with pieces of goat skin. According to the ancient writer Plutarch, some young married women believed that being hit with the skins promoted conception and easy childbirth.

Whatever minor romantic connotations might have been part of Lupercalia, they didn’t translate to the new Christian holiday.

“It just drives me crazy that the Roman story keeps circulating and circulating,” Forbes says. “The bottom line for me is until Chaucer we have no evidence of people doing something special and romantic on February 14.”

So how did Chaucer create the Valentine’s Day we know today? In the 1370s or 1380s, he wrote a poem called "Parliament of Fowls" that contains this line: “For this was on Saint Valentine’s Day, when every bird comes there to choose his mate.”

This was a moment in Europe when a particular set of romantic ideas took shape. Chaucer and other writers of his time celebrated romance between knights and noble ladies who could never marry—often because she was married already—creating tropes of yearning and tragic obstacles that still drive our romantic comedies today.

By the 1400s, nobles inspired by Chaucer had begun writing poems known as “valentines” to their love interests. It was only at this point that stories began to appear linking Saint Valentine to romance.

But there’s one final twist in the myth of Saint Valentine. When Chaucer wrote of the day when every bird chooses a mate, Kelly argues that he was thinking not of February 14, but of May 3, a day celebrating one of the many other Saint Valentines. After all, England is still awfully cold in mid-February.

In Kelly’s view, Chaucer was looking for a way to celebrate King Richard II’s betrothal to Anne of Bohemia on that day and found that was the feast day for Valentine of Genoa. (He could have chosen the Feast of the Holy Cross, but that wouldn’t have sounded as nice in the poem.) But, since his contemporaries were more familiar with the Feb. 14 Saint Valentine’s Day, that was the date that became attached to the new holiday of romance.

In some ways, that may be a good thing.

“February is the worst month in cold climates,” Kelly says. “It’s great to have something to look forward to.”