The 2018 PyeongChang Winter Olympics have been the first Games in 34 years that Russia has not, technically, attended. After a years-long doping scandal, the International Olympic Committee stripped Russia of 41 medals and formally banned them from this year’s games, leaving their athletes to compete under the clunky banner of “Olympic Athletes from Russia.”

But 34 years ago the then-Soviet Union had a perhaps even more dramatic reason for not coming to an Olympic Games: they believed that American radicals were going to kidnap all of their athletes.

In 1984, a ragtag group of right-wing businessmen, advertising executives, and Soviet bloc immigrants hatched a plot in the sprawling suburbs of Southern California. The Los Angeles Olympics, just weeks away, had been designed to extol the merits of the free market: they’d be privately funded, run by businessmen, and if all went according to plan, would end with an unprecedented surplus of cash.

This grassroots group, known as the Ban the Soviets Coalition, had a different goal: keeping the Soviet Union athletes out of Los Angeles at all costs.

The Ban the Soviets Coalition recognized they might not be able to completely thwart the Soviet team. The Soviet Union had sunk billion of rubles into their athletic programs, viewing success on the Olympic stage as a validation of the communist system. So the coalition also had a Plan B: if the Soviets showed up they would attempt to trigger a mass defection, encouraging all the Soviet athletes to claim asylum in the United States.

Russian-language billboards would line the Los Angeles highways, offering instructions on how to claim asylum. “This is the Land of Liberty and This is a Telephone Number You Can Call,” read one proposed street sign. Safe houses would be established throughout Los Angeles, where fleeing athletes could find a place to stay and receive legal support. The Coalition claimed that its operatives had already begun to infiltrate the Los Angeles Olympic Organizing Committee and were well placed to assist defectors.

It was a radical plan, and a plan that few believed they could actually carry out. But it hit on a legitimate fear during the Cold War: defection of athletes. Athletes had been using the Olympics as a pathway to claim asylum for decades, with as many as 117 athletes defecting at the 1972 Munich Olympics. At the 1976 Olympics, a 17-year old Soviet diver named Sergei Nemtsanov attempted to defect to Canada. The Soviet delegation insisted the youth had been “brainwashed” and threatened to withdraw the Soviet team from the Olympics unless he was returned by Canadian authorities (two weeks later, Nemtsanov elected to return home). The entire episode was an enormous embarrassment for the Soviet team, and not one they were eager to repeat.

The Ban the Soviets Coalition was born out of tragedy and geopolitics. On September 1, 1983, Soviet jet fighters shot down a Korean Airlines passenger flight in Russian airspace, killing 269 passengers. Within days, Reagan had suspended all Soviet airline flights into the U.S. and abandoned other bilateral agreements. Yet for some Americans, including those who who comprised Reagan’s white, conservative, suburbanite base, these actions were insufficient, “a slap on the wrist.” Keeping the Soviets out of the Olympics, they argued, would be an important symbolic gesture. If they could not be a responsible part of the international community, why should their presence be welcomed at a sporting event premised on the idea of peaceful global exchange?

More than 100 groups rallied behind the Ban the Soviets banner. Some were Republican political groups, like the Conservative Caucus and the Young Americans for Freedom, while others were associated with Soviet bloc ethnic groups, like the Elderly Korean American Association of Orange County and the Baltic American Freedom League.

It didn’t take long for the Soviet media to catch wind of the Ban the Soviet Coalition’s plans.

“People will be seized and whisked away to clandestine hide-outs,” wrote the Literary Gazette, a Kremlin weekly. “And there all conceivable methods will be used to extort from them betrayal of their motherland. They will be wrapped in the Stars and Stripes–all in the light of the Olympic flame.”

The head of the Soviet Olympic Committee accused U.S. government officials of working alongside the Ban the Soviets Coalition to ”terrorize” and ”kidnap” Soviet athletes. Further, he claimed there were plans to use ”psychotropic” drugs to secure the defections of the Soviet athletes. Another Soviet newspaper suggested that the dissenters be kept in “some remote and quiet place” for the duration of the games—a very Soviet solution.

For Peter Ueberroth, the chairman of the Los Angeles Olympic Organizing Committee, this was a worst case scenario: a diplomatic guerilla war waged by a private citizens’ group, threatening to destabilize the games just weeks before the Opening Ceremonies. Ueberroth dismissed the coalition as “a nutty group,” and assured everyone involved that the Ban the Soviets Coalition were fringe extremists, not a legitimate threat to athletes. Their plans were almost certainly implausible given their $50,000 budget, Ueberroth insisted.

“I called them nutty at one time.” Ueberroth said to Sports Illustrated. “I said I would change and apologize if they’d change their name to a proper one. Maybe call themselves the Coalition to Hurt Athletes or maybe the Coalition to Play into the Hands of the Soviets.”

Still, the Soviets demanded that President Reagan formally condemn the actions of the Coalition, a political maneuver the president was highly unlikely to make. The Ban the Soviets Coalition was the same sort of grassroots, Southern California group that had propelled Reagan to both gubernatorial and presidential victories, made up of white, Evangelical, business-savvy voters with extreme anti-communist beliefs, a demographic historian Lisa McGirr called “suburban warriors.” To condemn the Coalition would snub his base, as well as legitimize the threat of a group the Olympics coordinators insisted was a fringe group of nobodies.

On May 8, 1984, the will-they-or-won’t-they tension ended with an announcement from the Soviets: they would not be coming to Los Angeles. Most of the Soviet bloc countries soon followed suit. The Soviet Union carefully labeled their choice as “nonparticipation,” distinguishing it from a boycott and insisting that their decision was intended to protect, rather than to politicize, the Games.

In its official statement, the Soviet Union said it was pulling out of the Games because the United States could not guarantee the safety of Russian athletes. The United States authorities were guilty of ”connivance” with ”extremist organizations” who were trying to create ”unbearable conditions” for Soviet athletes and coaches, the Soviets said. It was a clear gesture towards the activities of the Ban the Soviets Coalition.

On the day of the “nonparticipation” announcement, four of Moscow’s newspapers carried letters from readers, in “a carefully orchestrated campaign to show public support for a boycott.” One letter spoke of the ”terror” that athletes would face in Los Angeles. All agreed that the Soviet team would not be safe from American radicals and must stay away.

But to what extent did the actions of the Ban the Soviets Coalition actually influence the Soviet’s decision to withdraw from the Olympics? In the minds of the Coalition’s leaders, their plot was the deciding factor. “When we began, everyone said we could not get the Soviet Union out,” their spokesperson boasted. “But we did it against great odds. We were responsible for them dropping out.”

“I’m sure they decided to pull out because the U.S. would not muzzle our coalition and agree to turn defectors back over to the KGB, which was probably the major reason they withdrew,” he added.

However, the activities of the Coalition were far from the only issue on which Soviets and the LAOOC clashed. The Soviet Union wanted its athletes to stay aboard a ship they would dock in the Los Angeles Harbor, rather than in the Olympic Village—an idea the U.S. was predictably less-than-thrilled about. The Soviets also expressed concerns about the lack of foreign judges, fearing their athletes wouldn’t receive objective scoring.

Most commentators at the time saw the Soviet’s withdrawal as simply an act of revenge—payback for the embarrassment of the U.S. withdrawal from the Moscow games four years prior. The New York Times’ Moscow correspondent reported, “The Soviet withdrawal from the Los Angeles Olympics is nothing more than paying America back in kind for its boycott of the 1980 Games in Moscow… The American boycott, as intended, was a devastating blow to Soviet pride.”

The “revenge thesis” is less believable to some historians. Robert Edelman, Professor Emeritus at the University of California-San Diego, has argued that the local and federal dismissal of Soviet concerns about radical groups was central to the withdrawal, rather than mere posturing. Archives opened after the collapse of the USSR revealed that the Soviets expended great resources to prepare their athletes for the 1984 Olympic Games in Los Angeles. “Ultimately, it was fear of this less than fearsome group and State Department resistance that led the Politburo to keep their athletes home,” Edelman writes.

In the months that followed the “nonparticipation” decision, the Ban the Soviets Coalition quickly crumbled; they’d accomplished their goal, and most members moved on to other anti-communist causes. Despite the near-miss with a defection crisis, the Los Angeles Olympics were remembered as perhaps the most successful in history, ending with a stunning $232.5 million surplus and Peter Ueberroth as Time magazine’s “Man of the Year.”

But the Games also solidified what most observers already knew to be true: in the modern era, there was no way to fully separate the microcosm of the Olympics from the conflicts, tensions, and crises of the world at large.