Between 1906 and 1916, Eleanor and Franklin D. Roosevelt had six children, one of whom died in infancy. Nannies helped rear the children as politics and polio treatments drew Franklin away from the family for long stretches of time and as Eleanor juggled a heavy travel schedule and engagements related to her activism. “We never had the day-to-day discipline, supervision and attention most children get from their parents,” recalled son James.

Roosevelt acknowledged the burden the presidency placed on his offspring, who were in their teens and twenties when he took office. “One of the worst things in the world is being the child of a president,” he told an aide. “It’s a terrible life they lead.” The glare of the public spotlight took a toll on the private lives of the five surviving Roosevelt children, who combined for 19 marriages.

FDR and Eleanor gave their eldest child—and only daughter—the same birth name as her mother. At age 20, Anna wed a Wall Street broker 10 years her senior partly to escape the tensions between Eleanor and her husband and her domineering mother-in-law. “I got married when I did because I wanted to get out,” she said.

That marriage ended after Anna fell in love with newspaper reporter John Boettiger while campaigning for her father in 1932. Prior to wedding Boettiger in 1935, Anna and her two children lived in the White House, and she returned there in 1944 to assist her father as a hostess and secretary. Alarmed at her father’s declining health, Anna insisted the president’s physician consult a cardiologist, who diagnosed Roosevelt with congestive heart failure.

Anna accompanied her father to the Yalta Conference in February 1945 to monitor his schedule and ensure he followed doctor’s orders. Unwilling to upset her ailing father, she also facilitated secret meetings with his long-time mistress, Lucy Mercer, who was at Roosevelt’s side in Warm Springs, Georgia, when he died on April 12, 1945. Her relationship with Eleanor cooled when her mother learned Anna arranged Mercer’s clandestine visits, but the pair later co-hosted a radio discussion show. Anna died in 1975.

Named after his paternal grandfather, James Roosevelt followed the family’s well-trodden path to the Groton School and Harvard University. During the 1932 presidential campaign, 24-year-old “Jimmy” often appeared at his father’s side for support—literally. Unable to walk under his own power, Roosevelt would grasp his son’s arm for balance and take painstaking steps by shuffling his paralyzed legs clamped in heavy metal braces.

Jimmy took a paid White House position as a secretary in 1937 but left the following year after suffering severe ulcers and facing accusations that he cashed in on the family name to earn as much as $1 million a year in a previous job as an insurance agent.

During World War II, Jimmy served in the Pacific Theater as a lieutenant colonel with the Marines. After requesting combat duty, he commanded a Marine battalion in the Gilbert Islands and received the Navy Cross for saving three men from drowning. Following in his father’s political footsteps, he lost the 1950 race for California governor to incumbent Earl Warren before serving in the U.S. House of Representatives between 1955 and 1965. Married four times, Jimmy survived a 1969 stabbing by his third wife and died in 1991 as the last surviving Roosevelt child.

Named for Eleanor’s father—and Theodore Roosevelt’s brother—Elliott Roosevelt was the Roosevelts’ most rebellious child. He skipped college for high-paying media jobs and often attacked his father’s policies as a newspaper columnist.

Bucking the family’s naval tradition, the aviation buff joined the U.S. Army Air Corps. While Republicans alleged nepotism when he was commissioned as a captain during the 1940 presidential campaign, Elliott distinguished himself in wartime by piloting unarmed reconnaissance planes on 300 combat missions and earning the Distinguished Flying Cross and Legion of Merit. He commanded an aerial mapping unit that played a key role in the invasions of North Africa, Sicily and Normandy. And he accompanied his father to the Atlantic Charter and Casablanca summits with British Prime Minister Winston Churchill and the Big Three conference in Tehran. Elliott wrote his eyewitness accounts of the meetings in the 1946 bestseller As He Saw It.

Elliott’s eclectic post-war career included breeding Arabian horses, serving as mayor of Miami Beach and writing a series of mystery novels starring his mother as an amateur detective. His 1973 book, An Untold Story, revealed the intimate relationship between his father and private secretary Missy LeHand and caused a rift with his siblings, who publicly disavowed the book. Married five times, Elliott died in 1990.

Franklin and Eleanor’s third child—Franklin Roosevelt, Jr.—suffered from a heart condition and died in 1909 at the age of seven months. As author Joshua Kendall writes in First Dads, “The hypomanic, chronically upbeat FDR would essentially erase this infant from the family’s history” by giving the same name to his fifth child, born in 1914.

The name was prescient. “Franklin is the one who came closest to being another FDR. He has father’s looks, his speaking voice, his smile, his charm, his charisma,” said his brother James. After graduating from Harvard and the University of Virginia Law School, FDR, Jr. joined the U.S. Navy Reserve and was called to active duty in 1941. He earned a Purple Heart and a Silver Star for carrying an injured sailor to safety under fire when his destroyer was badly damaged in the invasion of Sicily. Young Franklin also commanded the destroyer escort USS Ulvert M. Moorein the Pacific and accompanied his father to the Atlantic Charter summit and Casablanca Conference.

After the war, Frank practiced law and represented Manhattan’s Upper West Side as a three-term congressman between 1949 and 1955. Unlike his father, FDR, Jr. lost his bid to win election as New York governor in 1966. He married five times and died in 1988.

After graduating from Harvard, the youngest Roosevelt child worked briefly as a retail clerk before serving in the U.S. Navy during World War II. After his father denied his application for sea duty in 1942, John wrote, “I don’t care what the ship looks like or is, as long as she at least floats for a while.” Eventually assigned to the Pacific, he served as a lieutenant commander aboard the USS Wasp and earned a Bronze Star.

After the war, John largely avoided the spotlight. He owned and operated a Los Angeles department store and later worked as an investment banker and fundraiser for the National Foundation for Infantile Paralysis, which his father had founded. John never sought political office but broke with his staunchly Democratic family in joining the Republican Party. Twice married, he died in 1981 at the age of 65.