Days after resigning from the U.S. Army on April 20, 1861, to take command of Virginian forces in the Civil War, Robert E. Lee left the Arlington estate where he had married Mary Lee and lived for 30 years. He would never return.

After Virginia seceded from the Union on May 23, 1861, Union troops crossed the Potomac River from the national capital and occupied the 200-acre property and house that had been built by George Washington Parke Custis, Mary’s father and the step-grandson of George Washington. After Mary Lee, confined to a wheelchair, sent a representative instead of appearing personally to pay a $92.07 tax bill, the government seized the property in 1864. With Washington, D.C., teeming with dead soldiers and out of burial space, Quartermaster General Montgomery C. Meigs formally proposed Arlington as the location of a new military cemetery.

On May 13, 1864, 21-year-old Private William Christman of Pennsylvania, who had died of peritonitis, became the first military man buried at Arlington. To ensure the house would forever be uninhabitable for the Lees, Meigs directed graves to be placed as close to the mansion as possible, and in 1866 he ordered the remains of 2,111 unknown Civil War soldiers killed on battlefields near Washington, D.C., to be placed inside a vault in the Lees’ rose garden.

More than a decade after Lee’s death, the Supreme Court ruled that the U.S. government had seized his estate without due process and ordered it returned to his family in the same condition as when it was illegally confiscated. If followed, the ruling could have required the exhumation of all of Arlington’s dead, but instead, Lee’s son officially sold the property to Congress for $150,000 in 1883.

In 1868, General John A. Logan, commander-in-chief of the Grand Army of the Republic, officially proclaimed May 30 as Decoration Day “for the purpose of strewing with flowers or otherwise decorating the graves of comrades who died in defense of their country during the late rebellion.” The first national commemoration took place at Arlington on May 30, 1868, with General Ulysses S. Grant in attendance and General James Garfield as the featured speaker. Decoration Day was eventually renamed Memorial Day.

Although the first military burial at Arlington National Cemetery didn’t occur until 1864, the burial ground holds the remains of those who fought in every war since the American Revolution. In 1892, soldiers killed in the Revolutionary War were reinterred from a Georgetown cemetery, and casualties from the War of 1812 have been reburied at Arlington as well.

Scattered among America’s honored dead are two Italian prisoners of war and one German prisoner of war captured during World War II. The three enemy combatants died in captivity in the Washington, D.C., area, and the Geneva Conventions required proper burials. With Arlington being the closest national cemetery, the men were buried there. In total, there are approximately 60 foreign nationals interred in Arlington, most being allied servicemen who perished in air disasters that included American soldiers.

After seizing Lee’s estate, the federal government set aside acreage to be a model community for emancipated, freed and fugitive slaves. Freedman’s Village included farmland, homes, a hospital, a school and a mess hall before shuttering in 1900. African-Americans who lived at the village were buried on the property, and their graves were incorporated into Section 27 of Arlington National Cemetery. Their headstones are inscribed with “citizen” or “civilian.”

Among the 5,000 unknowns entombed at Arlington are the unidentified remains of World War I, World War II and Korean War servicemen buried at the Tomb of the Unknown Soldier. An unknown from the Vietnam War was entombed in 1984, but the remains were exhumed in 1998, positively identified by DNA testing as those of Air Force First Lieutenant Michael Joseph Blassie and reinterred at a military cemetery in Missouri. Advancements in DNA testing may mean that all future remains will be able to be positively identified and no future interments at the Tomb of the Unknown Soldier will occur.

On every Memorial Day weekend since 1948, troops in the 3rd U.S. Infantry Regiment—the Army’s official ceremonial unit known as the “Old Guard”—have placed small American flags in front of all of Arlington’s U.S. tombstones. Each flag is planted precisely one foot in front of a grave marker and perfectly centered.