The nation’s most prominent military burial ground—Arlington National Cemetery, which officially opened on June 15, 1864—began with the seizure of a prominent Army officer’s hilltop home after he defected to the Confederacy during the Civil War.

General Robert E. Lee, a native Virginian who reportedly spent the night nervously pacing upstairs in his home, Arlington Estate, as he deliberated whether to lead the Union Army or fight for his home state’s Confederacy, resigned from the U.S. Army on April 20, 1861. He left for Richmond, Virginia, the next day, and told his wife, Mary Anna Randolph Custis Lee, the great-granddaughter of original First Lady Martha Washington, to vacate their house. With the high hilltop position overlooking Washington, D.C., Lee knew the Union forces were likely to seize the property, which was in a mostly rural area at the time.

“It wasn’t uncommon for Army headquarters to be placed in residences,” says George W. Dodge, author of the Arlington National Cemetery book in Arcadia Publishing’s Images of America series. “The Arlington house was high ground: If Confederates got it, they could bombard Washington, D.C.”

Although she resisted for a few weeks, Mary Custis Lee accepted what seemed the inevitable Union takeover of the Greek Revival home she had inherited from her father and fled to a nearby family house. She left the 1,100-acre property and its many heirlooms in the care of an enslaved woman named Selina Gray. Then, on May 24, Union troops moved into Arlington and occupied the grand house.

The Lees would never return to live at Arlington Estate, which was completed in 1818. But federal troops, under orders from Georgia-born Brigadier General Montgomery Meigs, made the property difficult to return to by burying the bodies of Union soldiers near the house. A furious Meigs, who detested Lee once his former friend turned on the United States for the Confederacy, said he was “grimly satisfied” as the tombstones started to fill the hill surrounding the house.

Meigs went a step further to plant a painful reminder to the Lees, if they ever returned to Arlington, about the carnage the Civil War caused. After the war, an estimated 2,111 unknown bodies were exhumed from battlefields and reburied in a mass grave deliberately placed in Mrs. Lee’s rose garden. This Tomb of the Civil War Unknowns, dedicated in September 1866, was intended to further mark the Lee property with the grim reality of war, Dodge says.

“I think a fair inference from the action he took was that he wanted to make the property uninhabitable in case of the remote chance the Southern states successfully seceded,” says Dodge, an Arlington, Virginia-based lawyer who does estate work. “The inference was to punish Lee.”

Cemeteries near Washington, D.C. started filling up from soldiers dying on battlefields and in hospitals in Union-controlled Alexandria, Virginia. So Meigs seized the opportunity to turn the Lee property into much-needed cemetery space to accommodate the mounting casualties.

The first body to be buried was William Henry Christman on May 13, 1864. The native of Lehigh County, Pennsylvania died of measles and had only served in the Army about 60 days. About a month later, on June 15, Secretary of War Edwin Stanton ordered that the new cemetery become Arlington National Cemetery, which at the time was about 200 acres. Within a year, more than 5,000 soldiers, mainly privates, were buried there. Today, the cemetery is 639 acres, where about 400,000 veterans and their eligible dependents are interred.

Dodge says that Meigs probably wouldn’t be surprised at what Arlington National Cemetery became, even though it initially was referred to as a potter’s field—a burial ground for poor soldiers whose families couldn’t afford to bring them home.

“It wasn’t really the cemetery held in high regard … but in the 1890s, you had Union officers that wanted to be interred there,” Dodge says.

Meigs himself was buried in Arlington National Cemetery behind the house when he died in January of 1892, and Meigs lived to see the cemetery start to gain prestige, Dodge says. Meigs’ death came a few years after General Philip Sheridan was interred near the front of the Arlington house upon his death in August of 1888. Sheridan was the highest-ranking general at the time to be buried at Arlington.

“His interment changes the status of the cemetery,” Dodge says about Sheridan. “Meigs is very instrumental in getting it on the path to a renowned cemetery.”

Arlington National Cemetery’s panoramic view of the nation’s capital and beautiful environment, along with its history, made it a prime place of burial a few generations after its founding.