The second wife of Henry VIII, Anne Boleyn was twice a resident of the Tower of London—once as a queen-in-waiting and once as a condemned prisoner. Boleyn married Henry in 1533 after the English king defied the Roman Catholic Church and annulled his marriage to Catherine of Aragon. Housed in the Tower of London prior to her coronation in June 1533, Boleyn would reign as queen of England for nearly three years.

Coupled with courtly intrigue and accusations of infidelity, Boleyn’s failure to give birth to a male heir ultimately proved to be her undoing. Accused of seducing the king into a cursed marriage, in May 1536 she was arrested on trumped-up charges of adultery, treason and even an alleged incestuous affair with her brother. Boleyn was confined to the Lieutenant’s Lodgings of the Tower of London, where she was tried and found guilty. She was beheaded by a French swordsman on a scaffold at the Tower on May 19, 1536. Catherine Howard, Henry’s fifth wife, would meet a similar fate when she was imprisoned and then executed at the Tower of London in 1542.

One of the longest-serving prisoners of the Tower of London was the famed Sir Walter Raleigh, who was confined to the citadel for some 13 years. A soldier and explorer who engineered the ill-fated English colony at Roanoke Island, Raleigh was knighted by Elizabeth I in 1585 and became one of the queen’s favorite courtiers. Despite his influential position, Raleigh was briefly imprisoned in the Tower in 1592 when it was revealed that he had secretly wed Elizabeth Throckmorton, one of the queen’s maids of honor.

Raleigh was confined to the Tower a second time in 1603 after he was accused of plotting against King James I. Stripped of most of his wealth, he would spend nearly 13 years detained in a part of the castle known as the Bloody Tower. While he was ostensibly a prisoner, Raleigh’s high social standing ensured that he had comfortable lodgings, and he was even joined in the Tower by his family. During this time he devoted himself to science and writing—composing his “History of the World” in 1614—and also fathered a son. Raleigh was released in 1616 and dispatched to Central America in search of the mythical gold city of El Dorado. The mission proved unsuccessful, and Raleigh was arrested and executed at the block after his forces attacked a Spanish outpost against the orders of the king.

Twelve-year-old Prince Edward V and 10-year-old Prince Richard of Shrewsbury—better known as “the Princes in the Tower”—are among the most famous prisoners to have disappeared within the bowels of the Tower of London. The two boys first arrived at the castle in 1483 after the death of their father, King Edward IV. The princes were originally housed in the Tower on the orders of their uncle Richard, Duke of Gloucester, but were stripped of their royal titles after the duke invalidated their father’s marriage, declared them illegitimate and claimed the throne for himself as King Richard III. Moved from their opulent royal apartments to the confines of the Garden Tower (later known as the Bloody Tower), the boys effectively became prisoners of the crown.

While there were initially sightings of the former princes playing in the Tower courtyard, by mid-1483 they had vanished without a trace. The would-be monarchs’ true fate remains a mystery. While a Flemish man claiming to be Prince Richard would invade England in 1497 and attempt to take the throne, he was later revealed to be a pretender and was executed at the Tower of London. The only clue would come in 1674 when the skeletons of two children were found during renovations to the Tower. While these were never proven to be the bodies of the Princes in the Tower, the discovery fueled speculation that the boys had been murdered, with their uncle Richard III the most likely culprit.

Guy Fawkes was a soldier and revolutionary who was imprisoned in the Tower of London for his role in the notorious Gunpowder Plot. A militant Catholic, in 1604 Fawkes became embroiled in an audacious plan to assassinate the Protestant King James I and other members of the British government by blowing up the House of Lords. After renting the storage room beneath Westminster Palace, Fawkes and his accomplices packed the cellar with 35 barrels of gunpowder, which they planned to detonate on November 5, 1605, during the opening of Parliament.

The plot was foiled after an anonymous letter led authorities to search the cellar, and Fawkes was captured when he was found guarding the door. He was then sent to the Tower of London and confined to the infamous cell known as the “Little Ease,” a cramped room that prevented its occupant from either standing up straight or lying down. Following intense interrogation and torture—most likely on the rack—Fawkes exposed the other men involved in the plot. Found guilty of treason, he was condemned to be hanged, drawn and quartered at the palace yard at Westminster, but he avoided this gruesome punishment by throwing himself from the gallows and breaking his own neck. The Gunpowder Plot would later become the inspiration for Guy Fawkes Day, a British holiday celebrated every November 5.

Lady Jane Grey’s meteoric rise and fall saw her go from ruler of England to a prisoner in the Tower of London in little more than a week. The teenage Grey was installed as queen of England in July 1553 after her cousin Edward VI died without a male heir. Desperate to thwart his Catholic half-sister Mary’s claim to the throne, Edward VI had chosen Grey as his successor in order to ensure the crown remained under Protestant control.

The new queen’s reign would last only nine days. After Mary raised a large band of supporters, the royal government abruptly switched their allegiance and declared her the rightful queen of England. Forced to relinquish the crown, Grey was taken prisoner and moved from the Tower’s royal apartments to the Gentleman Gaoler’s lodgings. While she was kept under constant guard, she was allowed occasional walks around the castle grounds and even given a weekly allowance. Grey was tried and found guilty of treason in November 1553 but was quickly pardoned by Queen Mary I. A final twist of fate came in January 1554, when Grey’s father’s participation in a Protestant rebellion led the royal government to proceed with its death sentence. Grey was then beheaded on the Tower Green on February 12, 1554.

By the 20th century, the Tower of London had primarily become a tourist site and storage facility for the Crown Jewels. But during World War II the castle was briefly restored to its role as a state prison when it held two high-profile Nazis captured on British soil. One of these men was Josef Jakobs, a German spy apprehended in rural England. Jakobs became the last man put to death at the Tower of London when he was executed by a firing squad in August 1941.

Even more famous was “Deputy to the Führer” Rudolf Hess, who served as Hitler’s second-in-command in the Nazi Party. Hess was captured in May 1941 after he parachuted into Scotland as part of a renegade plan to negotiate peace with the British. Doubtful of Hess’s motives, Prime Minister Winston Churchill had him sent to the Tower, making him the final state prisoner to be held at the castle. Hess would only remain for a few days, but rumors that he was hidden away in the Tower would persist for several months. Hess was later tried at Nuremberg and given a life sentence. He died at Spandau Prison, West Berlin, in 1987.