When British writer William Crackanthorpe visited the Mediterranean island of Elba in 1814, he was wildly curious about its most famous resident: the disgraced emperor Napoleon Bonaparte. Months earlier, Napoleon had been exiled to Elba in one of history’s greatest humiliations—and Crackanthorpe wanted to know how the disgraced emperor was spending his time.

He was received with the emperor’s usual flair. But during his visit, the writer noticed something odd about Napoleon. “At intervals… he seemed to relapse into a kind of reverie,” he wrote, “when his countenance assumed that fiendish appearance … I doubt not that he breathed vengeance within himself against us for having come to see him in his humility.”

He was right. Napoleon may have appeared subdued, but in his mind he was planning one of history’s greatest prison escapes. Within months, he’d make a run for it—and try to avenge himself against those who had forced him into exile.

Despite Napoleon’s bitterness about his life on Elba, his time on the large island off the coast of Tuscany was largely the result of his own negotiations. After his defeat and dethroning in 1814, Napoleon came to an agreement with the coalition of nations that had taken him down. In some ways, the terms of the Treaty of Fontainebleau, which he signed in April 1814, were harsh.

Napoleon had to give up his royal property, along with his right to rule and that of all of his current and future family members. However, he was able to keep the title of Emperor and even choose his own island nation—Elba—to rule.

Technically, Elba was part of France, but the treaty turned it into a principality, which, the treaty stipulated, was to be “possessed by him in all sovereignty and property.” True, Elba only had 12,000 residents, but Napoleon was entirely in charge of the 86-square-mile island. And though he claimed that he wanted to live on the island as a mere justice of the peace, thinking only of “my family, my little house, my cows and mules,” he had bigger plans.

Elba meant exile for Napoleon, but it was no prison. Napoleon specifically chose it because it had good weather and defenses, and he took up residence in a villa with harbor views built by the Medicis in the 1700s. He had another summer residence, too. Both buildings were outfitted with lavish furnishings and designed for parties and visitors, including his official babysitter, a British officer named Neil Campbell who watched his comings and goings.

Though Napoleon’s second wife, Marie Louise, didn’t join him on the island, his mistress, the Polish countess Marie Walewska, did. Her brief visit was purposely shrouded in secrecy, but islanders soon learned of it. They eagerly followed the movements and excesses of the miniature court Napoleon built on the island. Soon, however, they realized they were expected to pay for those expenses through taxes, and became more suspicious of the exiled emperor.

Campbell followed Napoleon’s growing lack of interest in the fate of the islanders and soon learned that Napoleon feared insolvency, especially when the money promised to him under the treaty did not materialize. But he didn’t realize the Emperor of Elba had begun to make plans to leave the island.

Napoleon claimed he was a “dead man” and that his time of greatness had passed. But in reality, he was biding his time. Ruling Elba gave him an excuse to build a military force: an army of 2,000, a 600-man Imperial Guard, and a small navy. Napoleon’s frequent communications with France and his continual stream of visitors concerned the British. But until February 26, 1815, they didn’t realize how dangerous those communications had been.

Through these visits, Napoleon learned that the British had begun to formulate plans to move him further away from France to St. Helena, an island in the South Atlantic. He also heard that, back in France, his supporters had begun to foment rebellion against the new king, Louis XVIII.

Characteristically, the power-hungry Napoleon had begun to worry that he was going to die in obscurity. Technically, he reasoned, he wasn’t required to stay on Elba, as he felt the terms of the treaty had been broken. Besides, he was needed in France. He consulted with his mother, who stayed with him on the island. “Go, my son!” she reportedly said. “Fulfill your destiny!”

He didn’t need much encouragement. When Campbell headed to England with a note saying Napoleon was becoming restless, the emperor saw his chance. He put together a small fleet of ships, including the brig Inconstant, which he painted like a British vessel and filled with an army of loyalists.

On February 26, the flotilla left the island with about 1,150 people aboard. He had given Elba and the English the slip without really bothering to hide his intentions or his preparations. Napoleon even met with officials in Elba to tell them he was leaving. Soon, he was back in France.

“A thousand ideas and projects are formed; resistance is nowhere decided,” he told an associate. “I shall arrive before any plan has been organized against me.”

The bold prison break worked: The French were surprised, the English ineffective, and Napoleon’s supporters ecstatic. He arrived in Paris as a hero. And though his second reign only lasted 100 days, it would never have been possible without the egotism and brashness of a man who simply decided he’d take matters into his own hands.