James Marshall didn’t come to California to find gold. But then he noticed a glinting rock in the dirt while constructing a new mill for local landowner John Sutter. It was 1848, and Marshall’s fate—and that of California—had just changed forever.

The Gold Rush that followed changed the lives of California’s Native Americans, too. Within years, they would be almost wiped out due to the massive immigration—and hunger for wealth—the Gold Rush inspired.

The Anglo settlers who flocked to California declared war on the Native Californians who had come before them. But Forty-Niners weren’t the first white people to oppress or even enslave Native Americans in California. The very land on which Marshall spotted the gold was part of a vast empire built on the slave labor of Indigenous peoples.

Without Native Americans, John Sutter—owner of the mill where gold was discovered and the area’s most influential landowners—would never have become so powerful. Sutter, a shrewd businessman, enslaved hundreds of Native Americans and used them as a free source of labor and a makeshift militia with which he defended his territory. He also helped set the stage for their genocide.

Before John Sutter became a land baron, he was Johann Suter, a debt-ridden shop owner in Switzerland. Rather than serve jail time for his debts, the 31-year-old left his home country—and his wife and five children—behind.

At the time, California was a Mexican province, and Sutter was tempted by its vast natural resources and its seemingly sparse population. Accompanied by a group of Native Americans he had “acquired” along with provisions and tools, he convinced the provincial governor to grant him 50,000 acres for a settlement and trade center he dubbed “Nueva Helvetia,” or New Switzerland, in 1841.

Sutter became Nueva Helvetia’s judge and military commander, with the authority to prevent what he characterized as “the robberies committed by adventurers from the United States” and “the invasion of savage Indians.” In order to acquire the land, he converted to Catholicism and became a Mexican citizen, and within a few years he had more than doubled his land holdings.

The land Sutter controlled may have been sparsely populated to Anglo settlers, but it was home to Native Americans who “found their homelands now the property of outsiders who viewed them as potential laborers,” writes historian Lisa Emmerich. Those native peoples presented both a threat and an opportunity to Sutter.

Sutter initially forged an amicable relationship with local Nisenan people, and turned them into a militia, outfitting them with uniforms and weapons and training them to defend his land.

Though Sutter’s land grant required that he treat Native Americans in a friendly way, he began to interfere with local tribes, disrupting local marriage customs and creating what one observer called a “harem.” Heinrich Lienhard, one of Sutter’s Swiss employees, recalled that Sutter had a room adjoining his office in which “a group of Indian women were invariably waiting.” Lienhard also accused Sutter of molesting Native American girls.

Sexual coercion was not the only way in which Sutter exerted his control of Native Americans. With the help of his militia, he also enslaved them. “Those who did not want to work were considered enemies,” a nearby rancher recalled. “Often the Sacramento River was colored red by the blood of the innocent Indians.”

Sutter told his overseer to keep his servants in line “strictly under fear” and did not hesitate to kill Native Americans who did not submit to hard labor on his ranch. “Sutter keeps 600 to 800 Indians in a complete state of Slavery,” wrote a visiting settler, James Clyman, when he visited Sutter’s ranch.”

Edwin Bryant, a newspaper editor from Kentucky who was hosted by Sutter on an expedition to California,described how the native workers were fed offal and leftover wheat bran from wooden troughs, eating their meals without utensils or bowls. Meanwhile, he was served a bountiful meal on china plates. The slaves slept in locked rooms without beds or furniture and were whipped and sometimes murdered when they refused to comply with his wishes.

Gregarious and welcome to white settlers, Sutter invited many early pioneers to his ranch, where they saw his treatment of Native Americans. His visitors, writes historian Benjamin Madley, were deeply affected by the treatment of workers they witnessed on Sutter’s land. “These encounters had a powerful psychic effect,” writes Madley, “fueling racism and emotionally hardening colonists…to cruelty toward California Indians.”

For Sutter, Native Americans weren’t just an economic powerhouse—they were currency. He traded native labor among local rancheros and to new settlers, shipping large groups of Native Californians to different employers and receiving as much as two dollars a day for their services. Sutter’s notorious hospitality to white settlers—a warm welcome that was in direct conflict with his promises to the Mexican government—was markedly different from the way he treated the Native Americans who upheld his growing wealth.

Not all of John Sutter’s workforce consisted of slaves. Though some workers were enslaved, others were “paid” in tin currency that could only be used at his store. Others—often chiefs whose support Sutter needed—were paid for their work.

Eventually, a measles epidemic wiped out a large portion of the Native American laborers on Sutter’s ranch, and he decided to build a sawmill on some nearby property to make up for the loss of work.

What happened next is well known: Sutter’s Mill became ground zero for the Gold Rush of 1849. But even the discovery of gold was facilitated by Sutter’s enslavement and coercion of native peoples—indeed, Marshall was led to the site where he noticed the gold nuggets by a Native American guide and the dirt there was dug by a group of Sutter-controlled Native Americans who knew about the gold, but did not value it.

The story didn’t end well for either Sutter or Marshall. After the presence of gold became known, squatters and thieves overran Sutter’s ranch, destroyed his building, looted his wealth and stole his livestock. His Native American workers deserted him and, as the new state of California assessed the legality of Mexican-era land grants, his claim to the lands granted to him in 1841 was declared invalid.

Impoverished and saddled with debt, Sutter petitioned the United States government for restitution until his death in 1880. Marshall didn’t fare much better: He went bankrupt and died in poverty after an unsuccessful career as a gold miner.

But perhaps the biggest losers were the Native Americans of Gold Rush-era California. In the 20 years that followed the discovery of gold, 80 percent of the state’s Native American population was wiped out—victims of displacement, disease and a genocide wrought in the sake of power and gold. John Sutter had set the stage for their destruction—but his cruelty was just the beginning.