When Anne Sullivan came to Tewksbury, she wasn’t yet the renowned “miracle worker” who would teach Helen Keller to communicate. It was 1876, and 10-year-old Annie was a blind child living in abject poverty. Her years at the poorhouse—a facility designed to house poor people in a time before social services— were “a crime against childhood,” she later remembered.

Residents at the Massachusetts poorhouse milled about like forgotten animals. As Anne and her brother slept on the institution’s iron cots in a gigantic dormitory, rats ran up and down the spaces between beds.

In 1883, a massive investigation exposed the conditions at Tewksbury—but the institution was far from unique. Throughout the 19th and early 20th centuries, poorhouses were a reality for society’s most vulnerable people. These locally run institutions filled a need in a time before Social Security, Medicaid and Section 8 housing became a reality. They also exposed the stigma and shame society placed on those who were unable to support themselves.

The concept of the poorhouse originated in England during the 17th century. Municipalities were expected to care for their poor, and made a distinction between people who were old and unable to care for themselves and the able-bodied. People who were able to work were expected to do so—and could be imprisoned if they refused.

They lived in workhouses, bare bones facilities designed to make poverty seem even less attractive. In these facilities, poor people ate thrifty, unpalatable food, slept in crowded, often unsanitary conditions, and were put to work breaking stones, crushing bones, spinning cloth or doing domestic labor, among other jobs.

In the United States, the idea emigrated along with English colonists. In 1660, Boston built its first workhouse—a brick building intended for “dissolute and vagrant persons.” Massachusetts’ poor people had more than the workhouse to fear: Towns could also banish poor people or even auction them off to the lowest bidder. “Warning out” allowed towns to exile poor newcomers or make it clear they were not willing to pay to support them.

The vendue system allowed cities to auction off poor individuals to private bidders. The individual who bought the poor person then put them to work in exchange for reimbursement of what it cost to clothe and feed them. Sometimes, people had another option—asking the Overseer of the Poor, a town official, for relief. In some cases, the overseer would provide them with town-sponsored food, clothing or firewood.

By the early 19th century, the poorhouse system had won out over warning or vendue—and their construction coincided with an increasingly negative attitude toward poor people. These facilities were designed to punish people for their poverty and, hypothetically, make being poor so horrible that people would continue to work at all costs. Being poor began to carry an intense social stigma, and increasingly, poorhouses were placed outside of public view.

Paupers struck a difficult bargain in exchange for shelter and food. As historian Debbie Mauldin Cottrell writes, many states required them to give an oath “swearing to their lack of worldly goods and to their need for assistance,” a ritual designed to weed out those who didn’t want to publicly swear that they were poor. Once they were an “inmate” of the facility, they had to submit to an often draconian control of what they ate and wore and how they worked and acted. “Consequently, it was the most desperate, those with the least pride, who often populated poor facilities,” Cottrell notes.

Often, conditions in poorhouses were alarming. However, life in the poorhouse was not always miserable. Historians have documented the ways poor people used workhouses and poor farms as places in which to build community during their most vulnerable moments. As historian Ruth Wallis Herndon has noted, many women returned to the Boston Almshouse again and again and maintained connections to the outside world while inside the poorhouse. “For most men, on the other hand, the Almshouse was an unfamiliar place in an unfamiliar city,” she writes.

For men on the move, there were alternatives to poorhouses: the tramp house. These tiny, temporary homes were erected for vagrants and itinerant people—often men—passing through communities. Often amounting to little more than shacks, these houses provided the bare basics, like mattresses and firewood, to people in towns near railroads, and generally weren’t publicized since communities didn’t want to advertise their charity toward vagrants.

As society tried to hide institutions designed to help the poor, poorhouses took on another form: the poor farm. Like poorhouses, these institutions were carefully regulated places for indigent people to live and work. However, poor farms were located in rural areas and the outskirts of cities instead of city centers. Instead of performing industrial or domestic labor, residents did farm work instead.

The poorhouse faded out around the time of the Great Depression as the federal government became more involved with social welfare. Most remaining poor farms and poorhouses closed in the 1930s and 1940s, though a few remained in places like Texas until the 1970s.

Though the poorhouses are no longer, their memory is preserved in testimony by people like Anne Sullivan. “I doubt if life, or eternity for that matter, is long enough to erase the errors and ugly blots scored upon my brain by those dismal years,” she wrote later.