George Washington was excessively fond of hazelnuts. Chester Arthur was a celebrated gourmand who put away mutton chops by the dozen. James Garfield enjoyed nothing more than a steaming bowl of squirrel soup. We’ve heard about the quirky favorite foods of America’s commanders-in-chief, but what do we know of the cooks who labored over these meals? Nowadays, many cooks attain celebrity status, and the position of “executive chef”—first filled by René Verdon when the Kennedys inhabited the White House—brings respect and renown. In the past, however, presidential cooks toiled under conditions that were anything but glamorous.

The first “first chef” was a slave named Hercules whom George Washington brought from Mount Vernon to the new President’s House in Philadelphia. A talented cook, Hercules presided over a kitchen brigade of slaves who whipped up everything from cornmeal mush to elaborate dinners hosted by the first lady. Dapper and charming, he bought himself expensive suits, watches and hats with the money he earned from selling kitchen byproducts like rendered fat and animal skins. On Washington’s 65th birthday, Hercules finally escaped to freedom.

The second president, John Adams, had simpler tastes than his predecessor, as well as no slaves and a relatively tight household budget. He employed a white servant couple, John and Esther Briesler, to cook and serve basic dishes like cabbage pudding and oyster stew. The gourmet habits of Thomas Jefferson, by contrast, became the stuff of legend. He took one of his slaves, James Hemings (brother of Sally), to France with him to study French cuisine under some of the most famed chefs of the time. Hemings became a brilliant cook in his own right, serving up leg of mutton and macaroni pie alongside consommés and the newfangled treat ice cream. To thank Hemings for his service, in 1793 Jefferson promised to free him as long as he passed along his knowledge of French cuisine to another slave. Hemings immediately began teaching his younger brother and was freed two years later.

Once construction was completed on the White House in 1800, servants and slaves alike were housed in the basement, near the smell and heat of the kitchen rooms. Little is known about the daily lives of those who worked in the presidents’ service: Few slaves were literate, and it can be assumed that there was minimal time for personal reflection in the face of 4 a.m. wake up calls and 16-hour days. Quarters were furnished with basic furniture in poor condition, and rats and mold were everyday problems. Still, White House cooks put food on the table for some of the most celebrated politicians of the age.

Stay tuned for a look at more of the unsung heroes of the presidential kitchen, along with how Jacqueline Kennedy’s penchant for French food brought White House cuisine into the modern era.