Raised on a farm in Middleton, Tennessee, Stewart moved to Memphis in 1948, after graduating from high school. He worked at Sears, at First National Bank, and then was drafted into the United States Army. After serving for two years, Stewart returned to his job as a bank clerk in Memphis in 1953.
Stewart was a part-time fiddle player and joined a local country music group called the Canyon Cowboys. He worked days as a banker at Union Planters Bank. In 1957, Stewart launched his own record label, then called Satellite Records, which issued country music records. His sister, Estelle Axton, mortgaged her home to invest in her brother's venture by buying an Ampex 300 tape recorder. In 1959, the label moved into the former Capitol Theatre in Memphis. The auditorium was converted to studio space and the stage converted to a control room. To save money, Stewart did not level the floor. This created unique acoustics, which showed up in the recordings made there. After Stax went bankrupt in 1976, Stewart kept a low profile and intensely protected his privacy. When he was inducted into the Rock and Roll Hall of Fame in 2002 (Non-Performer), he sent his two granddaughters to the induction ceremony to accept the award on his behalf.