Williams was born in Plaquemine, Louisiana, ran away from home at age 12 to join Billy Kersand's Traveling Minstrel Show, then moved to New Orleans. At first Williams worked shining shoes and doing odd jobs, but soon became known as a singer and master of ceremonies. By the early 1910s he was a well regarded local entertainer also playing piano, and was composing new tunes by 1913. Williams was a good business man and worked arranging and managing entertainment at the local African-American vaudeville theater as well as various saloons and dance halls around Rampart Street, and clubs and houses in Storyville.
Williams started a music publishing business with violinist/bandleader Armand J. Piron 1915, which by the 1920s was the leading African-American owned music publisher in the country. He toured briefly with W.C. Handy, set up a publishing office in Chicago, then settled in New York in the early 1920s. In 1921, he married blues singer and stage actress Eva Taylor with whom he would frequently perform. He supervised African-American recordings (the 8000 Race Series) for New York offices of Okeh phonograph company in the 1920s in the Gaiety Theatre office building in Times Square. He recruited many of the artists who performed on that label. He also recorded extensively, leading studio bands frequently for OKeh, Columbia and occasionally other record labels.
He mostly used "Clarence Williams' Jazz Kings" for his hot band sides and "Clarence Williams' Washboard Five" for his washboard sides. He also produced and participated in early recordings by Louis Armstrong, Sidney Bechet, Bessie Smith, Virginia Liston, Irene Scruggs, and many others. King Oliver played cornet on a number of Williams' late 1920s recordings. He was the recording director for the short-lived QRS Records label in 1928.
Most of his recordings were songs from his publishing house, which explains why he recorded tunes like "Baby Won't You Please Come Home", "Close Fit Blues" and "Papa De-Da-Da" numerous times.
In 1933, he signed to the Vocalion label and recorded quite a number of popular recordings, mostly featuring washboard percussion, through 1935.
In 1943 Williams sold his extensive back-catalogue of tunes to Decca Records for $50,000 and retired, but then bought a bargain used goods store. Williams died in Queens, New York City in 1965 and was interred in Saint Charles Cemetery in Farmingdale, Long Island, New York. On her death in 1977, his wife was interred next to him. Their grandson is Clarence Williams III.
Clarence Williams' name appears as composer or co-composer on numerous tunes, including a number which by Williams' own admission were written by others but which Williams bought all rights to outright, as was a common practice in the music publishing business at the time. Clarence Williams hits include "I Wish I Could Shimmy Like My Sister Kate" (as publisher - not composer), "Baby Won't You Please Come Home", "Royal Garden Blues", "Tain't Nobody's Business If I Do", "Shout, Sister, Shout" and many others. In 1970, Williams was posthumously inducted into the Songwriters Hall of Fame.