Paley graduated in 1950 from City College of New York and received a master’s degree in mathematics from Yale in 1953. He taught the subject for many years.
In 1952, at the age of 24, he made his first solo recording with the subsequent release of Folk Songs From the Southern Appalachian Mountains on the newly-started Elektra label. By that time, he had already introduced urban audiences to Depression-era rural music, sharing what have become staples of the genre, such as "Little Maggie" and "Shady Grove."
Paley played with the New Lost City Ramblers from 1958 to 1962, a prolific period during which the group released 11 albums and made more than 150 public appearances. During those years he helped spearhead an old-time music revival.
As well as being a member of the New Lost City Ramblers, Paley enjoyed a solo career during which he collaborated with others on various recording sessions.
Paley moved to England, from Sweden in 1965 and shortly thereafter formed the New Deal String Band. In later years Paley performed and recorded with his son Ben, a fiddle player.