Born and raised in Pine Bluff, Arkansas, with some time based in Little Rock, Gordy had a fascination with the piano but did not make it a career for many years. He spent much of his childhood focusing attention on ragtime and New Orleans' style jazz. His brother Chester played trumpet, so there was some additional talent in the family.
In his twenties he literally followed a traditional jazz band around the country, making friends with the piano player, and occasionally sharing the seat. Gordy eventually became a fine rhythm player in both jazz and country bands of the 1930s to mid 1940s, working at hotels and even for high school dances, where there was work. He also played on a few recordings, many of which he did not receive credit so we can only guess about them. When Jim Bulleit founded his progressive Nashville label Bullet Records in 1946, Gordy came along for the ride as a session pianist for future famed artists such as Chet Atkins and Homer and Jethro (possibly including some WSM Grand Ole Opry broadcasts, since that was they studio they used), and was based in Nashville for the rest of his life.
From late 1948 to 1949, John recorded twelve sides for Bullet that would be among the last for that label. He became known as "Poppa" John during this period, and his signature Salty Dog Rag was soon covered by many country artists. Then for a time, Del Wood held the Nashville spotlight. At the urging of Atkins, Gordy, Wood and other artists migrated to RCA Victor in 1954 and 1955, once RCA set up a high tech studio in Nashville to supplant the antiquated and inadequate studios that existed up to that point in the Music City.
A number of his sides with his quintet or sextette (depending on the date) were integrated into a 1955 LP, and he recorded a second LP in 1957. A final compilation was released some time after his death in 1961.
Poppa John's style was an important link between the down home Nashville style and the more sophisticated near-jazz arrangements of Lou Busch, and like other artists such as Marvin Ash or Paul Lingle, he was woefully under-recorded.