Herschberg grew up in Chicago, served in the Air Force during the Korean War, and then settled in Los Angeles in the early 1950s. He took a job in Hollywood with Decca Studios in 1956, working closely with Decca A&R man and arranger Sonny Burke. In 1962, a corporate takeover of Decca by MCA put the studio staff out of work, but many, including Burke and Herschberg, ended up working at United Western Studios. There, Frank Sinatra recruited Burke to produce albums for his new label Reprise Records, and Burke enlisted Herschberg to help engineer them. Herschberg would engineer some of Sinatra's biggest hits of the '60s, including Strangers In The Night for which Herschberg, along with Eddie Brackett who had engineered the title track, would win a Grammy for the Best Engineered Recording in 1966.
Later he became chief engineer for Warner Bros. Records, where he remained from 1982 to 1994, working on records by James Taylor, Paul Simon, George Harrison, Eric Clapton and Rickie Lee Jones, among many others.