As musician, writer, and record producer, Kaye was intimately involved with an array of artists and bands. He was a guitarist for poet/rocker Patti Smith from her band's inception in 1974, and co-authored Waylon, The Life Story of Waylon Jennings. He worked in the studio with such artists as R.E.M., James, Suzanne Vega, Jim Carroll, Soul Asylum, Kristin Hersh, and Allen Ginsberg. His seminal anthology of sixties' garage-rock, Nuggets, is widely regarded as defining the genre.[6] You Call It Madness: The Sensuous Song of the Croon, an impressionistic study of the romantic singers of the 1930s, was published by Villard/Random House in 2004.