Levine grew up in industrial Detroit. The familial, social, and economic world of 20th century Detroit is one of the major subjects of his life's work. His portraits of working class Americans and his continuous examination of his Jewish immigrant inheritance (both based on real life and described through fictional characters) has left a monumental testimony of mid-20th century American life. It can be best found in books such as "They Feed They Lion," the National Book Award-winning "What Work Is," "A Walk with Tom Jefferson," and in his "New Selected Poems." Growing up, Levine faced the anti-Semitism embodied by a local celebrity, the pro-Hitler radio priest Father Coughlin.

Levine began to write poetry while he was going to night school at Wayne University (now Wayne State University) in Detroit and working days at one of that city's automobile manufacturing plants. He earned a Master of Fine Arts degree from the Iowa Writers' Workshop at the University of Iowa, where he studied with Robert Lowell and John Berryman.

Levine's working experience lent his poetry a profound skepticism in regard to conventional American ideals. In his first two books, On the Edge (1963) and Not This Pig (1968), the poetry dwells on those who suddenly become aware they are trapped in some murderous processes not of their own making.

In his first two books, Levine was somewhat traditional in form and relatively constrained in expression. Beginning with They Feed They Lion, Levine's poems are typically free-verse monologues tending toward trimeter or tetrameter. The music of Levine's poetry depends on tension between his line-breaks and his syntax. The title poem of Levine's book 1933 (1974) is a good example of the cascade of clauses and phrases one finds in his poetry.

On November 29, 2007 a tribute was held in New York City in anticipation of Levine's 80th birthday. Among those celebrating Levine's career by reading Levine's work were Yusef Komunyakaa, Galway Kinnell, E. L. Doctorow, Charles Wright, Jean Valentine, and Sharon Olds. Levine himself read several new poems. He thanked his students and asked them to refrain from asking for any more letters of recommendation.