Millhauser was born in New York City, grew up in Connecticut, and earned a B.A. from Columbia University in 1965. He then pursued a doctorate in English at Brown University. He never completed his dissertation but wrote parts of Edwin Mullhouse and From the Realm of Morpheus in two separate stays at Brown. Between times at the university, he wrote Portrait of a Romantic at his parents' house in Connecticut. His story "The Invention of Robert Herendeen" (in The Barnum Museum) features a failed student who has moved back in with his parents; the story is loosely based on this period of Millhauser's life.

Until the Pulitzer Prize, Millhauser was best known for his 1972 debut novel, Edwin Mullhouse. This novel, about a precocious writer whose career ends abruptly with his death at age eleven, features the fictional Jeffrey Cartwright playing Boswell to Edwin's Johnson. Edwin Mullhouse brought critical acclaim, and Millhauser followed with a second novel, Portrait of a Romantic, in 1977, and his first collection of short stories, In The Penny Arcade, in 1986.

Possibly the most famous of his short stories is Eisenheim the Illusionist. This is based on the pseudo-mythical tale of a magician who stunned audiences in Vienna in the latter part of the 19th century, and was made into the successful film, The Illusionist (2006) directed by Neil Burger and starring Paul Giamatti and Edward Norton.. The film is not entirely true to Millhauser's work and the same could be said about Millhauser's adaptation of the original tale; both adaptations romanticise the rumour. Millhauser is the only critically-renowned author to have explored the extraordinary tale of Eisenheim the Magician, which, while being entirely implausible, was nevertheless verified by his huge audiences at the time, who claimed that Eisenheim would create spirits in front of audiences, nightly. The Illusionist, in turn, grossed more than US$120 million worldwide.

Millhauser's stories often treated fantasy themes in a manner reminiscent of Poe or Borges, and with a distinctively American voice. As critic Russell Potter has noted, "in (Millhauser's stories), mechanical cowboys at penny arcades come to life; curious amusement parks, museums, or catacombs beckon with secret passageways and walking automata; dreamers dream and children fly out their windows at night on magic carpets."

Millhauser's collections of stories continued with The Barnum Museum (1990), Little Kingdoms (1993), and The Knife Thrower and Other Stories (1998). The unexpected success of Martin Dressler in 1997 brought Millhauser increased attention. His short story "Eisenheim the Illusionist" was adapted for the 2006 film The Illusionist,

Two recent short stories, "History of a Disturbance" in the March 2007 New Yorker and "The Wizard of West Orange" in the April 2007 Harper's Magazine, address Buddhist themes. Millhauser lives in Saratoga Springs, New York and teaches at Skidmore College.