Glazer grew up in East Harlem and the East Bronx in New York City. His parents, working class Polish immigrants, spoke Yiddish in the home, and his father was a sewing machine operator. Glazer attended public school as a child and eventually the City College of New York. At the time, the early 1940s, CCNY was known as a hotbed of radicalism, and Glazer fell in with a number of other young Marxists who were hostile to Soviet-style communism. Glazer, Irving Howe, Daniel Bell, and Irving Kristol would meet in an alcove of the CCNY cafeteria and "spen[d] their days feverishly trying to understand how the socialist ideal of political and economic justice had ended in Joseph Stalin’s murderous tyranny."

As Glazer would later recall, "one of the characteristics of [our] group was a notion of its universal competence...culture, politics, whatever was happening we shot our mouths off on...It was a model created by the arrogance that if you’re a Marxist you can understand anything and it was a model that even as we gave up our Marxism we nevertheless stuck with." Looking back years later Irving Kristol would remark that "even at City, [Glazer] was never much of a radical."

World War II led to a belief among some of the CCNY leftists, including Glazer, that fascism was a greater threat than capitalism, and that the United States, as a country that fought the fascists, ought to be viewed more favorably. Glazer became a member of the anti-communist left, and only mildly criticized Joseph McCarthy when writing of him in the magazine Commentary.

Looking back at the McCarthy era over 40 years later as an interviewee in the film Arguing the World, Glazer reflected on the stance he and some other liberal anti-communists took: "Even at the time and also in retrospect we never managed to figure out a good position, one that was respectable and moral and responsive to all the complicated issues raised...I still don't think we have one." In general much of Glazer's work in the 1950s had strong strains of patriotism and optimism about the future, including a belief that the overwhelming majority of immigrants would come to identify completely with American values and thus assimilate into American society.

In 1960 Glazer began writing articles about ethnic groups in New York City, and these would eventually be collected and published in 1963 as the book Beyond the Melting Pot: The Negroes, Puerto Ricans, Jews, Italians and Irish of New York City, arguably Glazer's most well-known work. While Daniel Patrick Moynihan was listed as co-author, and the book itself would often be referred to as "Moynihan and Glazer", Moynihan wrote only the chapter on the Irish and much of the conclusion, with the rest being the work of Glazer.

In essence, as one retrospective noted 25 years later, Glazer and Moynihan suggested that "the melting pot metaphor didn't hold water." The book argued that the children and grandchildren of earlier immigrants to New York City, including Puerto Ricans, Jews, Italians, and Irish, had retained their ethnic consciousness and that the phenomenon of persistent ethnic identities over the course of generations would continue to endure. This conclusion was fairly novel for the early 1960s, a time when there was relatively little interest in studying ethnic groups.

Glazer and Moynihan also argued that the "disproportionate presence of Negroes and Puerto Ricans on welfare" was one of the primary racial problems in the city, though they suggested the 1970s could end up being a "decade of optimism" for those two groups. Years later when marking the 25th anniversary of the book, Glazer would admit that the 1970s (and indeed later years) had not brought significant change for these groups, and that many African Americans and Puerto Ricans remained members of the "great dependent class." James Traub has argued that Beyond the Melting Pot was "one of the most popular, and most influential, works of sociology of its time."

During Kennedy's presidency Glazer worked in the Housing and Home Finance Agency, predecessor to the Department of Housing and Urban Development, and in Lyndon Johnson's administration he was a consultant with the Model Cities Program. By this time Glazer was becoming skeptical of the War on Poverty and of Washington-based reform efforts in general. As he would argue years later in his book The Limits of Social Policy, Great Society programs were not really the answer because "the breakdown of traditional modes of behavior is the chief cause of our social problems" and he did not think that breakdown could be addressed by government. Glazer's view was an example of the culture of poverty arguments that were gaining traction by the mid-1960s as explanations for social inequalities (Moynihan's influential, if controversial, The Case For National Action, was perhaps the most prominent work in this vein).

By 1964 Glazer was teaching at UC-Berkeley and he bore witness to the famous Free Speech Movement that year. Despite his earlier experience as a student radical, Glazer was one of many professors who viewed the student protesters as extremists. Even in the late 1990s Glazer continued to condemn the students' "enthusiastic and euphoric rejection of forms and norms," and in 2005 he pointed out that the revolts at Berkeley, Columbia University, and many other campuses constituted "a disorder that made no sense to those of us who had come from harder circumstances." One leader of the FSM, Jackie Goldberg, reflecting back on 1964 years later decried Glazer and his ilk for espousing "an armchair intellectual liberalism" and viewing "protesting" as nothing more than sending a letter to one's congressman.

As the Free Speech Movement raged in Berkeley, Glazer's friends from City College Daniel Bell and Irving Kristol were discussing founding a new journal, which would come to be called The Public Interest when it debuted in 1965. A conference essay by Glazer, "Paradoxes of American Poverty," would appear in the journal's first issue. In the summer of 1973 he succeeded Bell, becoming co-editor along with Kristol, a post which he held until 2003. When The Public Interest ceased publication in 2005, Glazer wrote a piece acknowledging the rightward drift on the part of the journal over the years and the tendency it had to publish far more conservative than liberal pieces, something which he saw "as a failing on our part."

In 1969 Glazer began a long teaching career at Harvard after being awarded one of five positions created to focus on the problems of the cities.

Glazer continued to publish books on race and ethnicity throughout the 1980s and 1990s. We Are All Multiculturalists Now, published in 1997, perhaps created the biggest stir. In it Glazer argued that multiculturalism was now the dominant ethic in public schools, while "assimilation" had become a dirty word. For Glazer it was a simple reality which could not be denied, but he remained deeply ambivalent about multiculturalism, arguing that it "is not a phase we can embrace wholeheartedly, and I hope my own sense of regret that we have come to this will not escape the reader."

Writing in what one commentator deemed a "rueful" tone, he suggested his earlier arguments regarding issues like affirmative action and the future prospects for African Americans were essentially wrong—the civil rights legislation of 1964 and 1965 had not allowed blacks to fully integrate into American society, their situation was worse now than it had been 20 years before, and a multicultural curriculum in schools was essentially the result.

The book was heavily criticized by conservatives, with Dinesh D'Souza accusing Glazer of "cowardice" in a review in The Weekly Standard, while a critique in National Review suggested Glazer was wrongly advocating for "resignation and accommodation" to multiculturalism rather than the "forthright opposition in defense of our constitutional republic and its liberal-democratic virtues" that was needed. James Traub, on the other hand, argued that "Glazer is still the neoconservative who wrote The Limits of Social Policy," but that his "own logic leaves him with nothing to offer—except the admittedly specious comforts of multiculturalism."

As the term "neoconservative" became common parlance during the administration of president George W. Bush, Glazer pointed out that it had been "hijacked" and now meant something quite different than it once had. According to Glazer, "in its early application, in the 1970s, it referred to the growing caution and skepticism among a group of liberals about the effects of social programs. It was later applied to a vigorous and expansionist democracy-promoting military and foreign policy, especially in the wake of the dissolution of the Soviet Union." In a 2003 letter to The New York Times Glazer argued that "there is very little connection between those called "neoconservatives" 30 years ago and neoconservatives today, who are defined entirely by their hard stance on foreign and military policy."

Glazer's most recent book is the 2007 publication From a Cause to a Style: Modernist Architecture’s Encounter With the American City, an essay collection "that traces the diminishment of Modernist architecture from a social revolution ... which asserted that traditional architecture 'had come to an end' ... down to a mere style, and one almost universally resented outside the profession."