The circumstances of his birth in Bristol in 1741, and parentage are somewhat doubtful, and it is unclear whether his father was a rich Bristol merchant or a certain William Alexander, a London alderman, who died in 1762. He was educated at Eton, where he was contemporary with Charles James Fox, the 2nd Baron Lyttelton and William Beckford. Alexander bequeathed him some £2000...a little fortune that soon disappeared in a course of splendid extravagance, which gained him the nickname of Count Combe; and after a chequered career as private soldier, cook and waiter, he finally settled in London (about 1771), as a law student and bookseller's hack.

In 1776 he made his first success in London with The Diaboliad, a satire full of bitter personalities. Four years afterwards (1780) his debts brought him into the King's Bench Prison, and much of his subsequent life was spent in prison. His spurious Letters of the Late Lord Lyttelton (1780) imposed on many of his contemporaries, and as late as 1851, a writer in the Quarterly Review regarded these letters as authentic, basing upon them a claim that Lyttelton was "Junius." An early acquaintance with Lawrence Sterne resulted in Combe's anonymous Letters supposed to have been written by Yorick and Eliza (1779). Periodical literature of all sorts...pamphlets, satires, burlesques, "two thousand columns for the papers," "two hundred biographies"...filled up the next years, and about 1789 Combe was receiving £200 yearly from the Pitt government as a pamphleteer.

In 1790 and 91, the six volumes of a Devil on Two Sticks in England won for Combe the title of "the English le Sage". In 1794—1796 he wrote the text for Boydell's History of the River Thames, and in 1803 he began to write for The Times. From 1809 to 1811 he wrote for Ackermann's Political Magazine the famous Tour of Dr Syntax in search of the Picturesque (descriptive and moralizing verse of a somewhat doggerel type), which, owing greatly to Thomas Rowlandson's designs, was an immense success. It was published separately in 1812 and was followed by two similar Tours, "in search of Consolation," and "in search of a Wife," the first Mrs Syntax having died at the end of the first Tour. Then came Six Poems in illustration of drawings by Princess Elizabeth (1813), The English Dance of Death (1815—1816), The Dance of Life (1816—1817), The Adventures of Johnny Quae Genus (1822)...all written for Rowlandson's caricatures; together with histories of Oxford and Cambridge, and of Westminster Abbey for Ackermann; Picturesque Tours along the Rhine and other rivers, Histories of Madeira, Antiquities of York, texts for Turner's Southern Coast Views, and contributions innumerable to the Literary Repository.

In his later years, notwithstanding a by no means unsullied character, Combe was courted for the sake of his charming conversation and inexhaustible stock of anecdote. He died in London on 19 June 1823.