Gay was born at Bridge of Weir, in Renfrewshire, Scotland, eldest child of William Gay and his wife Jane née Tagg. Gay senior was a religious man, an engraver of patterns for wallpaper and calico, his mother came from an educated family. The family moved not long afterwards to the town of Alexandria, where Gay was educated at a boarding school. At 14 he became a monitor at the school and winning a bursary went to the University of Glasgow. His father wanted him to be a minister, but the boy felt he could not conscientiously follow that profession and went to London hoping to make a living there.

Destitution and illness followed and Gay had to go back to his people. Again he went to London but his strength was not sufficient and he had to go into hospital in Glasgow. Tuberculosis threatened, and a sea voyage was tried and he arrived at Dunedin, New Zealand, in 1885. Gay obtained work as a purser's clerk on vessels of the Union Line for nearly two years, when illness again led to his living with some relatives at Hawke's Bay who nursed him back to comparative health. In 1888 he went to Melbourne and obtained a mastership at Scotch College but teaching was beyond his strength. In 1891 he was in the Austin hospital, and in 1893 went to live at Bendigo. His first volume Sonnets and Other Verses, published in 1894, was followed by two other volumes Sonnets and Christ on Olympus and Other Poems in 1896. A small selection appeared in 1910 and The Complete Poetical Works of William Gay in 1911. A prose essay Walt Whitman: His Relation to Science and Philosophy was issued in 1895. Gay was a supporter of Federation and edited "The Commonwealth & the Empire" with Mary Sampson.Gay died on 22 December 1897.

Gay was a slight man of average height and was said to have had some resemblance to Tennyson. When an invalid at Bendigo one of his little volumes yielded him a profit of £40 and another was even more successful. This could only have happened with the help of friends as the volumes are without popular appeal. Gay was fortunate that so many discerning and kindly people were able to help him and care for him until his death, because he was worthy of care. His sonnets rank with the best that have been done in Australia, and in a few poems such as "The Crazy World" he has written poetry expressing simple, forceful and unstrained emotion. His life was short and marred by ill-health borne with courage. Although the amount of Gay's work was small, it holds an honoured place in the history of Australian poetry.