Gilder was born at Bordentown, New Jersey. He was the son of the Rev. William Henry Gilder, at whose seminary in Flushing, New York, he was educated. He was the brother of William Henry Gilder, Jeannette Leonard Gilder and Joseph Benson Gilder. Gilder studied law at Philadelphia.

During the American Civil War, he enlisted in the state's Emergency Volunteer Militia as a private in Landis's Philadelphia Battery at the time of the Robert E. Lee's 1863 invasion of Pennsylvania. After the Confederates were defeated in the Battle of Gettysburg, Gilder and his unit were mustered out in August.

With Newton Crane, he founded the Newark Register and later was editor of Hours at Home and edited Scribner's Monthly (afterwards the The Century Magazine). In 1881 he succeeded Dr. Josiah Gilbert Holland as editor in chief of Century, a position he held up to the time of his death. Gilder took an active interest in all public affairs, especially those which tend towards reform and good government, and was a member of many New York clubs. He was one of the founders of the Society of American Architects, of the Authors' Club, and of the International Copyright League. He was a founder of the Anti-Spoils League and a member of the American Academy of Arts and Letters. He was a close friend of George MacDonald, Scottish poet, author, and preacher. They collaborated in various ventures such as MacDonald's American lecture tour in the '70s. Gilder received the degree of LL.D. from Dickinson College in 1883.

Gilder's wife, Helena de Kay (1846—1916), was a talented painter and a founder of the Art Students League and Society of American Artists. She also modeled for, and was an unrequited love of, the painter Winslow Homer. Their son, Rodman de Kay Gilder (1877—1953), became an author and married Comfort Tiffany, a daughter of Louis Comfort Tiffany. A celebrated plaster sculpture of the family by Augustus Saint-Gaudens is owned by the Metropolitan Museum of Art [1].