Brady was born in Brookline, Massachusets. He graduated from Dartmouth University in 1948, before moving on to receive his Master's Degree from Yale in 1949, followed by a Ph.D. in 1952. He stayed at Yale as English instructor for almost a decade before, in 1960, moving to Dartmouth University. From 1963 until 1967 he taught at Pennsylvia State University before moving to Hunter College and the City University of New York, where he was head of the school's doctoral program in English from 1969 to 1972.
At the time of his unexpected death in September 1986, Brady was Professor of English at the Graduate Center of the City University of New York, and general editor of the Yale editions of the Boswell papers. With Marlies Danziger, he was working on the last volume of the trade editions of Boswell's journal, covering the years 1789-1795, which was completed after his death and issued in 1989.