Charles Fifer was born in 1922 in Evanston, Illinois. He was educated at New Trier High School, Northwestern University, and Yale University, where he wrote his PhD dissertation on the correspondence of James Boswell.
During World War II he served with the 34th Infantry Division in Italy and France. In 1955 he married Norma Crow, a fellow English teacher who later taught for many years at Crystal Springs Uplands School in Hillsborough, CA.
Charles taught at Iowa State, Lawrence University, the University of Illinois, and at Stanford from 1956 until his retirement in 1991. In addition to teaching courses in 18th-century literature, which he once described as "one of the less inhabited periods," he was for many years the Director of Freshman English, a program that served the whole university by ensuring that every undergraduate had competent writing skills.