Bartlet Hall was constructed in 1821 and dedicated on September 13, 1821. Built as a dormitory for the Andover Theological Seminary, it was the third brick building in Seminary Row and flanked the south side of Bartlet Chapel to its left. Capt. William Bartlet of Newburyport, MA was again the benefactor contributing $19,574 for the brick four story building. Bartlet referenced to it as the "College Edifice which you were pleased to grant me permission to erect on your grounds for the accommodation of the students of the Theological Seminary."

The interior design "included 32 studies, with two connecting bedrooms each having a fireplace with a broad hearth. Mrs. Bartlet choose the furnishings for the rooms and it was reported that her selection were much better than those picked by Squire Farrar for Phillips Hall. "Elijah Kellog wrote "Spartacus" in the southwest corner room, first story, while at the Seminary in 1843. The Seminary bookstore was located in room No. 4 in 1884."

When the Seminary moved to Cambridge, MA in 1908 all their real estate holdings in Andover were purchased by Phillips Academy. The Academy had actually "rented Bartlet Hall in 1906 and 1907 and fitted up with shower baths, steam heat and other conveniences. It accommodated fifty students in single and double suites and two resident instructors."

A fire on December 8, 1914 on the fourth floor gutted the top floor and roof. Water damage below necessitated a complete reconstruction and restoration of the interior, including original design elements and a new roof. Work was completed in May 1915 and the building was rededicated on Founders Day, October 1915.

With the re-design of the campus in the 1920's the fourth floor of Bartlet Hall was completely removed in the summer of 1929 to conform to the architectural style of the new buildings and to reduce the fire hazard.