This building was built in 1809 and was the first Seminary building constructed on "The Hill". The buildings construction was funded by Madame Phoebe Foxcroft Phillips and her son, Col. John Phillips for the sum of $16,000. The brick structure with slate hip roof was built primarily as a dormitory for Seminary Students. "It was four stories high, divided down the middle with a front and rear entry on each side." It is stated that the building was modeled after the dorms at Brown University.

It contained rooms for a chapel, a library-reading room, thirty studies and accommodation for students. Samuel Francis Smith, the noted minister and hymnist, wrote "The Morning Light Is Breaking" and "Yes, May Native Land, I Love Thee" in the rear northeast corner, second floor of this building while a student at the Seminary in 1832. Smith penned "America" - My Country Tis of Thee, in the Blunt House, now known as the America House at 139 Main Street.

When the Andover Theological Seminary moved to Cambridge in 1908 Phillips Academy purchased all of the real estate holdings of the Seminary on July 1, 1908. The building became a dormitory but in 1911 the building was deemed structurally unsound and was to be torn down. The building was saved and after a complete examination of the structure it was determined the building could be repaired. Holt Fairchild Co. of Boston was given the contract to restore the building. The entire east wall was torn our, a new foundation was placed under that wall and all the rest of the foundation was strengthened. Stone steps reset and new columns added to support the floors above. New plumbing updated to dormitory. Work was completed by the spring term of 1912. The building would house 28 boys, 14 on each side of the building. Total cost of restoration was $17,000.

With the opening of Samuel Phillips Hall in 1924 the former Phillips Hall was renamed Foxcroft Hall in 1927. The forth floor, as was Bartlet Hall, was removed in the summer of 1929 to conform to the architecture of the new campus buildings and to eliminate a firs hazard.