The property was once part of the Nathan W. Hazen Estate at 111 Main Street which extended from Locke Street to Morton Street. John H. & Frances A. Flint purchased the Nathan W. Hazen estate from his daughter Mary P. Hazen in 1888 and began to divide the property into house lots which spurred development on both Locke Street and southward along Main Street.
The house was built for Joseph A. & Martha (Stewart) Smart by William S. Jenkins, local contractor, who lived at 118 Main Street. Joseph A. Smart was born in Andover on May 18, 1848 son of Alexander & Catherine Smart. With the death of his father in 1859 Joseph went to work at the Smith & Dove Manf. Co. to help support his family. He later apprenticed with Charles Mayer the local harness maker. Joseph's ability with accounting him employment at the Merrimack Mutual Fire Insurance Co. where he worked for 47 years and later elected President of the company.
Joseph was a partner with Burton Flagg of Smart & Flagg Insurance Co. He was a Trustee of the Andover Savings Bank, Director and Vice President of the Andover National Bank, and a Trustee of Memorial Hall Library. Joseph, under the Governors Guild and Draper, served as Trustee and Chairman of the Board for the State Infirmary at Tewksbury and the State Farm in Bridgewater. He was a member of St. Mathew's Lodge A. F. and A. M., the Andover Club, Director of the Andover Press, officer of the Indian Ridge Association, President and Director of the Andover Home for Aged People, Trustee of the Cornell Fund and as the local paper wrote, "Referee in the readjustment of the Phillips Academy and Seminary properties."

The Smart family owned the 14 room house from 1888-1945. Joseph married in 1875 to Martha Stewart, the same year he was elected secretary and treasurer of the Merrimack Mutual Fire Insurance Co. They had one daughter Annie Smart who married John C. Angus, later appointed Postmaster in Andover, and grand daughter Mary Stewart Angus, whom residents remember as the Manager of the Andover COOP.

Joseph died on Dec. 4, 1912 and was interred in the Christ Church burial ground. After the death of Martha Smart the house was sold to George A. & Dorothy Colloten on May 10, 1945. George Colloten died on July 13, 1947 His wife Dorothy remained until 1951 when the property was sold to James P. & Alice McCormick, who through several deed changes in 14 years, then sold to L. John & Martia Davidson on Feb. 10, 1965.

John Davidson was an investment real estate developer and had several properties in Andover and Quechee, VT. Davidson sold to Donald & Barbara Richmond in June 1976. The property remains with the Richmond family under the name ARCO Properties LLC as of June 2007.
The former single family home was converted into apartments and contains 14 rooms, 5 units, 5 bedrooms and 5 baths.