This property was once part of the Nathaniel Swift Jr. homestead built at 53 Main St. on the corner lot of Chestnut St. This home is on the approximate site of the Swift barn or carriage house which was moved to 8 Chestnut St.

Nathaniel Swift Jr. (1805-1875) was the son of Nathaniel Swift 1778-1840, one of the first trustees and founders of the National Bank in Andover. After his death on September 29, 1875 the property was inherited by his wife Almena and children, George, Anna Hartwell, Martha and Charlotte Swift. The Swift children continued to live in the residence, the son and two daughters later selling the property in 1904 to Edward F. Searles, a millionaire of Methuen, MA. The deed however came with life tenancy for daughters Martha 7 Charlotte. Daughter Anna had married Dr. James Richards. Dr. Richards had moved to Manchester, NH from Andover in 1879 and after his wife Ellen A. (Brown) died in 1880 he returned in 1881. He remarried on November 14, 1882 to Anna H. Swift, daughter of Nathaniel & Almena Swift, sharing the Swift home at 53 Main Street until the building of his new house at 94 Main Street was completed in 1887. The Richards home was later moved, now located at 2 Punchard Ave.

The Searles Estate sold the property to the Trustees of Andover Associates in March 1922. The Andover Savings Bank purchased a portion of the property in 1923 but the house and buildings were reserved for the use of the Andover Associates. The former Swift house was moved to the rear south west corner of the lot on Chestnut Street. The carriage barn was also moved west to 8 Chestnut St. and converted into a residence. The mansion was purchased by the Knights of Columbus, the Roman Catholic mens fraternal charitable organization. Their meetings and social activities continued here. The first floor rooms became retail and professional space in the 1950’ and 60’s. The Andover Savings Bank later acquired the house form the Knight of Columbus and razed the house for parking in 1967. Andover Townsman – June 9, 1967 – Swift House – 130 year old Greek Revival house to be torn down for a parking lot for the Andover Savings Bank.

The Knights of Columbus purchased the lot behind the new bank and the Swift Mansion was moved over and converted into the K of C Lodge. The Andover Associates, trustees, John C. Angus, Frank Buttrick and John H. Campion, then sold the lot where the carriage house was located but "however grantors reserve the barn and the foundations there-under now situated upon the granted premises and the right to enter and remove the same."
This lot was then sold by the Knights of Columbus by Joseph L. Burns president and Frank S. McDonald treasurer to Jeremiah J. & Josephine Daly on October 1, 1924. Jeremiah Daly was an Andover Physician and his family built and owned the Daley Block that once stood north of the Library at 8-10 North Main Street.

Jeremiah and Josephine built their new home at 12 Chestnut St. in 1925 - 26, which included room for his professional practice. The Daly family continued ownership of the property until 1990 when it was sold to Prescott Village Development Corporation in June 1990. In April 2002 the deed was held by Twelve Chestnut Street LLC.