Following the War of 1812, the United States Army Corps of Engineers proposed that a fort be built on Hog Island Ledge, in Casco Bay at the entrance to the harbor at Portland, Maine. Named for the colonial proprietor of Maine, Sir Ferdinando Gorges, it was constructed to support existing forts, including Fort Preble in South Portland and Fort Scammel built on nearby House Island in 1808. Congress, however, did not fund construction of Fort Gorges until 1857. The walls of the fort were begun the next year, and when the American Civil War broke out in 1861, work quickly advanced.
The fort was designed by Colonel Reuben Staples Smart. The chief architect in charge of construction was Thomas Lincoln Casey, who later became Chief of Engineers. It is similar in size and construction to Fort Sumter, but is built of granite instead of brick.
The fort was completed in 1865 as the war ended. Modern explosives made the fort obsolete by the time it was completed. A modernization plan was begun in 1869, but funding was cut off in 1876, with the third level of the fort still unfinished. During the modernization project, sod-covered sand was added to the top level of the fort to protect gun encasements and powder magazines from attacks.
The fort was last used by the military during World War II, when it was used to store submarine mines. It was acquired by the city of Portland in 1960 and placed on the National Register of Historic Places. Owing to a large number of injuries on the fort, it is widely considered to be unsafe to visit.