The abandoned Bannerman Castle is deteriorating on the Hudson River in New York. The island it sits on opened to visitors in 2020.National Archives and Records Administration, Joey Hadden/Business Insider The abandoned Bannerman Castle sits on Pollepel Island in New York's Hudson River. Once a fortress for weapons, the castle is only accessible by private boat. Today, the facility hosts tours and is used as a theater venue.