It's never been lived in, but the castle marks the spot where an explorer abandoned his journey for a city of gold.