He didn’t hit his stride till he was 50, and now the architect, as inventive and bold as ever, hangs out with everyone from Harrison Ford to Jay-Z