"My place was so good – Prince came over once and he was like, ‘Oh, I see why you don’t want to go out…’”