THE QUEEN was 'starting to get fed up' with one key aspect of royal duties due to people's behaviour, according to a royal author.