Prince William is reportedly unhappy with the way his brother Prince Harry responded to Queen Elizabeth’s announcement that Harry and his wife Meghan Markle have officially stepped down from their royal roles.

The Duke of Cambridge, 38, is “really sad and genuinely shocked” about his brother’s behaviour, according to a report by The Sunday Times.

A statement released on Friday by Buckingham Palace confirmed that Harry, 36, and Meghan, 39, won’t return as working royals. The announcement came just over a year after the Duke and Duchess of Sussex first shared their plans to step down, which came with a one-year review period. As part of stepping down, they will no longer keep their patronages (their royal involvement with numerous U.K. charities).

“Following conversations with The Duke, The Queen has written confirming that in stepping away from the work of The Royal Family it is not possible to continue with the responsibilities and duties that come with a life of public service,” the palace statement said.

In response, Harry and Meghan pushed back with a statement that pointedly referenced the Queen’s: “We can all live a life of service. Service is universal.”

Sources close to William said he’s “very upset by what has happened,” and found Harry and Meghan’s behaviour “insulting and disrespectful.”

“You don’t answer the Queen back — it’s just not done,”