The Duke of Sussex is said to have considered wearing a morning suit, as he was so embarrassed with the removal.
Despite King Charles III allowing Harry to wear uniform for the vigil inside Westminster Hall on Saturday, his grandmother's initials were not visible.
The initials were not removed from Prince William's uniform.
Prince Andrew, meanwhile, retained the initials on his vice-admiral's uniform as Queen Elizabeth II's children held a similar vigil on Friday evening.
Queen Elizabeth II's grandchildren during the vigil on Saturday evening
POOL
Prince Harry and Prince William in their military uniform
POOL
This, according to The Sunday Times, is the main source of Harry's frustration as, like him, Andrew is no longer a working royal.
The "ER" initials are traditionally only worn by those "in service" of the monarch, so the absence of the cypher is thought to be down to dress regulations – but why Andrew retained his remains unclear.
A friend of Harry's told the newspaper: "He is heartbroken. To remove his grandmother’s initials feels very intentional."
Harry, who previously wore the cypher as a Personal Aide de Camp, is said to have considered wearing a morning suit instead.
The Duke of Sussex wore a Blues and Royals No 1 Uniform, KCVO Neck Order and Star, Afghanistan Operational Service Medal, Golden, Diamond and Platinum Jubilee medals and Army Pilot Wings.
The Prince of Wales stood at the head, while the two brothers were flanked by cousins Zara Tindall, Peter Philips, Princesses Beatrice, Princess Eugenie, Lady Louise Windsor and her brother James, Viscount Severn.
The late monarch's eight grandchildren looked deep in thought as they stood guard around her coffin on Saturday evening.
They had their heads bowed as members of the public filed past them.
Harry, who saw action on the front line during two tours of duty in Afghanistan, has previously been denied the chance to wear his military uniform as he publicly mourns, because he is no longer a working royal.
Despite being a former Army officer, Harry has been in civilian dress for official events, including walking behind Queen Elizabeth II's coffin on Wednesday.