Then-senior constable Kristian James Samuel White fired his Taser at Clare Nowland minutes after being called to the Yallambee Lodge nursing home at Cooma in southern NSW.
She died in hospital a week after the incident in the early hours of May 17, 2023.
The 35-year-old was handed a two-year good behaviour bond and ordered to complete community service in March after a jury found him guilty of manslaughter.
The Crown quickly lodged an appeal against the "manifestly inadequate" penalty, with the state's most senior prosecutor on Friday arguing the sentencing judge made a number of errors.
NSW Director of Public Prosecutions Sally Dowling argued Justice Ian Harrison should have taken into account victim impact statements from Mrs Nowland's family as evidence of the broader harm wreaked on the community.
Several relatives previously told the court they had lost trust in the ability of police officers to protect vulnerable people.
"It's shock, it's loss of confidence ... it's torn the fabric of society," Ms Dowling told three judges in the NSW Court of Criminal Appeal.
While Justice Harrison considered the statements when assessing the objective seriousness of the manslaughter, he didn't stress the harm done to the wider community.
Instead, Ms Dowling said Justice Harrison mistakenly used the statements to find White had suffered extra-curial punishment as it would be hard for him to continue living in the Cooma community.
It amounted to a mishandling of the evidence that fed into the delivery of an inadequate sentence, she argued.
During a two minute and 40 second interaction, White drew his weapon and pointed it at Mrs Nowland before saying "nah, bugger it" and discharging the Taser at her chest.
The elderly woman, who weighed less than 48kg, fell and suffered a fatal brain injury.
In his judgment, Justice Harrison said the need to deter other police officers from committing similar offences played "only a minor role" in his sentencing decision.
But Ms Dowling argued he made a mistake by not issuing a strong warning to other officers.
"(Police) occupy a trusted role in our community and they must take the utmost care to use those weapons carefully," she said.
"This case required a sentence that sends a clear message that an unjustified resort to force or violence when other options are available will result in serious criminal punishment."
Mrs Nowland was experiencing dementia symptoms, relied on her walking frame to move and was "extremely vulnerable" when she encountered White, the prosecutor said.
The great-grandmother was holding a knife but didn't advance towards White while brandishing it and had no discernible reaction to his requests to put the blade down.
"(White) did not give her any real chance to avoid being tasered," Ms Dowling said.
"There were many other options available to him".
Justice Harrison also made a mistake when he found the officer made a "terrible mistake" but his crime was less serious than other manslaughter offences, Ms Dowling argued.
"The truly senseless, impatient, and violent conduct of the offender requires a sentence that gives weight to denunciation," she said.
White's lawyers are expected to address the court on Friday afternoon.
