Police Minister David Elliott says the appointment, which is now the subject of two inquiries, is "an unnecessary distraction" for the Perrottet government which is trying to spruik its budget.
"My very, very strong view is that he shouldn't be going to New York and I'm pretty sure that that's the view of the majority of people in this state," Mr Elliott told Sky News on Thursday.
"I expect him to withdraw and do the honourable thing, because quite clearly when you become the story over an appointment, well, then the appointment might be not in the best interest of the people of NSW."
Labor has declared the appointment of the former Nationals leader to the three-year-posting in Manhattan is "very, very fishy".
Investment and Trade Minister Stuart Ayres also weighed in on the escalating issue, saying he had concerns after Mr Barilaro texted him about the plum job in December.
"He contacted me via text message ... indicating that he was interested in it," Mr Ayres told reporters on Thursday.
"I informed him that they would be publicly advertised and he, like any other private citizen, will be able to apply."
He admitted he had always held the view Mr Barilaro's appointment would be politically contentious but said he did not have the authority to direct Mr Barilaro to resign.
The government announced two weeks ago Mr Barilaro had been appointed the Senior Trade and Investment Commissioner, a job he created while trade minister.
The job comes with a $487,000 salary plus a $16,000 allowance, with Mr Barilaro due to start work in the Manhattan office on July 12.
Nearly $1 million has been spent refurbishing part of the Australian consulate in New York for Mr Barilaro to occupy.
A parliamentary inquiry on Wednesday heard senior Investment NSW bureaucrat Jenny West was told last year she had the job.
Investment NSW boss Amy Brown told the inquiry she had to withdraw the verbal offer to Ms West in October after a request was made by Mr Barilaro's office in September to move the trade commissioner positions from an in-house decision for her department to one requiring cabinet approval.
Days later Mr Barilaro announced his retirement as an MP, officially ending his time in parliament in December.
Mr Barilaro was offered the job in May and signed the contract earlier this month.
"Today we well and untruly know that the Minister Stewart Ayres has his hands over this dirty business," Labor's Prue Car said.
"The fact that John Barilaro texted minister Ayres and then miraculously this job, this plum job in New York City gets re-advertised and John Barilaro gets reappointed.
"This thing is very, very fishy. The people of NSW are not mugs.
"They know that something is up here."
The parliamentary inquiry will resume on Wednesday when Ms West will appear as a witness.
Premier Dominic Perrottet has also announced a separate review into the recruitment process, which he expects within weeks.
"I will look at the recommendations and any action that needs to be taken I will," he said on Tuesday.
AAP has contacted Mr Ayres for comment.