More than 300 objections were put forward in response to the bid from Serco, the group managing asylum seeker accommodation for the Home Office.
In a meeting on Wednesday evening, members of the public expressed their anger and frustration at the plans to Stafford Borough Council's planning committee.
Signs reading "Say no to Serco" were displayed in a bid to pile pressure on administrators.
With a council officer warning that police could be called, the meeting was briefly paused due to disruption from the public seating area.
The shelved plans were set to hit a site on Weston Road, Stafford
Serco manages asylum seeker accommodation for the Home Office
Serco
Boos and hisses rung out, reports Stoke-on-Trent Live, and a supporter of the application was interrupted by comments referring to child sex crimes in Telford.
Up to 481 asylum seekers could have been housed at the former Staffordshire University halls of residence.
171 bedrooms were earmarked for urgent stays, while a further 310 dispersed accommodation bedrooms were intended for single adults for extended periods.
Residents living near the site feared for the safety of local children nearby. There were also concerns that crime could rise in the area.
Councillor Frances Beatty said: “There is no lack of empathy but parents are understandably anxious about how the way of life of their children will be affected, walking to school or at play.
“There is no indication that the local NHS services will have the capacity and specialisms to meet the needs of this population.
"Our GP surgeries are at full stretch – and there is a national shortage of GPs.”
Lisa Dysch, who spoke in support of the application, said: “It cannot be denied that more facilities like the one we are proposing are required.
“We recognise that this is the first time asylum seekers will be accommodated within the borough and understand this has resulted in questions and fears.”
The application was recommended for approval by council planning officers.
Committee members were told there had been no objections raised by statutory consultees.
But the committee went against the recommendation and voted to refuse permission.
Seven members voted against the proposal, with two voting in favour and one abstention.