Residents said they feel like they 'are not being heard' as they leave a public meeting to raise concerns over a proposed children's home.