If I did not want more people to live near me, I would simply not live in the most economically productive region in the country. People who hate other people are free to sell their homes and move to the country.
Why introduce "hate other people" into the discussion? Would you maybe allow that what some people actually "hate," are jammed highways and thoroughfares, not enough parking, and everything crowded? To the extent that people have a say (a vote) in zoning regulations, people are entitled to voice their opinion about the population density of the area where they live.