Borders of countries are fundamentally human constructs. There is no morality associated with crossing them legally or illegally. This is the difference between a law declaring something illegal because they think it is better for society (a parking ticket, say) and a law created that require moral turpitude (murder, say).
I don't have hard data yet but I'm pretty sure some cities have suburbs outside them, connected via road, that rich people use as tax havens so they can live near a city without being subject to the laws and taxation of the city
Right but if you go into a country then you're in the country, not in the outskirts. You still pay taxes (generally...), and, in many countries, don't get any social services.
If anything, many formally-colonial countries are leeching off their illegal immigrants, not the other way around.