If you're not developing an iOS/macOS app, you can skip Xcode completely and just use the `swift` CLI, which is perfectly cromulent. (It works great on Linux and Windows.)
There'a great indie app called Notepad.exe [1] for developing iOS and macOS apps using macOS. You can also write and test Swift apps for Linux easily [2]. It also supports Python and JavaScript.
If you hate Xcode, this is definitely worth a look.
So wait this thing is real? Calling it notepad.exe gave me the impression that it's just an elaborate joke about how you can code any program in Notepad...
Even if you're developing for macOS you can skip xcode. I've had a great time developing a menubar app for macOS and not once did I need to open xcode.
I would avoid it for Linux and Windows. Even if they are "technically supported", Apple's focus is clearly macOS and iOS. Being a second- (or even third-) class citizen often introduces lots of issues in practice ("oh, nobody teenaged that functionality on Windows"...)