In UW's case, that would be even more difficult than usual because of its organization. The mathematics department is in the College of Arts and Science and the CS department is in the engineering school. These are practically like separate universities within UW.
Indeed -- CS is now the "Paul G. Allen School of Computer Science and Engineering", so it's not just a department. I never tried to move to CS, and I have no idea if there would have been a path to do so. When I first got hired at UW in 2006, some faculty in the CS department were very supportive of Sage, even giving me some temporary office space in one of their old buildings.
I would have tried to explore that option before leaving completely. I know two professors who have switched fields, one was physics professor who did work in string theory, switched to genomics and got position at a medical school. Another was a CS professor who actually went in to math.
You sometimes see CS professors who have degrees in pure math just browsing different department's websites.
