Then use a laptop instead? Or you have one of those "modern" banks that's app only?

This is probably the only real solution. It also makes sense from a getting mugged or breaking your phone perspective. At this point, my phone is probably more important than my IDs and passports.