I wish I had a Google Keyboard that could easily run on Whisper Medium. This is already great. But unfortunately would be too much inference cost, incredibly slow. The problem with Whisper is not the inference quality: medium and large are incredible. Is that the base model is not enough, and the only one with fast inference in mobile devices.

FUTO keyboard is trying to do this. I think they have some kind of distillation of Whisper running on-device.

They are just shipping the same whisper-small that everyone else is using, and did not much to improve their models since release. Other models have been "coming soon" forever. https://keyboard.futo.org/voice-input-models

Have been using https://github.com/notune/android_transcribe_app And pretty happy with it. Fully local and fast and accurate

This is actually really good. I'm writing with it right now. It's just not the best setup as a keyboard. Because for example you cannot easily switch back to uh the normal keyboard with keys.

This uses Parakeet v3 which is a lot lighter but still very good accuracy