Firefox also has on-device translations for what its worth.

That depends on what counts as “a handful of languages” for you.

You can use llm for this fairly easily:

    uv tool install llm

    # Set up your model however you like. For instance:
    llm install llm-ollama
    ollama pull mistral-small3.2

    llm --model mistral-small3.2 --system "Translate to English, no other output" --save english
    alias english="llm --template english"

    english "Bonjour"
    english "Hola"
    english "Γειά σου"
    english "你好"
    cat some_file.txt | english

https://llm.datasette.io

Setting aside general-purpose LLMs, there exist a handful of models geared towards translation between hundred of language pairs: Meta's NLLB-200 [0] and M2M-100 [1] can be run using HuggingFace's transformers (plus numpy and sentencepieces), while Google's MADLAD-400 [2], in GGUF format [3], is also supported by llama.cpp.

You could also look into Argos Translate, or just use the same models as Firefox through kotki [4].

[0] https://huggingface.co/facebook/nllb-200-distilled-600M [1] https://huggingface.co/facebook/m2m100_418M [2] https://huggingface.co/google/madlad400-3b-mt [3] https://huggingface.co/models?other=base_model:quantized:goo... [4] https://github.com/kroketio/kotki

You can use bergamot ( https://github.com/browsermt/bergamot-translator ) with Mozilla's models ( https://github.com/mozilla/firefox-translations-models ).

Not the easiest, but easy enough (requires building).

I used these two projects to build an on-device translator for Android.

https://ai.meta.com/blog/nllb-200-high-quality-machine-trans... https://www.youtube.com/watch?v=AGgzRE3TlvU

ollama run gemma3:1b

https://ollama.com/library/gemma3

> support for over 140 languages

If you need to support several languages, you're going to have to have a zoo of models. Small ones just can't handle that many; and they especially aren't good enough for distribution, we only use them for understanding.