This is incredibly impressive work, especially at 19! The insight about LLMs being better at critiquing than producing translations is brilliant, and using that to build a hybrid system that outperforms individual models is exactly the kind of innovative thinking the field needs. Congrats on the company talk, well deserved recognition for solid research and execution!
Hi Alex, lovely work. I have just started working on a similar idea, to have level-aware language support in a podcast player (extending Anytime Player) to help the listener recognise words, phrases and meaning in native content.
I'm a non-techy person learning from scratch so it's slow going but as a language learner who selfishly wants more tools to support immersion learning if you want to support audio I think something podcast based would be super interesting.
Nice work and blog! You might consider crossposting some blog entries to LessWrong for more visibility of your research. People are very interested in new ideas on benchmarking.
That's a good idea, thanks for the suggestion. I'm thinking of writing one big blog article about everything I've found, loosely based on the talk, and I'll crosspost it there.
I've been doing a lot of experiments evaluating LLM translation performance, and I used what I learnt (that LLMs make mistakes, but different LLMs make different mistakes, and they're better at critiquing translations than producing them) to make a hybrid translator (https://nuenki.app/translator) that beats everything else.
And I was invited to do a talk about that to a company, which was really cool! I'm 19, doing this in my gap year before uni.