andrew-w
Joined 27 karma
- andrew-wThanks for trying it out! character.ai has put their model behind a waitlist, so it's hard to compare. As far as I can tell, they don't appear to make any specific claims about speed or interactivity in their press release.
- thanks for trying us out!
- Just added a signup at the bottom of the technical report: https://lemonslice.com/live/technical-report
- glad to bring a little joy into the world :)
- It works with any style of character! Check out the embedded videos in our tech report. Peachy and the toilet are my favorite. https://lemonslice.com/live/technical-report
- Thanks! We think we can cut down the latency to <2s which should make it feel even more natural.
- Thanks! What kind of use case are you thinking about?
- It's something we are considering. What use cases do you have in mind?
- We've been very inspired by interactive character experiences powered by traditional VFX + puppetry (turtle talk with crush is a favorite). I think that sort of interactive entertainment will become more commonplace as tech like ours continues to improve. Looking forward to connecting!
- Thanks for the feedback. This is definitely a demo where every piece matters for maximizing the enjoyment factor. We spent the most effort on optimizing video quality and latency, but not a lot on tweaking the character prompts that go into the LLM. Turns out that matters a lot too.
- Not relying on facial keypoints means we can animate a wide range of non-humanoid characters. My favorite is talking to the Doge meme.
- One way this differs is in the model architecture. Our approach relies on a single pass of a diffusion transformer (DiT), whereas Live Portrait relies on intermediate representations and multiple distinct modules. Getting a DiT to be real-time was a big part of our work. Quoting the Live Portrait paper: "Diffusion-based portrait animation methods [...] are usually [too] computationally expensive." As you hinted at, we had to compromise on resolution to get there (this demo is 256x256), but we think that will improve over time.
- I spent about 2 hours recording videos with different characters. Of course, the one I made as a joke for myself and never intended to share was the most enjoyable to watch :)
- Just added as a public character :)
- Thanks for the feedback. Optimizing for speed meant we had fewer LLMs to choose from. OpenAI had surprisingly high variance in latency, which made it unusable for this demo. I think we could probably do a better job with prompting for some of the characters.
- We're back online! One of our cache systems ran out of memory. Oops. Agree on improved messaging.
- We've talked about doing something like that. Feels like it should work in theory.
- Cool use case! Thanks for sharing your thoughts.
- Makes sense, thank you!
- I agree, I think power users are happy to go to specific platforms, but APIs open up more use cases that can reach a broader audience. What kind of application would you use it for? We don't have specific plans at the moment, but are gauging interest.