Spaces:
Running
Can this work with 16 gb VRAM ?
need help
You can actually achieve the same "real time feel" with even less. I built a fully local voice assistant: https://github.com/acatovic/ova and it uses up half of my 12GB card and runs very well on a Macbook as well, with RTT responses typically in sub second range. The main thing here is that TTS models now are very good and very cheap - I use Kokoro in the above repo, but I have also tried Pocket TTS which is very good and very fast. If you want bit more emotion and can afford more flops, Chatterbox Turbo is excellent.
Similarly ASR models are also very cheap. So the last stumbling block is a good local LLM but fortunately those options now are cheap as well.
When I say "cheap" I mean memory footprint wise.
I want to build something with is full offline in model terms end to end and have basic we search and return capability like from google or something.
I see you have voice clone?
You can actually achieve the same "real time feel" with even less. I built a fully local voice assistant: https://github.com/acatovic/ova and it uses up half of my 12GB card and runs very well on a Macbook as well, with RTT responses typically in sub second range. The main thing here is that TTS models now are very good and very cheap - I use Kokoro in the above repo, but I have also tried Pocket TTS which is very good and very fast. If you want bit more emotion and can afford more flops, Chatterbox Turbo is excellent.
Similarly ASR models are also very cheap. So the last stumbling block is a good local LLM but fortunately those options now are cheap as well.
When I say "cheap" I mean memory footprint wise.
Hi! I've added the demo to the speech-to-speech repo, along with instructions to run it locally. The usage example should run with 16gb VRAM 🤗