Suggestion: Integrating speech-to-text into AutoGPT?

[EntropyOrSloth]

It's great that AutoGPT can use 11ElevenLabs for text-to-speech. Has anyone been working on integrating speech-to-text such as into AutoGPT also, so that you can just tell it what you want verbally? I'm thinking lovo.ai or, maybe even better, OpenAI's own 'Whisper'?

[KonsoleLabs]

go for whisper

[sabatoleo]

This is a cool integration. Would love to contribute in case you like @EntropyOrSloth

[sabatoleo]

Cool, looking forward to see your results @jacksparrow124 :)

[jacksparrow124]

Hi! I've been working pretty hard recently on the project, and we have text to speech and speech to text answers for apple done, but we have encountered an issue with windows. I would love if some of you on macs could test it out and make sure its running smoothly. Please let me know if you encounter any issues or have any questions. In its current stage, it speaks, but it is still terminal run... Also, the features are NOTHING compared to auto-gpt 🙂.

https://github.com/jacksparrow124/HM-GPT

[Irongate2G]

I have a local version of Balabolka working using their balcon.exe file. It requires changes to the speech files though.

[jacksparrow124]

Currently we're only working with one thing (just to get started) but you could either fork the repo or copy the code... Feel free to copy the code back here so i can see what the changes are for future reference... 🙂

[TimHuey]

I'm running AutoGPT locally in a WSL VM. On the local:8000 webpage I told AutoGPT to take the file stored in the workspace named thelinemeeting.flac and use whisper to transcribe and then summarize the timestamped file. Store the transcription file as thelinemeetingtrans.txt and the summary as thelinemeetingsummary.txt in the workspace.

It said it would do it. but nothing has happened yet as far as I can tell. I'll let it go overnight and see what is in the working directory.