Fast and private chat
Isa is a privacy-first AI chatbot with a focus on low latency voice and text communication. I designed and developed Isa to run completely locally with open source models. Isa uses Python with FastAPI for the back end, Ollama to serve the LLMs (Llama3 and Gemma2), Google's gTTS for text to speech, and open AI's Whisper for voice transcription. The frontend is fully custom and written in javascript.
Defining a system prompt
In the system prompt I give the LLM the basic guidelines for how our commands should be handled.
The intended response is a JSON object that contains the response, an optional command, and any preset suggested responses to be displayed as buttons.
AI language coach
Isa can also be prompted to converse in Spanish, French, and a handful of other languages. The fast voice based communication is great for learning a new language.