- [12. Can you help me write the pyproject.toml file for uv including PySide6, mlx-whisper, and Kokoro?](#12-can-you-help-me-write-the-pyprojecttoml-file-for-uv-including-pyside6-mlx-whisper-and-kokoro)
- [13. How to Initialize \& Install using uv](#13-how-to-initialize--install-using-uv)
- [13.0.1. Note on Python Version:](#1301-note-on-python-version)
- [14. Create basic PySide6 boilerplate for main.py, main\_window.py, and the tab modules](#14-create--basic-pyside6-boilerplate-for-mainpy-main_windowpy-and-the-tab-modules)
The project is a 100% offline, privacy-first Spanish speech practice assistant running natively on an Apple MacBook Pro M3. It creates a low-latency conversational feedback loop where you speak in Spanish, receive a generated audio response from a local LLM, and get automated feedback comparing your spoken pronunciation against the expected text.
Your speech is recorded through the MacBook Pro's built-in microphone array and transcribed to Spanish text using mlx-whisper.
## 2.2. Inference Stage:
Transcribed text (along with recent session history from SQLite) is sent via HTTP to a local llama.cpp server running Gemma 4.
## 2.3. Synthesis Stage:
Gemma’s response is passed to Kokoro-82M, generating high-quality Spanish speech audio offline.
## 2.4. Output & Evaluation Stage:
Audio plays directly through your MacBook speakers via sounddevice. Concurrently, a feedback engine compares your spoken text against target phrases to highlight pronunciation accuracy and tracks your progress in SQLite.
* Frameless/Standard QMainWindow: A clean modern window titled "Spanish Voice Practice AI."
* QTabWidget: The core navigation anchor.
* Tab 1 (Conversación): The main interaction zone shown above.
* Tab 2 (Control & Logs): (Not pictured) Would contain database viewing, LLM parameter sliders (temperature, max tokens), and system performance logs (latency checks).
2. Chat Display (QTextEdit)
* Displays a scrolling log of the conversation.
* Uses rich text formatting to differentiate between User, Assistant, and any system alerts.
* Transcripts from mlx-whisper are inserted here automatically.
3. Feedback Panel (QFrame)
* This panel appears (or updates) dynamically after the user finishes speaking and Whisper transcribes the audio.
* It displays the results generated by the PronunciationFeedbackEngine:
* Visual Score Bar: A colored bar/meter showing accuracy.
* Diff View: Uses HTML color coding (e.g., Red for omissions, Green for correct words) to visually compare Gemma's target text against the actual spoken transcription.
4. Input Area (QHBoxLayout)
* Combined Input: Primarily focused on voice, but includes a text fallback.
* Voice Trigger: A prominent visual button (RECORD (O)) and a keyboard shortcut (e.g., holding Spacebar) to initiate recording via the STTEngine.
* Send Button: A classic arrow icon for sending text input or manually submitting a recorded chunk.
5. Status Bar (QStatusBar)
* Provides real-time feedback on what the background systems are doing (e.g., "Whisper Transcribing...", "Gemma Generating...", "Kokoro Speaking...", "Ready").
* Displays the current active SQLite database session ID.
# 7. Code Structure
1. GUI Framework Alignment: You’ve migrated this project to PySide6 (as noted in your comment), but double-check that your UI imports and sub-classes across main.py, main_window.py, chat_tab.py, and control_tab.py are strictly using PySide6.QtWidgets rather than PyQt6.
2. Local macOS Optimization Opportunities: Since you are running locally on Apple Silicon (M3), using standard cloud/hybrid pipelines (edge-tts) or baseline CPU bindings might introduce unnecessary network latency or underutilize your Unified Memory.
│ └── control_tab.py # Tab 2: Logs, DB Inspector, and local LLM parameter controls
└── data/
└── conversation_history.db # Local SQLite database storage
```
# 8. Key Technical Recommendations for app/modules/
1. Speech-to-Text (stt_engine.py)
* Apple Silicon Edge: If you are currently using standard openai-whisper or whisper.cpp Python bindings, consider using mlx-whisper. It leverages Apple's MLX framework to run Whisper directly on the M3 Neural Engine/GPU with significantly lower latency and minimal CPU impact.
2. Text-to-Speech (tts_engine.py)
* Offline Native TTS: edge-tts is simple and fast, but it requires an active internet connection to stream Microsoft's cloud endpoints.
* If you want a 100% offline local setup for Spanish voice practice, look into Kokoro-82M (e.g., via kokoro or kokoro-onnx). It supports high-quality, natural Spanish pronunciation locally on macOS without needing cloud requests.
3. Database Layer (db.py)
* Make sure your db.py exposes thread-safe connection handling (or a repository pattern) so background Qt threads (QThread) handling STT/TTS don't crash SQLite when logging conversations while the UI is rendering tables.
* Python Packages: You have app/__init__.py, which turns app into a package. If you plan to import modules within chat_tab.py using absolute imports like from app.modules.llm_controller import LLMController, ensure you run your entry point (main.py) from the repository root (e.g., python -m app.main). Alternatively, consider adding an empty __init__.py inside app/modules and app/ui if you run into any relative import issues down the road.
* Data Directory: Keeping data/ at the root is ideal for local database storage (like SQLite/PostgreSQL dumps), audio cache files, or user session states.
* Docs & Assets: Placing screenshots or UI reference mockups in doc/images/ and markdown notes in doc/notes.md keeps the repository root tidy.
# 10. Recommended .gitignore
```zsh
# ==========================================
# macOS System Files
# ==========================================
.DS_Store
.AppleDouble
.LSOverride
Icon?
._*
# ==========================================
# Python & Bytecode
# ==========================================
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
# ==========================================
# Virtual Environments
# ==========================================
.venv/
venv/
ENV/
env/
env.bak/
venv.bak/
# ==========================================
# Application Data & Cache
# ==========================================
# Keep the data/ directory structure, but ignore dynamic files inside it
data/*
!data/.gitkeep
# Local log files and audio caches
*.log
*.wav
*.mp3
*.ogg
# ==========================================
# Environment & Secrets
# ==========================================
.env
.env.local
*.pem
# ==========================================
# IDEs & Editors
# ==========================================
.vscode/
.idea/
*.swp
*.swo
```
# 11. Tree Commands
To tell tree to display hidden files, pass the -a (all) flag:
```zsh
tree -a
```
## 11.1. Excluding noisy hidden folders
If you run tree -a, it will also crawl into the .git directory, which prints a massive, messy wall of internal Git objects.
To see your .gitkeep and .gitignore without cluttering the output with .git/, use the -I (ignore) flag:
```zsh
tree -a -I '.git'
```
```zsh
.
├── .gitignore
├── app
│ ├── __init__.py
│ ├── main.py
│ ├── modules
│ │ ├── feedback_engine.py
│ │ ├── llm_controller.py
│ │ ├── stt_engine.py
│ │ └── tts_engine.py
│ └── ui
│ ├── chat_tab.py
│ ├── control_tab.py
│ └── main_window.py
├── data
│ └── .gitkeep
└── doc
├── images
└── notes.md
```
```zsh
stephenlohning@Scotty 141_Spanish_Assistant % tree -a
.
├── .git
│ ├── HEAD
│ ├── config
│ ├── description
│ ├── hooks
│ │ ├── applypatch-msg.sample
│ │ ├── commit-msg.sample
│ │ ├── fsmonitor-watchman.sample
│ │ ├── post-update.sample
│ │ ├── pre-applypatch.sample
│ │ ├── pre-commit.sample
│ │ ├── pre-merge-commit.sample
│ │ ├── pre-push.sample
│ │ ├── pre-rebase.sample
│ │ ├── pre-receive.sample
│ │ ├── prepare-commit-msg.sample
│ │ ├── push-to-checkout.sample
│ │ ├── sendemail-validate.sample
│ │ └── update.sample
│ ├── info
│ │ └── exclude
│ ├── objects
│ │ ├── info
│ │ └── pack
│ └── refs
│ ├── heads
│ └── tags
├── app
│ ├── __init__.py
│ ├── main.py
│ ├── modules
│ │ ├── feedback_engine.py
│ │ ├── llm_controller.py
│ │ ├── stt_engine.py
│ │ └── tts_engine.py
│ └── ui
│ ├── chat_tab.py
│ ├── control_tab.py
│ └── main_window.py
├── data
│ └── .gitkeeper
└── doc
├── images
└── notes.md
16 directories, 29 files
stephenlohning@Scotty 141_Spanish_Assistant %
```
## 11.2. Setting up Git
* Setting __push.followTags = true__ changes default behavior so that whenever you run git push, Git automatically pushes annotated tags that point to commits reachable from the branch you are pushing.
## 11.3. Step-by-Step Commands
Run these commands in order in your terminal:
```zsh
# 1. Stage all your initial project files and .gitignore
git add .
# 2. Make your first commit (this resolves HEAD and creates your branch)
git commit -m "Initial project structure and config"
# 3. Ensure your active branch is named 'main'
git branch -M main
# 4. Now create your annotated tag on the new commit
git tag -a v0.1.0 -m "Initial project structure and config"
# 5. Push your branch and tag up to Forgejo
git push -u origin main
```
After running git commit, git branch will display * main, and your git tag command will succeed!
# 12. Can you help me write the pyproject.toml file for uv including PySide6, mlx-whisper, and Kokoro?
Here is a complete, production-ready pyproject.toml file tailored specifically for 141_Spanish_Assistant using uv.
It includes:
* PySide6 for the GUI.
* mlx-whisper for Apple Silicon GPU/Neural Engine accelerated speech-to-text.
warning: The `tool.uv.dev-dependencies` field (used in `pyproject.toml`) is deprecated and will be removed in a future release; use `dependency-groups.dev` instead
Using CPython 3.12.12 interpreter at: /opt/homebrew/opt/python@3.12/bin/python3.12
warning: The `tool.uv.dev-dependencies` field (used in `pyproject.toml`) is deprecated and will be removed in a future release; use `dependency-groups.dev` instead
× No solution found when resolving dependencies:
╰─▶ Because only kokoro-onnx<=0.5.0 is available and kokoro-onnx==0.5.0 depends on numpy>=2.0.2, we can conclude that kokoro-onnx>=0.5.0 depends on numpy>=2.0.2.
And because your project depends on kokoro-onnx>=0.5.0 and numpy>=1.24.0,<2.0.0,wecanconcludethatyourproject'srequirementsareunsatisfiable.
warning: The `tool.uv.dev-dependencies` field (used in `pyproject.toml`) is deprecated and will be removed in a future release; use `dependency-groups.dev` instead
Using CPython 3.12.12 interpreter at: /opt/homebrew/opt/python@3.12/bin/python3.12
warning: The `tool.uv.dev-dependencies` field (used in `pyproject.toml`) is deprecated and will be removed in a future release; use `dependency-groups.dev` instead
× No solution found when resolving dependencies:
╰─▶ Because only kokoro-onnx<=0.5.0 is available and kokoro-onnx==0.5.0 depends on numpy>=2.0.2, we can conclude that kokoro-onnx>=0.5.0 depends on numpy>=2.0.2.
And because your project depends on kokoro-onnx>=0.5.0 and numpy>=1.24.0,<2.0.0,wecanconcludethatyourproject'srequirementsareunsatisfiable.
warning: The `tool.uv.dev-dependencies` field (used in `pyproject.toml`) is deprecated and will be removed in a future release; use `dependency-groups.dev` instead
× No solution found when resolving dependencies:
╰─▶ Because only kokoro-onnx<=0.5.0 is available and kokoro-onnx==0.5.0 depends on numpy>=2.0.2, we can conclude that kokoro-onnx>=0.5.0 depends on numpy>=2.0.2.
And because your project depends on kokoro-onnx>=0.5.0 and numpy>=1.24.0,<2.0.0,wecanconcludethatyourproject'srequirementsareunsatisfiable.
warning: The `tool.uv.dev-dependencies` field (used in `pyproject.toml`) is deprecated and will be removed in a future release; use `dependency-groups.dev` instead
× No solution found when resolving dependencies:
╰─▶ Because only kokoro-onnx<=0.5.0 is available and kokoro-onnx==0.5.0 depends on numpy>=2.0.2, we can conclude that kokoro-onnx>=0.5.0 depends on numpy>=2.0.2.
And because your project depends on kokoro-onnx>=0.5.0 and numpy>=1.24.0,<2.0.0,wecanconcludethatyourproject'srequirementsareunsatisfiable.
warning: The `tool.uv.dev-dependencies` field (used in `pyproject.toml`) is deprecated and will be removed in a future release; use `dependency-groups.dev` instead
× No solution found when resolving dependencies:
╰─▶ Because only kokoro-onnx<=0.5.0 is available and kokoro-onnx==0.5.0 depends on numpy>=2.0.2, we can conclude that kokoro-onnx>=0.5.0 depends on numpy>=2.0.2.
And because your project depends on kokoro-onnx>=0.5.0 and numpy>=1.24.0,<2.0.0,wecanconcludethatyourproject'srequirementsareunsatisfiable.
There were two independent issues happening in your terminal output:
Resolution Failure (Fixed): The first run failed because of the numpy constraint conflict.
Build Failure (README.md Missing): The second run succeeded in resolving all 109 packages, but failed at the build step because pyproject.toml declared readme = "README.md", but no README.md file existed in your project root yet.
Additionally, because your Python code lives inside an app/ folder rather than a src/ or spanish_assistant/ package folder, Hatchling (the build backend) needs a small configuration hint to know what to include in editable mode (uv sync).
Step 1: Create a placeholder README.md
Run this in your terminal from the project root:
``zsh
touch README.md
echo "# 141 Spanish Assistant" > README.md
```
Step 2: Use the Updated pyproject.tomlUpdate your root pyproject.toml with this version. It configures [tool.hatch.build.targets.wheel] so Hatchling cleanly packages the app/ directory without throwing editable build errors:
```zsh
[project]
name = "spanish-assistant"
version = "0.1.0"
description = "Local, offline Spanish speech practice assistant running on Apple Silicon"
readme = "README.md"
requires-python = ">=3.10,<3.13"
authors = [
{ name = "Stephen", email = "stephen.lohning@oxnee.com" }
]
dependencies = [
# GUI Framework
"pyside6>=6.6.0",
# Local Speech-to-Text (Apple MLX Engine)
"mlx-whisper>=0.2.0",
# Local Text-to-Speech (Kokoro 82M via ONNX Runtime)
"kokoro-onnx>=0.5.0",
"soundfile>=0.12.1",
# Audio Recording & Playback
"sounddevice>=0.4.6",
"numpy>=2.0.2",
# HTTP Client for local llama.cpp server
"requests>=2.31.0",
# Pronunciation Feedback & Text Analytics
"editdistance>=0.8.0",
"jiwer>=3.0.0",
]
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
# Configures Hatchling to recognise the app/ directory
self.log_viewer.append("[SYSTEM] Esperando conexión con base de datos SQLite...")
log_layout.addWidget(self.log_viewer)
layout.addWidget(log_group)
```
## 14.5. Verification
Run the boilerplate using uv:
```zsh
uv run python -m app.main
```
This will launch a GUI window featuring tabbed navigation, text interaction, and async-ready buttons.
If you'd like to get a sense of how asynchronous Qt event loops handle UI reactivity without freezing, this tutorial provides a great hands-on walkthrough.
Async event loop integration with Qt
This video demonstrates how to run an infinite or asynchronous task inside a Qt application without locking up the user interface.
