Vectorizer

Text Embedding Generator and Vector Database Integration

Built a high-performance text embedding tool in Rust that processes project files for AI retrieval. This tool integrates with Qdrant vector database for storage and retrieval of embeddings.

Technical Highlights

• Implemented async processing with Tokio and thread management for optimal performance
• Designed a robust CLI with clap that follows modern design patterns similar to Ruff and uv
• Used mpsc channels and oneshot patterns for efficient inter-thread communication
• Implemented sophisticated error handling with anyhow and custom error propagation
• Created command-line progress indicators with automatic refresh for better UX

Features

• Created a configuration system supporting both global and per-project settings
• Developed Neovim integration for automatic embedding of saved files
• Used the All-MiniLm-L(6/12)-V2 models for generating high-quality text embeddings

GitHub Repository