From [2024-07-29 Mon]
#dailyreport #rag #embeddings #encoder #vectordatabase
Today I have been choosing vector database for my little
project of RAG for cheepest PC.
The best open-source solutions that I consider:
SQLite-VEC, Redis, Clickhouse, ElasticSearch. (•́ᴗ•̀✿)
I am thinking now how to organize wide and massive amount
of information about PC configuration and files.
¯\_(ツ)_/¯
Main approaches to implement vector database:
- Approximate Nearest Neighbor (ANN) indexing
- locality-sensitive hashing (LSH), product quantization
(PQ), or hierarchical navigable small world graphs (HNSW)
- Tree-based Indexing - k-d trees or ball trees to
partition the vector space and facilitate efficient
nearest neighbor search.
- Compression and Quantization - Reducing the
dimensionality and precision of vector embeddings, sparce
vectors
- Parallelization and Distributed Processing
Mantra for: My body wants joy to be fried. My mind wants
to succeed. My soul don't need anything.
