Repository navigation
feature: embedding support #70
Description
Add support to embeddings to the API and the llama backend: https://github.com/ggerganov/llama.cpp/blob/e4422e299c10c7e84c8e987770ef40d31905a76b/llama.cpp#L2160
- go-llama.cpp
- go-gpt4all-j.cpp
- go-gpt2.cpp
Reacted by hugo4715, Giancarlo Santos, Sharon (Sean) Rolel, mylesdc, David LOIRET, Kent Lam, Kars Barendrecht, Lim Chee Kin and josephrocca
Activity
limcheekin commentedon May 3, 2023
Just curious to find out what is the use/purpose of embeddings above.
For the following use case of Retrieval Augmented Data QA:
https://blog.langchain.dev/tutorial-chatgpt-over-your-data/
Can't we use the following embedding models? I plan to use
gpt4all-jwith one of the following embeddings model.Please advise. Thank you.
mudler commentedon May 5, 2023
embeddings support has been merged to master. It is experimental and currently it's available only on
llama.cppbased models, so any feedback is more than welcome!To enable it you can set
embeddings: truein the model's YAML config filemudler commentedon May 6, 2023
I've published a sample using embeddings over here: https://github.com/go-skynet/LocalAI/tree/master/examples/query_data
mudler commentedon May 10, 2023
further optimizations in #222 - now embeddings can be used with bert on any model - and there is also a huge performance impact!
v4rm3t commentedon May 24, 2023
Hello! I am trying to run a gpt4all-j model for building a local chatbot. How can I use an embedding using BERT and implement it for chat completions endpoint?
Currently, I am running it on Mac Mini i7, 32gb RAM. I am planning to upgrade it to a higher resource(vRAM) cloud server in future. Is it possible to make a fast chatbot API using own document embeddings?
michelec1000 commentedon Jul 10, 2023
Thank you for the example! But it can't be included in the API? Currently I think you run those commands inside the container, right? Is there already the scenario that calling a certain path executes the query on the documents?