- [x] More clear instructions to run the model - [x] Add completion feature (python) - [x] Add chat feature (python) - [x] Add samples folders with several examples - [x] Check the llama-cpp-python repo - [ ] Add batching infer feature (python) - [ ] Add quantization methods - [ ] Add server API support - [ ] Add more arguments to control the model (based on the libgemma interface) - [ ] Add the stream return for the completion function
Activity
namtranase commentedon Mar 3, 2024
@bachvudinh add features that you think we should do here
namtranase commentedon Mar 17, 2024
google/gemma.cpp#52
I found the dicussion about the server implementation, would be awesome if we can make it in this repo also.
namtranase commentedon Mar 18, 2024
The server implementation from llama.cpp could be a rich resource for us to improve the examples:
https://github.com/ggerganov/llama.cpp/blob/master/examples/server/server.cpp
bachvudinh commentedon Mar 18, 2024
Thank you for the suggestion, @namtranase. I'll delve into their code to see if we can adapt their ideas for our project.
namtranase commentedon Mar 25, 2024
@bachvudinh @phamson02 any suggestions for new features/applications are welcome!
namtranase commentedon Apr 5, 2024
https://www.kaggle.com/competitions/data-assistants-with-gemma
Could be useful for gemma-cpp-python/examples!
omkar806 commentedon Apr 19, 2024
add a feature like open ai api endpoint ...
I would like to have feature like this most probably using fast api
namtranase commentedon Apr 22, 2024
Thank you @omkar806 for comment this feature, we will check on it and maybe make a new release with it!