Skip to content

TODO features #1

Open
Open

Description

@namtranase
  • More clear instructions to run the model
  • Add completion feature (python)
  • Add chat feature (python)
  • Add samples folders with several examples
  • Check the llama-cpp-python repo
  • Add batching infer feature (python)
  • Add quantization methods
  • Add server API support
  • Add more arguments to control the model (based on the libgemma interface)
  • Add the stream return for the completion function

Activity

  1. self-assigned this
    on Feb 23, 2024
  2. pinned this issue on Feb 23, 2024
  3. namtranase commented on Mar 3, 2024

    @namtranase
    OwnerAuthor

    @bachvudinh add features that you think we should do here

  4. namtranase commented on Mar 17, 2024

    @namtranase
    OwnerAuthor

    google/gemma.cpp#52
    I found the dicussion about the server implementation, would be awesome if we can make it in this repo also.

  5. namtranase commented on Mar 18, 2024

    @namtranase
    OwnerAuthor

    The server implementation from llama.cpp could be a rich resource for us to improve the examples:
    https://github.com/ggerganov/llama.cpp/blob/master/examples/server/server.cpp

  6. bachvudinh commented on Mar 18, 2024

    @bachvudinh
    Collaborator

    Thank you for the suggestion, @namtranase. I'll delve into their code to see if we can adapt their ideas for our project.

  7. namtranase commented on Mar 25, 2024

    @namtranase
    OwnerAuthor

    @bachvudinh @phamson02 any suggestions for new features/applications are welcome!

  8. namtranase commented on Apr 5, 2024

    @namtranase
    OwnerAuthor

    https://www.kaggle.com/competitions/data-assistants-with-gemma
    Could be useful for gemma-cpp-python/examples!

  9. omkar806 commented on Apr 19, 2024

    @omkar806

    add a feature like open ai api endpoint ...
    I would like to have feature like this most probably using fast api

  10. namtranase commented on Apr 22, 2024

    @namtranase
    OwnerAuthor

    Thank you @omkar806 for comment this feature, we will check on it and maybe make a new release with it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions