Repository navigation
Local RAG implementation for skills discovery when they grow past a certain limit #212
Description
How to use GitHub
- Please use the 👍 reaction to show that you are interested into the same feature.
- Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
- Subscribe to receive notifications on status change and new comments.
Feature request
Which Nextcloud Version are you currently using: 34.0.0
Is your feature request related to a problem? Please describe.
The skills' names and descriptions are extracted and put in the context of the model for the skills discovery by the model. For 20 skills, this could roughy take 5500 tokens, or 2750 tokens on average (see #170 (comment)).
But skills have many uses from documentation for process automation to document writing adapted from a style, not to mention they are combined with the admin provided skills too. See #170 (comment)
Describe the solution you'd like
One of the ideas could be to use local RAG to not inject everything but provide skill results to the model based on need.
The other would be to use subagents that will see all the skills (maybe could also be implemented for tools) and respond with the appropriate ones, which the main model can then call directly.
Describe alternatives you've considered
- Limiting the no. of skills injected
- Only injecting the skills names instead of names + short descriptions
Activity
TheBest-Name commentedon Jul 17, 2026
Okay I have a few more ideas one being when the docker container is first put up, have a config option to either have a local version or an external version and possibly a third version as well local will just download it separately so that way if you choose an external backend it doesn't bloat the docker container.
Okay the second option for an external would be to borrow the implementation for the context agent backend if you could coordinate with them and just kind of borrow their functionality then that would be one less thing you would have to run however it would be completely optional if you select that option for that back end
That would make it a lot more accessible and then you can choose what kind of model and everything else you want to run, I would assume.
Of course it would be the same thing for the small LLM idea you could just do a llm from the llm container or else you could choose a external LLM provider or use a local small model.
And this could be me being stupid but what if you could also think of using both again this may be me going a little crazy here but it might be something to assume using both that way the first model even if it doesn't explain it well the second model can then kind of clean up the explanation of what the tool's supposed to do before going into the vector search, I'm not sure how much that would help but it may be something.
Either way it would still be nice to have it be external because then you can control how many tokens get used and you can throttle or whatever else you need to do to control your CPU and GPU and credits if you use credits.
Thank you for your consideration!
kyteinsky commentedon Jul 20, 2026
yeah having an external endpoint/service sounds like a good idea too.
TheBest-Name commentedon Jul 20, 2026
I'm just curious because this thought just came to me, if you were to put it to an external backend would you also be able to use an external backend for the skills themselves to or to be able to hook external skills also up to that same backend?
I was just thinking because then you could have the next cloud skills but then you could also have skills that are from different other accounts kind of almost like an MCP server for skills.
You could link up and reuse your skills for multiple different AIs using the same backend.
Also for an external API, what API would you want to choose? Like an openAI? As you would put more support to it also would you be adding other APIs for different backends or is that usually something that they all share nowadays?
kyteinsky commentedon Aug 11, 2026
the skills are just files so can be uploaded to nextcloud easily, the external part should not necessary, also better from a security perspective.
and since these are normal files, different users can share skills among themselves and place them in the Skills folder for the agent to discover the skill.
not sure either but yeah if there is a widely used API, it would be a good idea to support it.
TheBest-Name commentedon Aug 11, 2026
Well when I meant an external backend I meant an external backend for the RAG. Like an external vector database.
If you did use an external vector database then when using other apps like context chat that also need a vector database if I remember right then you could either reuse that one or you could maybe use something more distributed or something of that nature.
Now that I think about it, I think I read an AI headline that said that there was a new skills thing that Google and others were putting together where there would be stuff to find in a file like a manifest I only barely remember it though.
Thank you for reading and taking into consideration my comments.
TheBest-Name commentedon Sep 4, 2026
This is more of a skills feature in necessarily just the rag but there should really be a tool that the MCP server can provide out to other tools that give access to the skills through the rag.
I just tried it recently and I'm not exactly sure if there is a tool right now that allows a user to access next cloud skills from beyond next cloud using the external MCP server.
marcelklehr commentedon Sep 16, 2026
The skills tools are exposed via MCP, but currently there is no list skills tool, indeed. Can you open a new issue for this?