You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Raising priority as full-text search can greatly enhance the usefulness of a content-heavy website.
I don't mind full-text search is a separate page altogether and takes some time to load (i.e., if the full search index needs to be downloaded to the Browser first)
DocSearch will crawl your documentation website, push its content to an Algolia index, and allow you to add a dropdown search menu for your users to find relevant content in no time.
As discussed with @marvinchin today, Marvin is planning to explore using the Lunrjs library to implement a built-in full text search. This library is also used by MkDocs.
I'm increasing the priority because Algolia DocSearch is undergoing a major revamp and they haven't been able to provide the search support for our module websites this semester so far. The sooner we reduce reliance on third-party search the better.
If anyone would like to take up this issue, please feel free, I think this would be a rather fun thing to do. The library I mentioned above is more or less ready for use. I am currently just doing a fun infinite loop of "making it better and more marketable" but not actually doing any marketing 🤔😅
I came across several related alternatives as well in the course of doing this as well you can consider. All of them follow a CLI + wasm frontend architecture:
Stork
TinySearch
Pagefind - very recent. closest cousin of the library above. It implements the same idea of sharding index files. Currently, the main reason you might not want to use this is the downside of extra network request latency (it does not have the option of not-fragmenting index files), whereas my library by default does not shard index files but offers said option, the reason being to cater to the larger majority of use cases which do not need sharding. (includes 2103 site which generates just ~3MB index) This greatly improves search latency.
Please don't let my selling here from stop you from exercising your own judgement as well.
Feel free to come to your own reasoning, and choice, and post back here. I would love to hear your thoughts.
Some non exhaustive guidelines for implementation:
Consider how to map our current keywords feature to the new solutions
We currently don't delete files when they have been removed during live preview. This will likely be necessary for any file based indexing solution you choose to maintain the "state" of content accurately.
Hello I've been looking at this issue and one problem I've encountered is how contents in components that are hidden to the user during the initial render (e.g. Panels) are not included in the search results. This is because libraries like Pagefind indexes the content only after the HTML files have been built. This rendering problem is also faced by other plugins like dataTable (@Tim-Siu) and Mermaid (@yiwen101 @LamJiuFong)
This behaviour is also similar to the Algolia DocSearch we use now that automatically adds algolia-no-index to content hidden by MarkBind's Vue components, causing content hidden in panels to similarly not show up in search results.
With this in mind, I'm just making sure if the behaviour of the results of the full text search we want to implement should include content that are included in panels, or it is ok for them to not show up in the search results
This behaviour is also similar to the Algolia DocSearch we use now that automatically adds algolia-no-index to content hidden by MarkBind's Vue components, causing content hidden in panels to similarly not show up in search results.
With this in mind, I'm just making sure if the behaviour of the results of the full text search we want to implement should include content that are included in panels, or it is ok for them to not show up in the search results
@jingting1412 I think it is fine (even necessary) to omit content from collapsed panels. But we can index content from expanded-by-default panels, right?
Activity
damithc commentedon Dec 14, 2018
Raising priority as full-text search can greatly enhance the usefulness of a content-heavy website.
I don't mind full-text search is a separate page altogether and takes some time to load (i.e., if the full search index needs to be downloaded to the Browser first)
amad-person commentedon Jan 29, 2019
Are we open to integrating existing solutions for full-text search?
Docsearch (free, open-source):
damithc commentedon Jan 29, 2019
Ideally, we should have a decent built-in solution and the ability to integrate other third-party solutions.
yamgent commentedon Jan 30, 2019
As discussed with @marvinchin today, Marvin is planning to explore using the Lunrjs library to implement a built-in full text search. This library is also used by MkDocs.
marvinchin commentedon Mar 13, 2019
Are we still looking to have built-in full text search for V2? 😅 I'm not sure that I can finish it by the end of the semester.
damithc commentedon Mar 13, 2019
Good to have, but not necessary. Same for the FOUC problem. Both have a good-enough workaround but not a full-fledged solution.
4 remaining items
damithc commentedon Jan 15, 2022
I'm increasing the priority because Algolia DocSearch is undergoing a major revamp and they haven't been able to provide the search support for our module websites this semester so far. The sooner we reduce reliance on third-party search the better.
ang-zeyu commentedon Jan 15, 2023
If anyone would like to take up this issue, please feel free, I think this would be a rather fun thing to do. The library I mentioned above is more or less ready for use. I am currently just doing a fun infinite loop of "making it better and more marketable" but not actually doing any marketing 🤔😅
I came across several related alternatives as well in the course of doing this as well you can consider. All of them follow a CLI + wasm frontend architecture:
Please don't let my selling here from stop you from exercising your own judgement as well.
Feel free to come to your own reasoning, and choice, and post back here. I would love to hear your thoughts.
ang-zeyu commentedon Jan 15, 2023
Some non exhaustive guidelines for implementation:
jingting1412 commentedon Mar 20, 2024
Hello I've been looking at this issue and one problem I've encountered is how contents in components that are hidden to the user during the initial render (e.g. Panels) are not included in the search results. This is because libraries like Pagefind indexes the content only after the HTML files have been built. This rendering problem is also faced by other plugins like dataTable (@Tim-Siu) and Mermaid (@yiwen101 @LamJiuFong)
This behaviour is also similar to the Algolia DocSearch we use now that automatically adds
algolia-no-indexto content hidden by MarkBind's Vue components, causing content hidden in panels to similarly not show up in search results.With this in mind, I'm just making sure if the behaviour of the results of the full text search we want to implement should include content that are included in panels, or it is ok for them to not show up in the search results
damithc commentedon Mar 20, 2024
@jingting1412 I think it is fine (even necessary) to omit content from collapsed panels. But we can index content from expanded-by-default panels, right?