[thecoolwinter]

Description

Modifies TreeSitterClient to support doing work on a background thread when doing so is beneficial. This allows the editor to display content while languages load, scroll smoothly while highlight queries perform in the background, and on large documents perform edits asynchronously to keep UI responsive.

Edits and highlights are only queued for background work when:

• An edit takes too long (Constants.parserTimeout) and a parser times out.
• An edit or highlight is too large (Constants.maxSyncEditLength, Constants.maxSyncQueryLength)
• The document is large enough to make tree-sitter slow no matter what (Constants.maxSyncContentLength)
• There are async jobs already being performed.

Changes:

• Created a TreeSitterState object that handles managing the language layer tree (language layers renamed to LanguageLayer)
  • Handles setting and resetting languages
  • Manages injection layers.
• Added PthreadLock for efficient, thread-safe locking via a wrapped pthread_mutex.
• TreeSitterClient
  • Simplified TreeSitterClient.swift to only contain async methods, initialization and configuration, and HighlightProviding conformance code.
  • Created async versions of applyEdit and queryHighlightsFor
  • Added two queues: queuedEdits and queuedQueries to hold any enqueued edits and queries.
  • Added two locks:
    • stateLock: Must be acquired before performing any actions on the state object
    • queueLock: Must be acquired before modifying the previously mentioned queues.
  • Added private methods for starting and stopping the background task in runningTask:
    • beginTasksIfNeeded starts the background task if there is any outstanding work and there is no existing task.
    • determineNextJob dequeues a job(s) to perform in the background task. Handles job ordering and greedily dequeues highlight jobs as determined by Constants.simultaneousHighlightLimit
    • cancelAllRunningTasks cancels the running task and removes all queued jobs.

Related Issues

• ✨ Editing Performance #132 - This PR is half of this issue. The other half has to do with text editing performance which this PR does not change.

Checklist

• I read and understood the contributing guide as well as the code of conduct
• The issues this PR addresses are related to each other
• My changes generate no new warnings
• My code builds and runs on my machine
• My changes are all related to the related issue above
• I documented my code

Screenshots

Async language loading. Swift has a terrible initial loading time and this recording shows that the document now opens instantly but performs the initial work on a background thread. Note that this initial loading time is nearly nonexistent in subsequent document loads due to the queries being cached.

Previously, this would have blocked the main thread for the entire duration of the language load (up to 5s in my testing).

Screen.Recording.2023-06-03.at.1.33.01.AM.mov

Async highlighting means extremely long documents can load and highlight without affecting displaying and scrolling. This is still slightly "hitch"-y due to NSTextStorage performance when applying highlight data to the text view.

Screen.Recording.2023-06-03.at.1.33.34.AM.mov

[matthijseikelenboom]

Excellent work!

In the future, we might need to make dedicated documentation for this repo. It's getting quite complicated by just reading the code. So maybe adding diagrams or other visual media might be an idea.