Consideration for text mutation #5

One use case for the proofreader API is going to be proof reading as the user types in text. Which raises the question of should this API expose ways for the developer to indicate that new text was appended so that proof reading can be efficient while the user is typing.

It turns out, this is not actually necessary to be exposed at the API level. User agents can diff the text passed to the proofreader API and infer that new text is purely an append or that there is a large substring match between the old and new text. User agents can then optimize their inference engine to roll back state (rewind KV cache) and process just the new content.

Raising this issue, so as to hear if there are any other developer scenarios that require communicating changes in content to the proofreader API. If not, perhaps it would be good to include in the explainer this perf consideration for implementors to optimize subsequent calls to the proofreader API where only a suffix for the text differs.

Testing with the prompt API, it appears chrome already has this subtle optimization to not recompute KV caches when there are substring matches are found in subsequent prompts. Something we would not have discovered unless we thought of this use case.