With all the hype going on, artificial intelligence (or rather machine learning (ML) and large language models (LLM)) is everywhere. Personally, I probably don't use ChatGPT (and similar alternatives) very often, but I do rely on things like GitHub Copilot (for smart autocompletion in VS Code) or Grammarly (for editing my blog article) every day.
I think we're still quite a few breakthroughs away from AGI, and current technology isn't enough to get us there (thankfully or not). That said, we're deep into the era of "AI-enhanced" apps, where the top apps may not have the best AI systems, but they integrate them in the best possible way.
That's why it was an interesting process, exploring OpenAI's API and trying to integrate it into Vrite's Rich Text Editor (RTE) - my open source headless CMS.
Extended WYSIWYG editor
For those unfamiliar, Vrite, in a nutshell, is a headless CMS for technical content, such as programming blogs or software documentation. It can be seen as two applications in one - a Kanban dashboard for content management and a WYSIWYG editor for writing, with additional development like embedded snippet editors and formatters Friendly features.
The latest major addition to Vrite is an early extension system that makes it easy to build integrations and extend Vrite's capabilities. To me, this seems like the perfect way to bring ChatGPT into the editor as an extension .
block operation
To be able to integrate ChatGPT into the editor using the extension system, a new API had to be introduced. I call it the Block Action API because it's specifically designed to add quick actions to the editor, operating on top-level content blocks such as paragraphs, headings, or images, like so:
Using the Block Actions API, an extension can read the JSON content of an active block and update it with the content in HTML, just like it does in the Vrite API (on the one hand, parsing JSON output is easier, and on the other hand, HTML ) is suitable for converting content into ).
From the UI side, block actions appear as buttons on the side of the actively selected block. They can directly invoke an action when clicked, or - like ChatGPT does - open a drop-down menu that prompts the user for more details.
The buttons had to be absolutely positioned, which required a custom TipTap extension and a deeper dig into the underlying ProseMirror (both libraries support the Vrite editor).
The process basically boils down to determining the position and size of a block node, given the choice of the entire top-level node or only its children ( source code ):
<span style="color:var(--syntax-text-color)"><span style="color:var(--syntax-text-color)"><code><span style="color:var(--syntax-comment-color)">// ...</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">BlockActionMenuPlugin</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-name-color)">Extension</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">create</span><span style="color:var(--syntax-text-color)">({</span>
<span style="color:var(--syntax-comment-color)">// ...</span>
<span style="color:var(--syntax-name-color)">onSelectionUpdate</span><span style="color:var(--syntax-text-color)">()</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-text-color)">{</span> <span style="color:var(--syntax-name-color)">selection</span> <span style="color:var(--syntax-text-color)">}</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-declaration-color)">this</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">editor</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">state</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">isTextSelection</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-name-color)">selection</span> <span style="color:var(--syntax-declaration-color)">instanceof</span> <span style="color:var(--syntax-name-color)">TextSelection</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">selectedNode</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-name-color)">selection</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">$from</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">node</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-literal-color)">1</span><span style="color:var(--syntax-text-color)">)</span> <span style="color:var(--syntax-error-color)">||</span> <span style="color:var(--syntax-name-color)">selection</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">$from</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">nodeAfter</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-declaration-color)">if</span> <span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-error-color)">!</span><span style="color:var(--syntax-name-color)">selectedNode</span><span style="color:var(--syntax-text-color)">)</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-name-color)">box</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">style</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">display</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-string-color)">none</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-declaration-color)">return</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-text-color)">}</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-text-color)">{</span> <span style="color:var(--syntax-name-color)">view</span> <span style="color:var(--syntax-text-color)">}</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-declaration-color)">this</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">editor</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">node</span> <span style="color:var(--syntax-error-color)">=</span>
<span style="color:var(--syntax-name-color)">view</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">nodeDOM</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">selection</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">$from</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">pos</span><span style="color:var(--syntax-text-color)">)</span> <span style="color:var(--syntax-error-color)">||</span>
<span style="color:var(--syntax-name-color)">view</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">nodeDOM</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">selection</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">$from</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">pos</span> <span style="color:var(--syntax-error-color)">-</span> <span style="color:var(--syntax-name-color)">selection</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">$from</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">parentOffset</span><span style="color:var(--syntax-text-color)">)</span> <span style="color:var(--syntax-error-color)">||</span>
<span style="color:var(--syntax-name-color)">view</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">domAtPos</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">selection</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">$from</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">pos</span><span style="color:var(--syntax-text-color)">)?.</span><span style="color:var(--syntax-name-color)">node</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-declaration-color)">if</span> <span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-error-color)">!</span><span style="color:var(--syntax-name-color)">node</span><span style="color:var(--syntax-text-color)">)</span> <span style="color:var(--syntax-declaration-color)">return</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">blockParent</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-name-color)">getBlockParent</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">node</span><span style="color:var(--syntax-text-color)">);</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">parentPos</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-text-color)">document</span>
<span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">getElementById</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-string-color)">pm-container</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-text-color)">)</span>
<span style="color:var(--syntax-text-color)">?.</span><span style="color:var(--syntax-name-color)">getBoundingClientRect</span><span style="color:var(--syntax-text-color)">();</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">childPos</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-name-color)">blockParent</span><span style="color:var(--syntax-text-color)">?.</span><span style="color:var(--syntax-name-color)">getBoundingClientRect</span><span style="color:var(--syntax-text-color)">();</span>
<span style="color:var(--syntax-declaration-color)">if</span> <span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-error-color)">!</span><span style="color:var(--syntax-name-color)">parentPos</span> <span style="color:var(--syntax-error-color)">||</span> <span style="color:var(--syntax-error-color)">!</span><span style="color:var(--syntax-name-color)">childPos</span><span style="color:var(--syntax-text-color)">)</span> <span style="color:var(--syntax-declaration-color)">return</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">relativePos</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-name-color)">top</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-name-color)">childPos</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">top</span> <span style="color:var(--syntax-error-color)">-</span> <span style="color:var(--syntax-name-color)">parentPos</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">top</span><span style="color:var(--syntax-text-color)">,</span>
<span style="color:var(--syntax-name-color)">right</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-name-color)">childPos</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">right</span> <span style="color:var(--syntax-error-color)">-</span> <span style="color:var(--syntax-name-color)">parentPos</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">right</span><span style="color:var(--syntax-text-color)">,</span>
<span style="color:var(--syntax-name-color)">bottom</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-name-color)">childPos</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">bottom</span> <span style="color:var(--syntax-error-color)">-</span> <span style="color:var(--syntax-name-color)">parentPos</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">bottom</span><span style="color:var(--syntax-text-color)">,</span>
<span style="color:var(--syntax-name-color)">left</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-name-color)">childPos</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">left</span> <span style="color:var(--syntax-error-color)">-</span> <span style="color:var(--syntax-name-color)">parentPos</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">left</span><span style="color:var(--syntax-text-color)">,</span>
<span style="color:var(--syntax-text-color)">};</span>
<span style="color:var(--syntax-declaration-color)">let</span> <span style="color:var(--syntax-name-color)">rangeFrom</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-name-color)">selection</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">$from</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">pos</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-declaration-color)">let</span> <span style="color:var(--syntax-name-color)">rangeTo</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-name-color)">selection</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">$to</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">pos</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-name-color)">box</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">style</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">top</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-string-color)">`</span><span style="color:var(--syntax-text-color)">${</span><span style="color:var(--syntax-name-color)">relativePos</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">top</span><span style="color:var(--syntax-text-color)">}</span><span style="color:var(--syntax-string-color)">px`</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-name-color)">box</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">style</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">left</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-string-color)">`</span><span style="color:var(--syntax-text-color)">${</span><span style="color:var(--syntax-name-color)">relativePos</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">left</span> <span style="color:var(--syntax-error-color)">+</span> <span style="color:var(--syntax-name-color)">parentPos</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">width</span><span style="color:var(--syntax-text-color)">}</span><span style="color:var(--syntax-string-color)">px`</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-name-color)">box</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">style</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">display</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-string-color)">block</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-declaration-color)">if</span> <span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">isTextSelection</span><span style="color:var(--syntax-text-color)">)</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-declaration-color)">try</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">p</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-name-color)">findParentAtDepth</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">selection</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">$from</span><span style="color:var(--syntax-text-color)">,</span> <span style="color:var(--syntax-literal-color)">1</span><span style="color:var(--syntax-text-color)">);</span>
<span style="color:var(--syntax-name-color)">rangeFrom</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-name-color)">p</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">start</span> <span style="color:var(--syntax-error-color)">-</span> <span style="color:var(--syntax-literal-color)">1</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-name-color)">rangeTo</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-name-color)">p</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">start</span> <span style="color:var(--syntax-error-color)">+</span> <span style="color:var(--syntax-name-color)">p</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">node</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">nodeSize</span> <span style="color:var(--syntax-error-color)">-</span> <span style="color:var(--syntax-literal-color)">1</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-text-color)">}</span> <span style="color:var(--syntax-declaration-color)">catch</span> <span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">e</span><span style="color:var(--syntax-text-color)">)</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-name-color)">box</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">style</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">display</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-string-color)">none</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-text-color)">}</span>
<span style="color:var(--syntax-text-color)">}</span>
<span style="color:var(--syntax-comment-color)">// ...</span>
<span style="color:var(--syntax-text-color)">},</span>
<span style="color:var(--syntax-text-color)">});</span>
</code></span></span>Replace editor content
The second part deals with the actual process of replacing block content with newly served content. The trickiest thing here is getting the correct range (start and end position in ProseMirror) of the block nodes. This is required to correctly replace ranges with TipTap's commands.
If you look closely at the last code snippet — its code already exists. On each selection update, the extent of the block and the block action UI positioning are updated.
It's much easier to actually replace the range with the new content. All it has to do is convert HTML to schema-following JSON and involve the appropriate commands ( source code ):
<span style="color:var(--syntax-text-color)"><span style="color:var(--syntax-text-color)"><code><span style="color:var(--syntax-comment-color)">// ...</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">replaceContent</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">content</span><span style="color:var(--syntax-text-color)">)</span> <span style="color:var(--syntax-error-color)">=></span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-name-color)">unlock</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">clear</span><span style="color:var(--syntax-text-color)">();</span>
<span style="color:var(--syntax-name-color)">setLocked</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-declaration-color)">true</span><span style="color:var(--syntax-text-color)">);</span>
<span style="color:var(--syntax-declaration-color)">if</span> <span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">range</span><span style="color:var(--syntax-text-color)">())</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-declaration-color)">let</span> <span style="color:var(--syntax-name-color)">size</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-literal-color)">0</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">nodeOrFragment</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-name-color)">createNodeFromContent</span><span style="color:var(--syntax-text-color)">(</span>
<span style="color:var(--syntax-name-color)">content</span><span style="color:var(--syntax-text-color)">,</span>
<span style="color:var(--syntax-name-color)">props</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">state</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">editor</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">schema</span>
<span style="color:var(--syntax-text-color)">);</span>
<span style="color:var(--syntax-declaration-color)">if</span> <span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">nodeOrFragment</span> <span style="color:var(--syntax-declaration-color)">instanceof</span> <span style="color:var(--syntax-name-color)">PMNode</span><span style="color:var(--syntax-text-color)">)</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-name-color)">size</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-name-color)">nodeOrFragment</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">nodeSize</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-text-color)">}</span> <span style="color:var(--syntax-declaration-color)">else</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-name-color)">size</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-name-color)">nodeOrFragment</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">size</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-text-color)">}</span>
<span style="color:var(--syntax-name-color)">props</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">state</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">editor</span>
<span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">chain</span><span style="color:var(--syntax-text-color)">()</span>
<span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">focus</span><span style="color:var(--syntax-text-color)">()</span>
<span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">insertContentAt</span><span style="color:var(--syntax-text-color)">(</span>
<span style="color:var(--syntax-name-color)">range</span><span style="color:var(--syntax-text-color)">()</span><span style="color:var(--syntax-error-color)">!</span><span style="color:var(--syntax-text-color)">,</span>
<span style="color:var(--syntax-name-color)">generateJSON</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">content</span><span style="color:var(--syntax-text-color)">,</span> <span style="color:var(--syntax-name-color)">props</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">state</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">editor</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">extensionManager</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">extensions</span><span style="color:var(--syntax-text-color)">)</span>
<span style="color:var(--syntax-text-color)">)</span>
<span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">scrollIntoView</span><span style="color:var(--syntax-text-color)">()</span>
<span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">focus</span><span style="color:var(--syntax-text-color)">()</span>
<span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">run</span><span style="color:var(--syntax-text-color)">();</span>
<span style="color:var(--syntax-name-color)">setRange</span><span style="color:var(--syntax-text-color)">({</span> <span style="color:var(--syntax-name-color)">from</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-name-color)">range</span><span style="color:var(--syntax-text-color)">()</span><span style="color:var(--syntax-error-color)">!</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-declaration-color)">from</span><span style="color:var(--syntax-text-color)">,</span> <span style="color:var(--syntax-name-color)">to</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-name-color)">range</span><span style="color:var(--syntax-text-color)">()</span><span style="color:var(--syntax-error-color)">!</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-declaration-color)">from</span> <span style="color:var(--syntax-error-color)">+</span> <span style="color:var(--syntax-name-color)">size</span> <span style="color:var(--syntax-error-color)">-</span> <span style="color:var(--syntax-literal-color)">1</span> <span style="color:var(--syntax-text-color)">});</span>
<span style="color:var(--syntax-name-color)">computeDropdownPosition</span><span style="color:var(--syntax-text-color)">()();</span>
<span style="color:var(--syntax-text-color)">}</span>
<span style="color:var(--syntax-name-color)">unlock</span><span style="color:var(--syntax-text-color)">();</span>
<span style="color:var(--syntax-text-color)">};</span>
<span style="color:var(--syntax-comment-color)">// ...</span>
</code></span></span>replaceContent()The function can then be called remotely from the extended sandbox by sending the appropriate message to the main frame.
To enable use cases like ChatGPT integration, where the content will be updated (i.e. replaced) multiple times in a row before the process completes, the function also locks the editor and updates the scope for the short time the function is called, and the UI positioning for each call . But why is this necessary?
API integration with OpenAI
The process of integrating OpenAI's API is well documented in its official documentation . Given that the official SDK is provided, the whole process can be done with just a few lines of code:
<span style="color:var(--syntax-text-color)"><span style="color:var(--syntax-text-color)"><code><span style="color:var(--syntax-declaration-color)">async</span> <span style="color:var(--syntax-text-color)">({</span> <span style="color:var(--syntax-name-color)">ctx</span><span style="color:var(--syntax-text-color)">,</span> <span style="color:var(--syntax-name-color)">input</span> <span style="color:var(--syntax-text-color)">})</span> <span style="color:var(--syntax-error-color)">=></span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">configuration</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-declaration-color)">new</span> <span style="color:var(--syntax-name-color)">Configuration</span><span style="color:var(--syntax-text-color)">({</span>
<span style="color:var(--syntax-name-color)">apiKey</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-name-color)">ctx</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">fastify</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">config</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">OPENAI_API_KEY</span><span style="color:var(--syntax-text-color)">,</span>
<span style="color:var(--syntax-name-color)">organization</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-name-color)">ctx</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">fastify</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">config</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">OPENAI_ORGANIZATION</span><span style="color:var(--syntax-text-color)">,</span>
<span style="color:var(--syntax-text-color)">});</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">openai</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-declaration-color)">new</span> <span style="color:var(--syntax-name-color)">OpenAIApi</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">configuration</span><span style="color:var(--syntax-text-color)">);</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">response</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-declaration-color)">await</span> <span style="color:var(--syntax-name-color)">openai</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">createChatCompletion</span><span style="color:var(--syntax-text-color)">({</span>
<span style="color:var(--syntax-name-color)">model</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-string-color)">gpt-3.5-turbo</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-text-color)">,</span>
<span style="color:var(--syntax-name-color)">messages</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-text-color)">[{</span> <span style="color:var(--syntax-name-color)">role</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-string-color)">user</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-text-color)">,</span> <span style="color:var(--syntax-name-color)">content</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-name-color)">input</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">prompt</span> <span style="color:var(--syntax-text-color)">}],</span>
<span style="color:var(--syntax-text-color)">});</span>
<span style="color:var(--syntax-text-color)">};</span>
</code></span></span>Now, all of this is true, but only if you're willing to wait typically +20 seconds for a single response! That's a lot for a single request. max_tokensFrom changing server location to optimizing requests by throttling or customizing other parameters, nothing works. It all boils down to the fact that current LLMs (at least at the GPT-3 level) are still pretty slow.
Having said that, the ChatGPT app still manages to be considered fairly fast and responsive. This is thanks to streaming and the use of Server Sent Events (SSE).
Stream Chat GPT Responses
OpenAI's API's chat completion and other endpoints support streaming via server-sent events, essentially keeping an open connection over which new tokens are sent as soon as they become available.
Unfortunately the official Node.js SDK doesn't support streaming and requires you to use workarounds to get it working, resulting in more code, just for connecting to the API ( source code ):
<span style="color:var(--syntax-text-color)"><span style="color:var(--syntax-text-color)"><code><span style="color:var(--syntax-declaration-color)">async</span> <span style="color:var(--syntax-text-color)">({</span> <span style="color:var(--syntax-name-color)">ctx</span><span style="color:var(--syntax-text-color)">,</span> <span style="color:var(--syntax-name-color)">input</span> <span style="color:var(--syntax-text-color)">})</span> <span style="color:var(--syntax-error-color)">=></span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">configuration</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-declaration-color)">new</span> <span style="color:var(--syntax-name-color)">Configuration</span><span style="color:var(--syntax-text-color)">({</span>
<span style="color:var(--syntax-name-color)">apiKey</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-name-color)">ctx</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">fastify</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">config</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">OPENAI_API_KEY</span><span style="color:var(--syntax-text-color)">,</span>
<span style="color:var(--syntax-name-color)">organization</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-name-color)">ctx</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">fastify</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">config</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">OPENAI_ORGANIZATION</span><span style="color:var(--syntax-text-color)">,</span>
<span style="color:var(--syntax-text-color)">});</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">openai</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-declaration-color)">new</span> <span style="color:var(--syntax-name-color)">OpenAIApi</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">configuration</span><span style="color:var(--syntax-text-color)">);</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">response</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-declaration-color)">await</span> <span style="color:var(--syntax-name-color)">openai</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">createChatCompletion</span><span style="color:var(--syntax-text-color)">(</span>
<span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-name-color)">model</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-string-color)">gpt-3.5-turbo</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-text-color)">,</span>
<span style="color:var(--syntax-name-color)">stream</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-declaration-color)">true</span><span style="color:var(--syntax-text-color)">,</span>
<span style="color:var(--syntax-name-color)">messages</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-text-color)">[{</span> <span style="color:var(--syntax-name-color)">role</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-string-color)">user</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-text-color)">,</span> <span style="color:var(--syntax-name-color)">content</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-name-color)">input</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">prompt</span> <span style="color:var(--syntax-text-color)">}],</span>
<span style="color:var(--syntax-text-color)">},</span>
<span style="color:var(--syntax-text-color)">{</span> <span style="color:var(--syntax-name-color)">responseType</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-string-color)">stream</span><span style="color:var(--syntax-string-color)">'</span> <span style="color:var(--syntax-text-color)">}</span>
<span style="color:var(--syntax-text-color)">);</span>
<span style="color:var(--syntax-name-color)">ctx</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">res</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">raw</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">writeHead</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-literal-color)">200</span><span style="color:var(--syntax-text-color)">,</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-text-color)">...</span><span style="color:var(--syntax-name-color)">ctx</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">res</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">getHeaders</span><span style="color:var(--syntax-text-color)">(),</span>
<span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-string-color)">content-type</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-string-color)">text/event-stream</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-text-color)">,</span>
<span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-string-color)">cache-control</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-string-color)">no-cache</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-text-color)">,</span>
<span style="color:var(--syntax-name-color)">connection</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-string-color)">keep-alive</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-text-color)">,</span>
<span style="color:var(--syntax-text-color)">});</span>
<span style="color:var(--syntax-declaration-color)">return</span> <span style="color:var(--syntax-declaration-color)">new</span> <span style="color:var(--syntax-text-color)">Promise</span><span style="color:var(--syntax-error-color)"><</span><span style="color:var(--syntax-declaration-color)">void</span><span style="color:var(--syntax-error-color)">></span><span style="color:var(--syntax-text-color)">((</span><span style="color:var(--syntax-name-color)">resolve</span><span style="color:var(--syntax-text-color)">)</span> <span style="color:var(--syntax-error-color)">=></span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">responseData</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-name-color)">response</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">data</span> <span style="color:var(--syntax-declaration-color)">as</span> <span style="color:var(--syntax-name-color)">unknown</span> <span style="color:var(--syntax-declaration-color)">as</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-name-color)">on</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">event</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-declaration-color)">string</span><span style="color:var(--syntax-text-color)">,</span> <span style="color:var(--syntax-name-color)">data</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">data</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-declaration-color)">string</span><span style="color:var(--syntax-text-color)">)</span> <span style="color:var(--syntax-error-color)">=></span> <span style="color:var(--syntax-declaration-color)">void</span><span style="color:var(--syntax-text-color)">):</span> <span style="color:var(--syntax-declaration-color)">void</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-text-color)">};</span>
<span style="color:var(--syntax-name-color)">responseData</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">on</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-string-color)">data</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-text-color)">,</span> <span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">data</span><span style="color:var(--syntax-text-color)">)</span> <span style="color:var(--syntax-error-color)">=></span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">lines</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-name-color)">data</span>
<span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">toString</span><span style="color:var(--syntax-text-color)">()</span>
<span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">split</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-literal-color)">\n</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-text-color)">)</span>
<span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">filter</span><span style="color:var(--syntax-text-color)">((</span><span style="color:var(--syntax-name-color)">line</span><span style="color:var(--syntax-text-color)">)</span> <span style="color:var(--syntax-error-color)">=></span> <span style="color:var(--syntax-name-color)">line</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">trim</span><span style="color:var(--syntax-text-color)">()</span> <span style="color:var(--syntax-error-color)">!==</span> <span style="color:var(--syntax-string-color)">''</span><span style="color:var(--syntax-text-color)">);</span>
<span style="color:var(--syntax-declaration-color)">for</span> <span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">line</span> <span style="color:var(--syntax-declaration-color)">of</span> <span style="color:var(--syntax-name-color)">lines</span><span style="color:var(--syntax-text-color)">)</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">message</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-name-color)">line</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">replace</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-string-color)">/^data: /</span><span style="color:var(--syntax-text-color)">,</span> <span style="color:var(--syntax-string-color)">''</span><span style="color:var(--syntax-text-color)">);</span>
<span style="color:var(--syntax-declaration-color)">if</span> <span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">message</span> <span style="color:var(--syntax-error-color)">===</span> <span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-string-color)">[DONE]</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-text-color)">)</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-name-color)">ctx</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">res</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">raw</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">end</span><span style="color:var(--syntax-text-color)">();</span>
<span style="color:var(--syntax-name-color)">resolve</span><span style="color:var(--syntax-text-color)">();</span>
<span style="color:var(--syntax-declaration-color)">continue</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-text-color)">}</span>
<span style="color:var(--syntax-declaration-color)">try</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">parsed</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-name-color)">JSON</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">parse</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">message</span><span style="color:var(--syntax-text-color)">);</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">content</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-name-color)">parsed</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">choices</span><span style="color:var(--syntax-text-color)">[</span><span style="color:var(--syntax-literal-color)">0</span><span style="color:var(--syntax-text-color)">].</span><span style="color:var(--syntax-name-color)">delta</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">content</span> <span style="color:var(--syntax-error-color)">||</span> <span style="color:var(--syntax-string-color)">''</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-declaration-color)">if</span> <span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">content</span><span style="color:var(--syntax-text-color)">)</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-name-color)">ctx</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">res</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">raw</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">write</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-string-color)">`data: </span><span style="color:var(--syntax-text-color)">${</span><span style="color:var(--syntax-text-color)">encodeURIComponent</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">content</span><span style="color:var(--syntax-text-color)">)}</span><span style="color:var(--syntax-string-color)">`</span><span style="color:var(--syntax-text-color)">);</span>
<span style="color:var(--syntax-name-color)">ctx</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">res</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">raw</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">write</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-literal-color)">\n\n</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-text-color)">);</span>
<span style="color:var(--syntax-text-color)">}</span>
<span style="color:var(--syntax-text-color)">}</span> <span style="color:var(--syntax-declaration-color)">catch</span> <span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">error</span><span style="color:var(--syntax-text-color)">)</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-name-color)">console</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">error</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-string-color)">Could not JSON parse stream message</span><span style="color:var(--syntax-string-color)">'</span><span style="color:var(--syntax-text-color)">,</span> <span style="color:var(--syntax-name-color)">message</span><span style="color:var(--syntax-text-color)">,</span> <span style="color:var(--syntax-name-color)">error</span><span style="color:var(--syntax-text-color)">);</span>
<span style="color:var(--syntax-text-color)">}</span>
<span style="color:var(--syntax-text-color)">}</span>
<span style="color:var(--syntax-text-color)">});</span>
<span style="color:var(--syntax-text-color)">});</span>
<span style="color:var(--syntax-text-color)">};</span>
</code></span></span>On top of that, you also have to support streaming between your API server and web clients, for Vrite this means integrating SSE with Fastify and tRPC . Not the cleanest solution, but still very stable.
From the front end (extending the sandbox to be precise), a connection to the new streaming endpoint has to be established and incoming data handled correctly ( source code ):
<span style="color:var(--syntax-text-color)"><span style="color:var(--syntax-text-color)"><code><span style="color:var(--syntax-declaration-color)">import</span> <span style="color:var(--syntax-text-color)">{</span> <span style="color:var(--syntax-name-color)">fetchEventSource</span> <span style="color:var(--syntax-text-color)">}</span> <span style="color:var(--syntax-declaration-color)">from</span> <span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-string-color)">@microsoft/fetch-event-source</span><span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-comment-color)">// ...</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">generate</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-declaration-color)">async</span> <span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">context</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-name-color)">ExtensionBlockActionViewContext</span><span style="color:var(--syntax-text-color)">):</span> <span style="color:var(--syntax-text-color)">Promise</span><span style="color:var(--syntax-error-color)"><</span><span style="color:var(--syntax-declaration-color)">void</span><span style="color:var(--syntax-error-color)">></span> <span style="color:var(--syntax-error-color)">=></span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">includeContext</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-name-color)">context</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">temp</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">includeContext</span> <span style="color:var(--syntax-declaration-color)">as</span> <span style="color:var(--syntax-name-color)">boolean</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">prompt</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-name-color)">context</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">temp</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">prompt</span> <span style="color:var(--syntax-declaration-color)">as</span> <span style="color:var(--syntax-declaration-color)">string</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-declaration-color)">let</span> <span style="color:var(--syntax-name-color)">content</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-string-color)">""</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-name-color)">context</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">setTemp</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-string-color)">$loading</span><span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-text-color)">,</span> <span style="color:var(--syntax-declaration-color)">true</span><span style="color:var(--syntax-text-color)">);</span>
<span style="color:var(--syntax-text-color)">window</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">currentRequestController</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-declaration-color)">new</span> <span style="color:var(--syntax-name-color)">AbortController</span><span style="color:var(--syntax-text-color)">();</span>
<span style="color:var(--syntax-text-color)">window</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">currentRequestController</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">signal</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">addEventListener</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-string-color)">abort</span><span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-text-color)">,</span> <span style="color:var(--syntax-text-color)">()</span> <span style="color:var(--syntax-error-color)">=></span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-name-color)">context</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">setTemp</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-string-color)">$loading</span><span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-text-color)">,</span> <span style="color:var(--syntax-declaration-color)">false</span><span style="color:var(--syntax-text-color)">);</span>
<span style="color:var(--syntax-name-color)">context</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">refreshContent</span><span style="color:var(--syntax-text-color)">();</span>
<span style="color:var(--syntax-text-color)">});</span>
<span style="color:var(--syntax-declaration-color)">await</span> <span style="color:var(--syntax-name-color)">fetchEventSource</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-string-color)">https://extensions.vrite.io/gpt</span><span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-text-color)">,</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-name-color)">method</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-string-color)">POST</span><span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-text-color)">,</span>
<span style="color:var(--syntax-name-color)">headers</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-string-color)">Content-Type</span><span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-string-color)">application/json</span><span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-text-color)">,</span>
<span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-string-color)">Accept</span><span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-string-color)">text/event-stream</span><span style="color:var(--syntax-string-color)">"</span>
<span style="color:var(--syntax-text-color)">},</span>
<span style="color:var(--syntax-name-color)">body</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-name-color)">JSON</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">stringify</span><span style="color:var(--syntax-text-color)">({</span>
<span style="color:var(--syntax-name-color)">prompt</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-name-color)">includeContext</span> <span style="color:var(--syntax-text-color)">?</span> <span style="color:var(--syntax-string-color)">`"</span><span style="color:var(--syntax-text-color)">${</span><span style="color:var(--syntax-name-color)">gfmTransformer</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">context</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">content</span><span style="color:var(--syntax-text-color)">)}</span><span style="color:var(--syntax-string-color)">"\n\n</span><span style="color:var(--syntax-text-color)">${</span><span style="color:var(--syntax-name-color)">prompt</span><span style="color:var(--syntax-text-color)">}</span><span style="color:var(--syntax-string-color)">`</span> <span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-name-color)">prompt</span>
<span style="color:var(--syntax-text-color)">}),</span>
<span style="color:var(--syntax-name-color)">signal</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-text-color)">window</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">currentRequestController</span><span style="color:var(--syntax-text-color)">?.</span><span style="color:var(--syntax-name-color)">signal</span><span style="color:var(--syntax-text-color)">,</span>
<span style="color:var(--syntax-declaration-color)">async</span> <span style="color:var(--syntax-name-color)">onopen</span><span style="color:var(--syntax-text-color)">()</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-declaration-color)">return</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-text-color)">},</span>
<span style="color:var(--syntax-name-color)">onerror</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">error</span><span style="color:var(--syntax-text-color)">)</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-name-color)">context</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">setTemp</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-string-color)">$loading</span><span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-text-color)">,</span> <span style="color:var(--syntax-declaration-color)">false</span><span style="color:var(--syntax-text-color)">);</span>
<span style="color:var(--syntax-name-color)">context</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">refreshContent</span><span style="color:var(--syntax-text-color)">();</span>
<span style="color:var(--syntax-name-color)">context</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">notify</span><span style="color:var(--syntax-text-color)">({</span> <span style="color:var(--syntax-name-color)">text</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-string-color)">Error while generating content</span><span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-text-color)">,</span> <span style="color:var(--syntax-name-color)">type</span><span style="color:var(--syntax-text-color)">:</span> <span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-string-color)">error</span><span style="color:var(--syntax-string-color)">"</span> <span style="color:var(--syntax-text-color)">});</span>
<span style="color:var(--syntax-declaration-color)">throw</span> <span style="color:var(--syntax-name-color)">error</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-text-color)">},</span>
<span style="color:var(--syntax-name-color)">onmessage</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">event</span><span style="color:var(--syntax-text-color)">)</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-declaration-color)">const</span> <span style="color:var(--syntax-name-color)">partOfContent</span> <span style="color:var(--syntax-error-color)">=</span> <span style="color:var(--syntax-text-color)">decodeURIComponent</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">event</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">data</span><span style="color:var(--syntax-text-color)">);</span>
<span style="color:var(--syntax-name-color)">content</span> <span style="color:var(--syntax-error-color)">+=</span> <span style="color:var(--syntax-name-color)">partOfContent</span><span style="color:var(--syntax-text-color)">;</span>
<span style="color:var(--syntax-name-color)">context</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">replaceContent</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">marked</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">parse</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-name-color)">content</span><span style="color:var(--syntax-text-color)">));</span>
<span style="color:var(--syntax-text-color)">},</span>
<span style="color:var(--syntax-name-color)">onclose</span><span style="color:var(--syntax-text-color)">()</span> <span style="color:var(--syntax-text-color)">{</span>
<span style="color:var(--syntax-name-color)">context</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">setTemp</span><span style="color:var(--syntax-text-color)">(</span><span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-string-color)">$loading</span><span style="color:var(--syntax-string-color)">"</span><span style="color:var(--syntax-text-color)">,</span> <span style="color:var(--syntax-declaration-color)">false</span><span style="color:var(--syntax-text-color)">);</span>
<span style="color:var(--syntax-name-color)">context</span><span style="color:var(--syntax-text-color)">.</span><span style="color:var(--syntax-name-color)">refreshContent</span><span style="color:var(--syntax-text-color)">();</span>
<span style="color:var(--syntax-text-color)">}</span>
<span style="color:var(--syntax-text-color)">});</span>
<span style="color:var(--syntax-text-color)">};</span>
</code></span></span>Unfortunately, the EventSource Web API for handling SSE (built into most modern browsers) only supports GETrequests, which POSTis very limited when requests with large JSON data are required. bodyAs an alternative, you can use the Fetch API or an off-the-shelf library such as Microsoft's Fetch Event Source .
Likewise, with streaming enabled, you will now receive new tokens as soon as they are available. Given that OpenAI's API uses Markdown in its response format, the incoming tokens need to be put together and parsed into the full message in HTML that the function accepts replaceContent. For this, I use the Marked.js parser .
Now, with each new token, a greater response is building. Every time a new markup appears, the full Markdown is parsed and the content is updated, resulting in a nice "typing-like effect".
While this process does have some overhead, it's not noticeable in use, whereas Markdown simply parses with each new markup as it might contain, for example, the end of a code block or the end of a formatted paragraph. So while this process may be optimized, it will not lead to any discernible performance improvement in most cases.
Finally, it's worth noting the use of AbortController, which can be used to stop the stream at any time the user chooses. This is especially useful for longer responses.
GIF
the bottom line
Overall, I'm happy with the results. Thanks to Markdown parsing, data flow, good typing, and good integration with the editor's existing content blocks - all this together creates a compelling user experience.
Now, there is definitely room for improvement. The Block Actions API and Vrite Extensions as a whole still have a lot of development work to do before they can be created by other users. Other UI/UX improvements to consider, like operating on multiple chunks at once (extra context for ChatGPT for example) and displaying the UI inline (much like Notion AI) instead of blurring the view, are just a few I'm considering example. That said, it will take some time before these ideas are implemented well.
Vrite is more than just a GPT-enhanced editor. It's a complete open-source CMS focused on technical content like programming blogs, including a code editor, API, Kanban admin dashboard, and easy publishing integration. So if you're interested in trying it out and possibly using it to power your blog, be sure to check it out!