Making an Editor
Recently, I was working on tools, considering how I was going to improve my theorem-proving experience with Coq. I’m an Atom user. I like it quite a lot. And before I liked Atom, I liked LightTable (I still do, to an extent). So naturally my first instinct was to write a plugin for Atom. I’d never done it before, but hey no time like the present, right?
I suppose I could have tried writing my plugin for LightTable. Clojurescript isn’t half bad, and there are many things to like about that project. But a thought occurred to me. Why just write a plugin? Why not make an editor? It was a good idea, and it kills several birds with one stone. Most notably, I need to make some shitty pots.
I chose to write an editor in Racket. I’d like to share my journey so far. Perhaps I’ll share the rest as I continue forward. I’ve learned some very interesting things! As for now, by the end of this post, we will have a fully-functional source code editor that is capable of not only editing its own source code, but also of reloading itself dynamically to reflect those changes to its code!
All code developed in these posts shall be licensed under the terms of the GNU General Public License, version 3. The full text of that license can be found here.
The Simplest Editor in Racket
Racket has a GUI library, unimaginitively but helpfully named
racket/gui. Have a peek at its excellent documentation. It has an editor widget built-in. Let’s use it to build the world’s simplest editor. After a
raco pkg new and a bit of typing, I have a simple
#lang racket/base ;; Scrividh is a text editor with some IDE features. ;; Copyright (C) 2017 Lucas Adam Michael Paul ;; ;; This program is free software: you can redistribute it and/or modify ;; it under the terms of the GNU General Public License as published by ;; the Free Software Foundation, either version 3 of the License, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; ;; You should have received a copy of the GNU General Public License ;; along with this program. If not, see <http://www.gnu.org/licenses/>. (module+ test (require rackunit)) (require "editor-main.rkt") (module+ test ;; Tests to be run with raco test ) (module+ main ;; Main entry point, executed when run with the `racket` executable or DrRacket. (editor-main) )
I want the actual code of the editor itself to be in a separate file. It’ll make some things easier later – particularly refactoring. So all functionality is delegated to
#lang racket/gui ;; Scrividh is a text editor with some IDE features. ;; Copyright (C) 2017 Lucas Adam Michael Paul ;; ;; This program is free software: you can redistribute it and/or modify ;; it under the terms of the GNU General Public License as published by ;; the Free Software Foundation, either version 3 of the License, or ;; (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; ;; You should have received a copy of the GNU General Public License ;; along with this program. If not, see <http://www.gnu.org/licenses/>. (provide editor-main) (define (editor-main) (define f (new frame% [label "Scrividh"] [width 200] [height 200])) (define c (new editor-canvas% [parent f])) (define t (new text%)) (send c set-editor t) (send f show #t))
It’s mostly copyright notice. Sorry about that. I’ll fold it in future code blocks. (Caution: do not run if you don’t have an external way to kill or close the program. It has no “Quit” button yet!) But look! It runs!
This much anyone could have done by looking no farther than a page into the Editor documentation. In fact, they could easily get copy/paste working by reading just a few lines further. But even that isn’t really very good. After all, this editor cannot even load or save files!
Let’s fix that. The
racket/gui library makes this easy. I’ll just show the
editor-main function, since that’s all that changed:
(define (editor-main) (define f (new frame% [label "Scrividh"] [width 200] [height 200])) (define c (new editor-canvas% [parent f])) (define t (new text%)) (define mb (new menu-bar% [parent f])) (define m-file (new menu% [label "File"] [parent mb])) (define m-edit (new menu% [label "Edit"] [parent mb])) (define mi-open (new menu-item% [label "Open"] [parent m-file] [callback (λ (i e) (define path (get-file #f f)) (when path (send t load-file path 'text)))] [shortcut #\o] [shortcut-prefix '(ctl)])) (define mi-save (new menu-item% [label "Save"] [parent m-file] [callback (λ (i e) (send t save-file #f 'text))] [shortcut #\s] [shortcut-prefix '(ctl)])) (append-editor-operation-menu-items m-edit #f) (send t set-max-undo-history 100) (send c set-editor t) (send f show #t))
We add menu items for Open and Save, and give them callbacks that ask the
text% object to do the actual work of opening and saving files. Adding menu items even has a way of giving us keyboard shortcuts, which is just beautiful. This is more like it!
We can actually use this to edit its own source code! It’s ugly, and doesn’t do syntax highlighting or proper indentation… yet. Let’s fix that RIGHT NOW!
As it happens, there is another library we can leverage for this. It’s called
framework, and its documentation is here. It’s maybe not quite as polished as
racket/gui’s, but still very good. It has a
racket:text% class that will do nicely as a drop-in replacement for
text%. I won’t show the whole code again, because there are only two small changes: Adding
(require framework) before the
provide, and replacing
racket:text% on line 27.
Wow. It is positively mind-blowing how much we can do in less than 100 lines of Racket. Granted, this is standing on the shoulders of geniuses who wrote some fantastic library code, which does far more than it has any business doing. Also, there are some serious limitations to how far we can keep going in this direction without starting to write some brilliant (and lengthy) code of our own.
But there’s still something we can do before we wrap up this post. Something I promised early on. I haven’t forgotten. There are still a couple of serious flaws with this program. First of all, it still doesn’t have a quit button. Second of all, it’s really irritating that you have to quit and then restart the editor every time you want to test a change. What’s the point of being able to edit your own source code if you have to restart every time you save?
Racket has a cool little function hidden away in the less-utilized corners of its library:
dynamic-rerequire. That’s not a typo. There are two “re”s in there. Yes,
dynamic-require is a thing. I’m talking about its lesser-known partner. Let’s have a look at the documentation:
(dynamic-require module-path 0), but with reloading support. The
dynamic-rerequirefunction is intended for use in an interactive environment, especially via
If invoking module-path requires loading any files, then modification dates of the files are recorded. If the file is modified, then a later
dynamic-rerequirere-loads the module from source; see also Module Redeclarations. Similarly if a later
dynamic-rerequiretransitively requires a modified module, then the required module is re-loaded. Re-loading support works only for modules that are first loaded (either directly or indirectly through transitive requires) via
… ooOOOOOOOOOOOOOOOOOOOOOooooooooooooooooh! That sounds like fun. It’s time to rewrite
main.rkt a bit. Get rid of
(require "editor-main.rkt"), and replace it with
(require racket/rerequire). Now we need a good loop:
(define editor-main-module "editor-main.rkt") (define (main-loop) (dynamic-rerequire editor-main-module) (let/ec break (let loop ([hot-swap-state #f]) (let* ([editor-main (dynamic-require editor-main-module 'editor-main)] [editor-custodian (make-custodian)] [hot-swap-state-prime (parameterize ([current-custodian editor-custodian]) (call/ec (editor-main hot-swap-state break)))]) (custodian-shutdown-all editor-custodian) (dynamic-rerequire editor-main-module) (loop hot-swap-state-prime)))))
This is slightly dense, so let’s do some unpacking. First, we abstract away the file location of
editor-main.rkt. That’s just so that in case we ever want to move the file, we know there’s only one place to change it in the code. The main loop first does a dynamic-rerequire of that editor-main module. This call comes first so that it’s
dynamic-rerequire that first pulls in
editor-main.rkt as opposed to
dynamic-require, which will not properly record the modification time of the source file.
Next, we record an escape continuation with
let/ec. Why? So that it can later be used to quit the program. There will be basically no other way out of this loop, and this is the easiest way (for me) to “break” out. Now we need to get our hands on the
editor-main function itself. We do this with a call to
dynamic-require. It gives us whatever provided identifier we ask for, so we ask for
editor-main, which is going to have a few changes of its own later. We also get ourselves a new custodian, which is a Racket thing you can read about here. Basically, if we execute some function with a custodian and then shut that custodian down, we can clean up after just about anything that code did, including its threads and GUI.
Finally, we call
editor-main with the new custodian, the
break continuation, and another escape continuation made on the spot right there with
call/ec. Why two escape continuations? One to be able to Quit, and the other to be able to go back to the beginning of the loop from anywhere within
editor-main without necessarily needing to return properly.
editor-main to return with something we name
hot-swap-state-prime. That’s funny;
editor-main didn’t accept any arguments before, and it certainly didn’t return anything. What’s this about? Well, it’d suck if reloading the editor caused you to lose all your unsaved work, or made the editor forget what file it was editing.
hot-swap-state is going to keep that around for us, so we can put it back once the GUI is rebuilt. That’s why we call
loop with the new,
hot-swap-state-prime, which becomes
hot-swap-state in the next iteration. On the first run through, there is no
hot-swap-state, so we call
#f to let it know it’s beginning fresh. But on every subsequent call, it gets whatever it gave itself when it was shut down before. So we can pass whatever we need to keep between instantiations.
Before looping, we need to call
dynamic-rerequire again, so that the new contents of the source file are reloaded before
dynamic-require pulls in the
editor-main function for the next call.
We should call this loop from the main module:
(module+ main ;; Main entry point, executed when run with the `racket` executable or DrRacket. (main-loop) (exit) )
Now we need to fix up
editor-main.rkt a bit. First, rename the
editor-main function to
build-gui, and make it take two arguments:
(define (build-gui main-thread hot-swap-state)
We’ll need auxiliary functions within the definition of
build-gui. Call them
restore-hot-swap-state, and put them right after the definition of the
(define (build-hot-swap-state) `(,(send t get-flattened-text) ,(send t get-filename) ,(send t is-modified?))) (define (restore-hot-swap-state) (match hot-swap-state [(list text-contents filename modified) ;=> (send t insert text-contents) (send t set-filename filename) (send t set-modified modified)]))
We also need two new menu items after “Save”:
(new separator-menu-item% [parent m-file]) (define mi-reinstantiate-editor (new menu-item% [label "Reinstantiate editor"] [parent m-file] [callback (λ (i e) (thread-send main-thread `(reinstantiate-editor ,(build-hot-swap-state))))] [shortcut #\r] [shortcut-prefix '(ctl alt)])) (define mi-quit (new menu-item% [label "Quit"] [parent m-file] ; TODO: Prompt to save modified files [callback (λ (i e) (thread-send main-thread 'end))] [shortcut #\q] [shortcut-prefix '(ctl)]))
And we need to call
restore-hot-swap-state when it is not false at the end of the function:
(when hot-swap-state (restore-hot-swap-state))
And finally, we need a new
editor-main that will set things up nicely:
(define ((editor-main hot-swap-state end-program) reload-program) (let ([main-window-es (make-eventspace)] [you-got-mail (thread-receive-evt)]) (parameterize ([current-eventspace main-window-es]) (build-gui (current-thread) hot-swap-state)) (match (sync ;; Don't change the code inconsistently (wrap-evt main-window-es (λ (v) 'closed)) (wrap-evt you-got-mail (λ (v) 'mail))) ['mail (match (thread-receive) [`(reinstantiate-editor ,hot-swap-state-prime) ;=> (reload-program hot-swap-state-prime)] ['end (end-program)] ;; Unknown message [m (displayln m)])] ['closed (end-program)] ;; Can't happen unless the code is changed incosistently [s (displayln s)])))
This is also a bit thick, so let’s unpack it. The
make-eventspace is making a new Racket thread, with a few extra trimmings that help keep GUI things nicely separated out. This allows the
build-gui function to return and the GUI to remain responsive while the
editor-main function sits and listens for messages.
you-got-mail is an event that is ready for synchronization when the thread recieves a message from another thread via
build-gui function is called with the current thread (so that it can send messages back) and the hot-swap-state as arguments.
Then we synchronize on two possible Events. If we get
'closed, it means the window was closed. If we get
end-program, using the continuation that pulls us totally out of the loop in
main.rkt. However, if we got mail, we need to open it to figure out the user’s intentions.
If the user wants to reinstantiate the editor, then we got a new hot-swap-state from the GUI callback. We’ll just invoke our
reload-program escape continuation with that value to send it back to the loop in
main.rkt, where it’s expected. If the user wants to quit, we invoke the
end-program escape continuation, breaking us totally out of the loop.
And there you have it. Try running the result. It doesn’t look any different except for the “Reinstantiate editor” and “Quit” buttons. But try changing
editor-main.rkt, saving, and then hitting “Reinstantiate editor”. Isn’t that fun?
Next time, tabs! Probably.
(edited on 2017-08-01 to fix a typo and be ever so slightly more professional in the intro)