The one thing we still cannot do is evaluate code in the repl of the running program – because ghcid doesn’t support it. However, I mostly do this to test that functions work as intended, so what we can do instead is write that code as an actual unit test case, and have ghcid also run tests when it reloads the code.
We end up with a process where we still write code in our editor, with the lsp server giving us information about our code. Instead of playing around in the repl to try things out, we write our experiments as unit tests. When we save our changes (tests or application logic), ghcid reloads the code, runs the tests, and restarts the application in a way that doesn’t lose important state. We never leave our editor. It’s neat on paper, but I wasn’t sure how it’d work in practice, so I tried it with a toy project.
At the time of writing this, I was trying to patch up my lack of skill with differential equations and non-linear dynamics. The book I was reading33 Differential Equations; Blanchard, Devaney, Hall; Cengage Learning; 2011 has a pleasant focus on computational methods44 Traditional first classes in differential equations – including the one I took many years ago – had a heavy analytic focus out of necessity. In practice in the industry, differential equations are solved computationally because in many cases calculus is not sufficient to solve them any other way., and the book comes with software for performing the exercises, but I want to write the code on my own.
At least in the early chapters, the software is not exactly rocket surgery. Here’s an example of my program producing a slope field in light grey to visualise the general solution to a differential equation, and then in black the specific solution that passes through the coordinate under the cursor.
When exploring ideas from the book, I want to write some code, write some tests, and then when I save, the window containing the visualisation should automatically update to reflect the latest changes. It’s not exactly a Lisp development flow, but contains some of its good parts; specifically those that are possible with the technologies mentioned above. It sounded easy on the surface to get there, but it wasn’t.
I’ll first focus on getting Eglot with hls up and running, because that’s the less interesting part. The list below contains a lot of steps, because getting it installed from scratch with a new project takes a lot of steps. After this initial installation, only steps 2 and 4 are required to enable hls in any project.
At first I ran cabal init to create a standard empty Haskell project, split
into executable, library, and test code.
I discovered later that ghci (and therefore also ghcid) don’t behave well when it comes to reloading multi-part projects like that. Thus, I ended up co-locating all code in one executable project. I wouldn’t want to do that for production code because it makes other types of development more difficult, but it works okay for this kind of toy project.
Overall, I like the idea of the Eglot-and-hls experience. The additional formatting it adds to the buffer is not intrusive. Some of the hints and code actions are convenient. The one drawback I have trouble adjusting to is that it does add latency to editing in a way that sometimes really messes with me. I am used to Vim commands that zoom me around the editing buffer, but I can’t get them to fire as reliably when Eglot is enabled. I’m not sure why, but it’s bad enough to prevent me from running Eglot/hls in other projects.
If I could take the time to go on, I’d play around with it a little more to get a better understanding of its sharp corners. I’d also read the manual instead of improvising. The manual tends to be a good way to learn about sharp corners and how to sand them.
One of the drawbacks of using ghci (and consequently ghcid) for live reload is that ghci can only track one set of source files at a time. This means that if the project is properly set up with distinct components for executable, library, and test code, ghci can only reload one of those components. To apply changes to any of the other components, the entire ghci process must be restarted.
Since part of the intended workflow was being able to edit both tests and program code and reload both, we are forced to co-locate all the code into one component. This is a non-starter for serious projects, but acceptable for the kind of toy project I was aiming for – and maybe even for the early evolutionary stages of experiments where this workflow is most powerful.
The code for the part that live-reloads ends up looking something like this, with comments explaining the mechanism.
`module Main (main) where
import Control.Concurrent import Control.Concurrent.Async import Control.Exception (bracketOnError) import Data.IORef import Foreign.Store import Test.Hspec (describe, hspec, it) import Test.Hspec.QuickCheck (prop)
– The main function runs on reload, and has the – effect of first running the tests. If they – succeed, it runs the update function which makes – sure to restart the processing thread while – reusing resources. main :: IO () main = do spec update
– Unit tests in a mix of example-based and property – tests. These are written instead of poking about – in the REPL. spec :: IO () spec = hspec $ do describe “forward euler solution” $ do prop “ever increases with example study” $ \x y -> – …
– A function that must start our process if it is – not running, or restart it (and reuse its – resources) if it is running. update :: IO () update = let – This contains an MVar with the resources – needed to run SDL. It also serves as a lock – that prevents multiple processes from running – simultaneously. resourceStore = Store 0
– This contains the Async of the process thread, – for cancelling during an update. asyncStore = Store 1
– Wait for resources to be available, reserve – them, and start a new thread that uses them. withResources action = do withStore resourceStore $ \resources -> async $ – Catch async exceptions during execution, – such as when this thread is cancelled by – the update procedure. The exception – already causes the action function to stop – running, so then we release its resources. bracketOnError (putStrLn “Running action.” » takeMVar resources) (\res -> putStrLn “Action interrupted!” » putMVar resources res) action
start = do – Waits for resources and then runs the render – loop with them. withResources $ (window, renderer, texture) -> do renderLoop renderer texture (State study Nothing) putStrLn “Render loop exited naturally.” – Getting here in the control flow means the – render loop terminated but not through an – async exception. That implies the user – requested an exit, e.g. by closing the – window. Thus we should destroy resources – rather than release them back for reuse. destroySdl window renderer texture – We also need to delete all stores so the – next time the update function is called, – it sees a blank slate and recreates – everything all over. deleteStore asyncStore deleteStore resourceStore in do lookupStore (case asyncStore of Store i -> i) »= \case Nothing -> do – If the thread id store does not exist, it – means we’re starting from a blank slate. – We should initialise resources fresh and – start a new thread to use them. initialiseSdl »= void . storeAction (Store 0) . newMVar start »= void . storeAction (Store 1) . newIORef Just tidStore -> – If the thread id store exists, we cancel – the thread its referencing, which will – return the resources it used, and then we – start a new thread. The new thread picks – whatever resources were in store. withStore tidStore $ \ref -> do readIORef ref »= cancel start »= writeIORef ref
`
I initially tried using the higher-level Rapid library for this, but I couldn’t
get it to work properly. I found it much easier to get stability, resource
clean-up, and support for stdout in all sub-processes when I implemented the
plumbing myself with IORefs and Async.
The following invocation launches ghcid in a way that automatically reloads and runs the development main function:
ghcid --command "cabal repl --repl-options='-fobject-code -ferror-spans -fdiagnostics-color=always'" \ --reverse-errors \ --reload src \ --restart diffeq.cabal \ --test Main
This is the result of accretion-by-confusion. I couldn’t get something to work, so I tried another command line argument, and that happened a few times in a row. This is probably not the ideal way to write this command, but it seems to work and for now I want to play around with it and see how convenient it is, before I spend more time on it.
However, at first I couldn’t get this to work at all. The ghcid process
started all right, but it didn’t reload when code changed. It didn’t tell me why
either. I actually gave up on using ghcid because I couldn’t get it to work,
and then by accident I ran this command in a Nix development shell and it
worked. I have no idea what’s going on. I thought direnv would make it
unnecessary for me to first start a Nix development shell, but in this case it
seems not.
So far I really like the ghcid-and-foreign-store experience for this type of programming. Instead of creating a complicated gui for defining differential equations and initial conditions and whatnot, I can change constants in the code and the visualisation updates. When I learn new things in the book I’m reading, I can add code for them and the visualisation updates.
As someone not very familiar with emacs and/or Haskell, it baffles me that this is so complicated. How is this not a solved problem?
With Python and VSCode, all of this just works, out of the box. Sure, Python is interpreted so it’s not completely fair, but given that this workflow is well known, vastly superior, and simple in principle even in compiled languages, it’s strange that the tooling is not more streamlined by now.
Well, this was true when I started drafting this article. I have since adopted robots to do much more of the actual typing of code, unless it’s a project where (a) I care about code quality, or (b) I try to learn something.