Journal
End-to-end testing with Playwright
In addition to the new stack for the Cushion rewrite, I also want to touch on the testing harness. Back when I started Cushion, I was big into testing—and I’m still proud that the Ruby backend has rock solid test coverage—but for whatever reason, the frontend doesn’t have much test coverage beyond the client-facing invoice page. I’m not sure why, considering I fully tested the TeuxDeux frontend when I worked on it before Cushion, but maybe I was too excited for rapid progress. In any case, testing the frontend is so easy now that there’s not much of an excuse not to.
Personally, I’ve never had a problem thoroughly unit testing components, but when it comes to code that interacts with the API—where you should rely on end-to-end testing—I always feel the weight of the initial setup and tell myself I’ll do it later, but never do. With the existing Cushion, if I wanted to set up e2e testing at this point, I’d need to find a way to run all the servers within GitHub Actions, which is a daunting task (for me). Alternatively, I could do what plenty of folks probably do, which is to run their e2e tests against the staging server. Then, you just need to make sure you clean up after each run.
Since I’m starting from scratch—and determined to start on the right foot—I can keep e2e testing in mind from the get-go. Of course, I’ll also unit test components, but the coverage from e2e tests will be invaluable for sanity-checking the actual user flows. It’s far too easy for unit tests to pass while an unwritten e2e test would be failing—especially if you resort to unit testing API interactions with spies, but your API changes and your spies don’t. (I can hear a few of you nervously laughing right now…)
By using Supabase as my pseudo-backend that provides a direct client-side connection to the database, it’s much easier to set up e2e testing from the beginning (as long as their API is available). They do also have a self-hosted version if I wanted to go full neckbeard, but hosted will suffice for me.
As for the test harness, I’m using Playwright, which has emerged as the go-to option for e2e testing. There are others, like Cypress, but I feel like Playwright speaks more my language and would be familiar to anyone who has used Jest or Vitest. For those who haven’t written any e2e tests before, it’s actually fun. You simply write line-by-line what the user should do, like “go to this page”, “find this input”, “fill it in”, “click this button”. Obviously this is pseudocode, but the actual API reads just as easily. Playwright also has a codegen feature where you can record yourself clicking through the browser and it’ll write the test for you. Seriously, with features like this, you have no excuse.
So far, I’ve been able to fully test the auth flow as well as an initial setup form. These two cases have given me enough experience in this new stack to know how to quickly handle most situations. Along the way, I’m even writing helper functions, like one-line sign in, etc. The only hiccup I’ve come across is how best to reset the database, so I can do a clean run each time. I’m sure you can run a query on the database directly before or after the tests run, but instead, I saw this as an opportunity to implement a feature folks have asked for in the past—resetting an account.
For Cushion, users often become subscribers for a couple years, then they might take a full-time job and no longer need to freelance or use the app. Several years later, they might return, but want to start fresh with an empty account instead of returning to their old data. This is where a reset feature comes in handy.
I do realize that this is a dangerous feature and resetting your account is not something to take likely—especially for the users who have been with Cushion since the beginning. With those folks, we’re talking about almost 10 years of data. Because of this, resetting your account will certainly come with a safeguard in the form of a dramatic “Are you sure?” prompt that makes you type exactly what this feature does. I imagine I’ll eventually opt for a database query to clear the database to save time on click-throughs, but for now, this works and unblocks me.
Next up, I’ll start to dig into the actual app by setting up all the models as I need them, and make something usable. I’m excited!