A few days ago, I attended viennajs, and both talks shared a common theme: testing! I think a lot of people (myself included) don’t test enough, and I believe it's because getting started with testing can feel intimidating. I'll share some tips at the end of this post…
During the evening, I discovered Mutation Testing! I had never heard of it before, but it came with the intriguing subtitle: "testing your tests". If testing is intimidating at first, then testing your tests might seem even more so - haha! But Lukas Stracke, the presenter, did a fantastic job gradually introducing the concept.
As we are in 2024, I recommend starting with vitest, and you’ll see that it’s not as complicated as it seems. Add the dependency, create a file like myCoolFunction.spec.ts, and you can run tests with the command vitest. You might begin with something like this:
it("should be great", () => {
const ret = myCoolFunction({ ... })
expect(ret).toBe("great")
})
As you add a few tests, you’ll quickly feel more confident making changes to your code since you’ll have these tests backing you up. 😉
You might even want to check your code coverage (I’ll let you look up how to enable this). Essentially, you’ll try to cover all lines and branches of your code to reach 100%. Spoiler alert: aiming for 100% might be too ambitious, and you may end up "fake testing" just to increase the numbers. You may catch yourself writing code like this:
it("should test my cool function", () => {
myCoolFunction()
expect(true).toBe(true)
})
This technically "covers" the code, but doesn’t test anything! In this example, it’s obvious, but in real code, things like this can happen… This was an "aha moment" for me that evening! 🙏 Thanks, Lukas.
And here’s where Mutation Testing comes in. To simplify, it’s a technique that alters the code in myCoolFunction using various strategies (called mutants). Then, you rerun all tests with these mutations, hoping that all mutants will be “killed.” If any mutant survives, it means your tests didn’t actually verify your behavior. At the end of mutation testing, you’ll get a score showing how many mutants were killed—the higher, the better.
If code coverage is high but the mutation testing score is low, it means your tests aren’t really meaningful. If your code coverage is low, then the mutation testing score isn’t as significant because, well, you’re barely testing! So the goal is to keep both scores high. 🎉
This was so enlightening! It’s why I love going to meetups—to learn!
More than a year ago, I discovered remult and it blew my mind. It was soo cool, that I switched all my projects to it. In addition to its features, the high code coverage (89%) showed me that Remult takes testing seriously.
After the meetup, I got curious—not just about code coverage but also about mutation testing! Knowing Noam, I had no doubt, but I still wanted to see for myself.
As soon as I got home, I created a branch and added stryker to the Remult codebase. It took about 15 minutes to set everything up and launch mutation tests. Lukas had warned us it could take time… In my case, it estimated 10 hours to run on 30,000 mutants! Perfect, I went to bed and checked the results in the morning.
Of course, great surprise in the morning, Remult had a mutation score of 82%—an amazing result without any special tweaks! 🥳🥳🥳 I was thrilled to see again that I had chosen the right stack.
I knew Noam was good, but this level of commitment to testing is impressive! You can tell that for Noam, testing is a crucial part of the library development workflow. Testing applications with Remult was already easy, but recent improvements introduced a TestApiDataProvider provider. With it, you can fully test your APIs with just a few lines of code. There’s even a live tutorial available here to walk you through it.
With these few lines of code, you can fully test your APIs, hooks, entities, SQL, and more. It’s A-M-A-Z-I-N-G! Once you try it, there’s no going back.
All this to say, I used to be like many others—rarely testing my code, always wishing for more time to implement tests. But now, there’s really no excuse. In just 15 minutes, you can set up TestApiDataProvider and save yourself SOOOO much time over the long run. Trust me, it’ll be the best 15-minute investment you’ll make.
