Prototypes

You may consider not doing TDD with prototypes. By prototype, I mean an app that will most probably be thrown away or completely changed very soon. It may be created when you are experimenting with some technology/framework or are just asked to conduct research on something.

TDD gives you a high code/flow coverage, which may need to be changed in case of prototype projects. If you are in such situation, consider resigning from TDD at this stage.

Short living apps

Similarly to prototypes, short living solutions might be the ones where TDD is not the best fit. It applies to one-time-use apps or scripts which are supposed to do one thing well and quickly.

Short living apps are the ones where you may consider not using Test Driven Development. I’d even argue whether you need any kind of tests for such programs. Testing such apps or even scripts may be art for art’s sake. If something is supposed to realize a given task without fireworks (like reusability, maintenance etc.), I wouldn’t bother with testing it. It’s better to focus on solving your current, short-lived problem than on practicing the profound TDD idea ?

Unclear requirements

One of the reasons not to use TDD might be unclear requirements. The idea of writing tests in Test Driven Development is to define the expected outcomes for predefined incomes. As we said in the previous article, Test Driven Development can help spot the edge cases and make the business requirements more close to the code. However, if you don’t have these requirements defined, TDD might be a waste of time (at this stage). If the requirements actually arrive or are completely changed, your significant set of tests will need to be changed or even removed.

MVP

If you’re creating an MVP, with very short time and budget, TDD might not be worth the struggle. It’s probably better to resign from using TDD in that case. You should rather focus on your product and the business value it needs to deliver. TDD might slow down the delivery time, which, in case of MVPs, can be critical. It’s better to deliver the minimum viable product without TDD than to not deliver it with TDD ?

However, remember to add some tests, even post-factum. MVPs have a high chance of survival, so you’d want your final product to be tested and testable. You need to find a good balance here, not to make your solution completely untestable when it’s ready for production use.

Tests writing beginner

If you are a total beginner in writing tests in general, Test Driven Development might not be the best approach to start with. You may consider not to use TDD at the beginning of your testing journey.

To make Test Driven Development effective, you should be able to write tests quickly and efficiently. Starting to learn software testing with Test Driven Development may make you feel overwhelmed and discouraged. If you’re a beginner, I suggest to first write some unit tests, get familiar with that and then try to learn Test Driven Development.

It would be great if you can find some smaller and well-defined task to start with TDD. If you don’t have such tasks, try solving one of the programming katas (e.g. this one) with TDD.

When not to use TDD – summary

Like everything, TDD is not a gold standard for every possible case. It should be used as often as possible, but not always. Remember to always weight the gains and losses from using Test Driven Development in a given project or application at a given time.

What’s your experience with Test Driven Development? Can you list any more cases where it’s not worth using it? Share your thoughts in the comments below!

The post When Not To Use TDD? appeared first on Yumasoft.

Moving design into code

With test driven development, you create your tests before writing production code*.

Having that in mind, one of the benefits of using TDD is that you need to move your design requirements into code before you even write a single line of business logic.

It means that you need to spend time thinking about good names for your tests. I often create empty tests straightaway after staring to work on a task. Spending time to create well-named test methods can help you quickly spot missing parts in the design.

*By production code I mean the actual business logic except the tests’ code. This is only to differentiate here, but I personally consider my tests part of the production code in real life. Tests are as important as all the code.

Finding edge cases

Going further, if you create many empty tests with well-thought names, you will surely find many edge cases. Thanks to test driven development, you’ll verify the business requirements by your code without writing even a single line of it. Discovering edge cases and boundary conditions will let you clarify the requirements or even change/adjust them before any production code is written ?

Structuring your code beforehand

You’ll see that TDD forces you to think about the code you’re about to write beforehand. You’re going to think about production code to-be-written: how and where to add it, what methods, parameters and classes are needed. Thanks to that, you can quickly notice that maybe the place you’d normally place the new code in is not the best choice. Maybe it requires too much changing instead of modifying.

Such approach leads to taking and rethinking architectural and structural decisions before writing a single line of production code. In the end, it saves you a lot of time. Without TDD, you’d probably just add this new code where you thought it’s appropriate and struggle to refactor it later. And refactoring without tests is dangerous ?

Allowing to refactor

Test driven development is based on red-green-refactor cycle. As soon as your test is green, you can safely (for sure much more safely than without tests, especially if you combine TDD with mutation testing) refactor the code you’ve just written.

In the end, the quality of your code is much higher than it would be without tests. You don’t feel the fear of refactoring, which is a common blocker for improving code quality in legacy systems.

Making implementation very easy

When you move from red to green phases of TDD, it’s time for implementation. From my experience, writing production code that satisfies TDD-produced tests is very easy.

I find it much easier than writing code not covered by TDD-based tests. You exactly know what to do and which tests must pass. Thanks to that, your implementation is minimal – it should only make the test(s) green.

Creating current code documentation

Implementing a new feature with test driven development will leave you with a comprehensive set of tests. From my experience, I write few times more tests with TDD than in a “traditional” tests-after-code mode.

As a nice side effect, you’re building and growing the only possible, current documentation of your code – set of your tests. Only tests can tell you the truth about your code. It can’t be achieved with any other kind of documentation. Notice one more advantage – you’re creating documentation… without writing any documentation! ?

Current, well-named tests are very helpful not only for developers, but also for the so-called business people.

Spotting architectural problems

Last but not least, test driven development helps to solve application’s architectural problems. Tests should be relatively easy to write. If the test is hard to write before production code exists, it might mean that the architecture of your solution is wrong. There’s a problem somewhere that should be addressed.

That’s why it’s very important to start working with TDD as early as possible in your project. There might be some cases where TDD it not suitable (maybe I’ll write a separate article about it ?), but in quite mature and stable projects, it should be a gold standard. Altering the architecture is much easier early in the project.

The post 7 Reasons To Start Doing Test Driven Development appeared first on Yumasoft.

In this article, I’m sharing how this can be done ? It will also be useful if you use another testing framework than NUnit, but the examples are based on it.

Starting point

You can find the code accompanying this article here: https://github.com/dsibinski/cypress-csharp-example

Our starting point is a React application with Cypress tests:

I assume that your tests specs are placed within cypress/integration folder. The only spec we have is called submit_spec.ts and it placed in a subfolder called form.

Now imagine, that your backend is written in C#. Maybe you’d like to perform some data initialization before Cypress tests are executed.

In our case, we have a C# backend, and we’ve already had a mechanism for running integration tests. This mechanism runs our server app with an in-memory database, which makes the maintenance of tests very easy.

We wanted to use the same mechanism for our new, E2E Cypress tests. It means that we would be able to write some C# initialization code (data creation mostly) and then run Cypress tests from NUnit test.

What we want to achieve is to have a C#-Cypress NUnit test look as follows:

Let’s now see how we can implement the RunCypressTest method ?

Implementing a method to run Cypress tests from NUnit

To make things easier, let’s create a base class all our test classes can use. We’ll call it CypressTestsBase and add a RunCypressTest method to it:

This code does the job. Few interesting parts:

• Line 12 – I’m getting there the path to the web app workspace where npx cypress run command can be executed. It might be more complicated in your scenario, so you should adjust accordingly here
• Lines 13-20: thanks to setting RedirectStandardInput to true, we can later execute a custom command via cmd; setting RedirectStandardOutput to true allows us to write everything npx cypress run command does to NUnit test’s output
• Line 28: TestContext.Out.WriteLine(cypressProcessOutput); actually writes the Cypress output to NUnit test result
• Line 29: here we handle errors during Cypress execution. npx cypress run executed via cmd will return a code different from 0 if anything goes wrong.

We can now change FormTests class to run its corresponding Cypress test:

Finally, it allows to run our C# test, which then runs its corresponding Cypress spec and prints the output:

Cypress errors in NUnit result

Let’s see what happens if something is broken on Cypress/web app’s side. Let’s run our test, but with our web app not running.

The NUnit test fails, and we can clearly see what was wrong:

You will see similar errors if something goes wrong in a Cypress test. Cypress also saves compressed videos and screenshots, so if something goes wrong, diagnosing such failed tests is very easy.

Summary

I hope you found this article on how to run Cypress tests from NUnit useful. Remember that it not only works with NUnit – it will be suitable for any testing framework. You might need to adjust some parts of the solution, e.g. printing the results to the test result’s output.

This solution allowed us to improve our testing by adding a new type of tests to our pipeline. Right next to proper naming and mutation testing, E2E tests add another layer of security and improve the quality of our software.

Once more, you can find the complete source code here. If you want, feel free to copy it and change it according to your needs ?

The post Run Cypress Tests From NUnit Test (C#) appeared first on Yumasoft.

At Yumasoft, we believe that testing is an inseparable part of software development. Mutation testing, which recently got a bit more attention in programming communities, is a great way of improving our tests. In today’s article, we’re exploring what mutation testing is and why everyone should be using it ?

What is mutation testing?

To make it clear, mutation testing isn’t just a new type of tests you can write. At least not in a casual sense of a test ?

Mutation testing is a technique for validating your already-existing tests. They basically allow to test our tests.

Mutation tests are not for people who don’t write tests (go away, we don’t like you!). They are for people who care about testing and want to make their tests better.

How does mutation testing work?

Mutation tests introduce mutations into our source code under tests (you can call it production code). Mutations can be any changes in the source code you can think of. Let it be a modification of a condition within an if statement, replacement of a string value or change of an arithmetic operator – you name it. All you need to know is that mutation testing is based on automatically introducing changes to your source code.

Our codebase with such a change (a single one) introduced is called a mutant. It makes sense – our production code is now somehow mutated.

What happens with such a mutant then? All our existing tests are executed against it. Such tests’ execution may produce two outcomes:

• all tests have passed – that’s a bad scenario. It means that our mutant has survived. In other words, it means that a modification of our production source code wasn’t detected by our tests suite
• at least one test has failed – that’s a good scenario. It means that our mutant has been killed. This is what we want, because it means our tests are well-covering and detect changes in the production code.

You can see the flow of mutation testing on the following schema:

Example of mutation tests

To make mutation tests more practical, let’s see an example. We’ll be using StrykerJS for production code written in TypeScript.

This is our production code:

We have of course written some tests for this code. When we execute these tests with jest, we can see the following result:

So we have 100% coverage for DiscountApplier class! What’s more, we have 100% statements, branch, functions and line coverage! This sounds perfect, right? Why should we bother more? ?

Let’s deep dive and run StrykerJS mutator against our app. After installing and configuring Stryker properly, we can execute npx stryker run command. That’s the final result:

Whoooaa… we see here some 82.14% value, not 100% anymore ? This is what I call mutant coverage.

Remember what we said above about survived and killed mutants? Pay attention to our mutations results above – we have 46 mutants killed, but 10 of them survived. It means that we have a potential to add more tests to cover these scenarios.

We will not be going through each mutant and adding the test for it in this article, but I can tell you what these results can give you. In the beginning, when our mutant coverage was 82.14% with 10 mutants survived, we had 6 unit tests written for DiscountApplier class. In the end, after working out and killing each mutant one-by-one, we finished with 12 unit tests and 100% mutant coverage.

In 10 minutes (time needed to add these 6 tests), I got twice as many unit tests as in the beginning. Now, both “traditional” and mutant coverages are 100%. My confidence in refactoring or modifying the production code is much higher now. Notice that we got this all “for free”. Mutation testing tool (StrykerJS in that case) did everything for us. This is huge! ?

How to start writing mutation tests

Just find a proper mutation testing tool for the framework/programming language you work with. Here are a few most popular examples:

• JavaScript/TypeScript/C#/Scala: Stryker mutator
• Ruby: mutant
• Java: Pitest
• PHP: Infection

Go straight to the docs of your tool of choice and start mutating ? The messages about each mutant are very clear, so adding new tests to kill them will be very easy. It’s like someone sitting next to you and telling you exactly what test you should add for your code. That’s actually a pretty decent solution for anyone who doesn’t like writing tests ?

I don’t see any reason why anyone would not use mutation testing. It’s such a great tooling that I don’t see any disadvantages in using it. It can greatly help you with TDD as well as an additional “layer” of inventing new test cases and patching your code. Believe me, you’ll see the benefits straightaway.

So – download one of the mutation tools and make your tests bulletproof today ? Or at least see how bad they are now ?

The post Make Your Tests Bulletproof With Mutation Testing appeared first on Yumasoft.

Consequences of bad tests’ names

You may think I’m exaggerating here. Why would I care about naming my tests methods? These are only tests… Let’s not waste time on thinking about such unimportant things. However, before we dig more into that, let’s see how typical badly-named tests look like. Below, I’m presenting the types of badly named tests that I see most often. I also used to name my tests like that ?

Random names

This is what I like the most:

Just random names. Surprisingly, I see that quite often. What does such a list of passing tests tell us? Why do they pass? What do these tests verify? We know nothing about them. We can only see that there are some tests. What they test, how and under what conditions – can’t say ?‍

Tested object described

This is another type of tests I sometimes encounter. A bit better, yet still far from fully informative:

Looking at these tests list, we can at least say a few things:

• there exists a UsersService class which is under tests here
• UsersService has at least two methods: GetUser and Login
• under some unknown conditions, GetUser and Login methods probably work

However, we still don’t know what it actually means that these tests pass. Does it mean that login and getting users are well-tested and work correctly? What conditions need to be met so the user can log in? What parameters do we have to provide to GetUser method to use it properly? Again – we can’t say.

Expected outcome described

Let’s go one step further. A bit improved tests might look as follows:

Someone tried to be a bit more descriptive here. That’s good, isn’t it? ? Let’s try to see how many things we can say about our codebase now:

• there exists a UsersService class which is under tests here
• UsersService has at least two methods: GetUser and Login
• GetUser method returns user data
• GetUser might sometimes throw UserNotFoundException
• Login logs in the user correctly under some conditions
• Login returns a user token
• Login might sometimes throw WrongCredentialsException

We already know much more about our codebase! However, notice that we still often use very uncertain words like “sometimes” or “some”. It doesn’t feel we’re sure about these outcomes. We still don’t know under what conditions these tests pass or exceptions are thrown.

Why badly named tests are bad?

I think you already know that at this point. Badly named tests are confusing. They don’t tell us enough about our codebase. One of the goals of tests is to be a live code documentation. But how can they be if they are very undescriptive? How can we know what happens in case such a test fails:

What do we know looking at that? Why could this test fail? If there’s a non-technical person looking as this result in a CI system while programmers are not around – how can (s)he tell what’s wrong and how to potentially fix that?

This result tells us nothing more than the Login process does not throw WrongCredentialsException. But we don’t know when it should throw this error. What are the conditions that should be met for this exception to be thrown? Again – we can’t say.

Notice that I used the best from all examples we had so far. What if this test was named Test1() and it failed? Does it mean anything? Is it something serious blocking the release of our app, or we can ignore it? ?

I guess you now see very well what I mean by badly named tests. I hope you also understand – and agree ? – that such tests’ names are bad.

Fortunately, we all here write tests, right? ? So, if we are already putting some effort in creating tests, let’s put a bit more to make them informative and useful for others.

Naming tests done well

There are a lot of unit tests naming conventions. You can read about them in many great articles like this one. There’s a lot of discussion around naming tests conventions.

However, independently of which naming strategy you choose, I think a well named test should tell someone who reads its name the following things:

• what is tested (class name)
• which part of it is tested (method name)
• what is the expected result
• what are the conditions for the expected result

Many of tests naming conventions fulfil these 4 requirements.

Let’s come back to our last example of badly named tests (but still its best version):

Let’s use one of the naming conventions I like:

MethodName_Should_ExpectedResult_When_Conditions

Applying this to our tests, the final result is as follows:

Now, for each test, we can see what is tested (UsersService), which part of it (e.g. Login method), what’s the expected result (e.g. ReturnUserToken) and under what conditions (e.g. CorrectUsernameAndPasswordProvided). This is very descriptive and valuable. Now, when something goes wrong:

We know a lot more than before. We can easily see that when wrong credentials are provided, the WrongCredentialsException is not thrown (while it should). It makes it easier to judge whether this is blocking for the release of our app or not. We probably don’t want to let users login with incorrect credentials ?

Another, very important factor for naming your tests well is that the whole team agrees on one naming convention. I think it’s even more important than choosing a particular naming convention. Choose whatever suits you and your team. There are a lot of options, but in the end it doesn’t really matter which one you agree on. Just stick to it and make sure it delivers all value you need.

How naming tests well can help the programmers

Correctly and descriptively named tests can help you as a programmer. Imagine that you are about to introduce a change in UsersService, but you have no idea how it works. The first thing I do in such cases is to open a corresponding tests file and scan all unit tests.

First, I expect that the tests file is named UsersServiceTests, because my team follows a common naming convention. Thanks to that, I can immediately open this class in my IDE and quickly scan all unit tests names:

This tells me a lot about the UsersService. If I want to see only how Login method works, I can easily filter out all tests with names starting with “Login”. Again – thanks to following the naming convention.

The next thing is that when you start making changes to the production code of UsersService and one of the tests fails, you know what’s wrong very quickly.

Another advantage of naming tests properly is when you are adding new tests. If you’re obliged to follow a descriptive tests naming convention, you need to think very well what your new code under tests should do and under what conditions. This makes your production code more structured and well-thought. Just give it a try, especially if you use TDD. I often start my TDD session with writing empty tests named correctly. This gives me many ideas of the production code I’m going to write straight after.

How naming tests well can help the business

I think I don’t need to explain that to you anymore. However, remember that your tests might be useful for other people than programmers. All non-technical people involved in the project can have a great use of the tests if you name them correctly.

I hope that your tests are executed as part of a CI process you use for your app deployment. As I wrote before, imagine having a late Friday release of your app. As a programmer, you’re the last one to commit this one last bug fix, and you go home. The project manager stays alone, waiting for the CI build to finish, so he can accept and deploy the new version of the app. But then, one of the tests fails. If your tests are named well, he will be able to easily judge whether this is blocking the deployment or not. I think that gives a lot of value to the business.

In case of some applications, where classes/services are created close to the domain of the app, business people can also explore the set of tests for a given service to grasp an idea of how it works and why it works as it does. Similarly, when you explore the tests class in your IDE. This wouldn’t be possible in every project, but it might also become a benefit of well-named tests in some cases.

Summary

I hope that after reading this article, you know why naming tests matters. Giving your test a descriptive name is a small effort, but it brings huge benefits. Both to your fellow programmers, but also to non-technical, so-called “business people”.

The post Why Naming Tests Matters? appeared first on Yumasoft.

Jest config

First, make sure that your jest configuration is correct. If you’re with TypeScript (and you should ?), I recommend using ts-jest configuration preset. Our config looks as follows:

If everything is fine, you should be able to execute npm test command (assuming it’s defined in package.json as "test": "jest") without issues. All your tests should be executed.

We now need to make StrykerJS working with that.

StrykerJS with jest configuration

If you install StrykerJS as described here, you’d expect that npx stryker run command runs mutations and everything is just fine. However, if you’re having wierd issues and errors with that, it probably means that Stryker is not properly configured to execute tests with jest.

When you run npx stryker run, it creates mutants (modifications in your source code) and runs tests against them. It means that StrykerJS must run your tests in the same way you run them. So, if you run tests with jest, Stryker must be able to do the same.

We struggled with many issues related to running StrykerJS with jest. Some of them were related purely to jest, but some of them to TypeScript as well. After numerous trial-and-errors and consulting with StrykerJS community members, we worked out a configuration that finally works fine for us.

Going straight to the point, here’s the stryker.conf.json file that would make StrykerJS work fine with jest and TypeScript:

To make it work, install the following npm packages: @stryker-mutator/core, @stryker-mutator/jest-runner and @stryker-mutator/typescript-checker.

Notice few important settings here:

• "checkers": ["typescript"] – this enabled a TypeScript checker plugin, which makes mutations being checked for types before running tests on them. It means that if a mutation introduced by Stryker doesn’t compile because of types issues, it won’t be taken into account for mutations (which makes perfect sense)
• “disableTypeChecks” – you still need to tell StrykerJS to not check types when compiling your code with type errors (mutants). It will compile them without types checking, but thanks to TypeScript checker plugin these ones will be filtered out for mutations
• "tsconfigFile": "tsconfig.json" – make sure to point to the TS config file you’re using here
• "concurrency": 4: this is another setting we experimented with to make Stryker working correctly. Without it, we were getting weird errors about lack of memory when running mutations (FATAL ERROR: Committing semi space failed. Allocation failed - JavaScript heap out of memory)
• "testRunner": "jest" – this one enables the jest as our tests runner
• "jest" – here you tell Stryker to use your jest.config.js file we talked about at the beginning

With these settings, I hope that you are able to see this beautiful output of npx stryker run in your console:

StrykerJS with jest and TypeScript – summary

I hope this is helpful for you. StrykerJS is a developing framework, so some things are still hard to figure out. We believe mutation tests are a great way of improving our tests, so we use them on a daily basis. If you ever have any issues with StrykerJS, I encourage you to reach out to the community on Slack. They are extremely helpful there ?

The post How To Use StrykerJS With jest And TypeScript appeared first on Yumasoft.

and you’re looking for how to show their results in TeamCity as follows:

Then this article will help you configure that ?

Prerequisites

To run jest tests on TeamCity, you need to use jest as your testing framework of course ? If you use TypeScript – which I definitely recommend – ts-jest package will also be useful.

Having these packages installed, your basic jest.config.js file should look similar to this:

You should also have a test script defined in scripts section of your package.json file:

With this setup, you are able to execute your tests with npm test command and see the results in the console:

Such tests results are however not recognized in any way by TeamCity build server. Let’s see how to configure that.

Installing TeamCity reporter

In order to see the tests results in TeamCity, we need the npm test command to produce a different output when run on TC server. This output must be compatible with TeamCity’s service messages.

Fortunately, you don’t need to implement it by yourself ?

First, install jest-teamcity npm package to your project. Next, add jest-teamcity to reporters configuration option in your jest.config.js file:

Adding TeamCity build step

Next, login to your TeamCity server. You need to know the location of npm.cmd file on your TeamCity server. It should be in NodeJS installation catalog. In our case, the path to the file is C:\Dev\nodejs\npm.cmd.

Now, go to TeamCity web administration interface and create a new build configuration or edit a current one. Then, add a new build step:

Configure the new step as follows:

• Runner type: Command line
• Step name: Jest JS tests
• Execute step: If all previous steps finished successfully (or anything else you prefer)
• Working directory: name of the catalog in your web app where you normally run npm test
• Run: Custom script
• Custom script: “your_path_to_nodejs\npm.cmd” run test, for example: “C:\Dev\nodejs\npm.cmd” run test
• Format stderr output as: error

If everything is configured correctly, during the next build you should see jest tests output recognized and nicely reported by TeamCity:

If that’s not the case, make sure that TEAMCITY_VERSION environment variable is set on your TeamCity machine. That’s how jest-teamcity reporter recognizes when to output standard jest output and when to use TeamCity’s service messages format.

Debugging jest tests on TeamCity locally

If you run npm test locally, you will see that the output is still normal, not the TeamCity format. That’s correct, because as we said before, the TEAMCITY_VERSION environment variable must exist in order to see TeamCity-formatted output. Normally we don’t want to see TeamCity-formatted tests output locally, of course ?

It means that if you want to debug this configuration locally and actually see the TC tests output on your dev machine, you need to define such environment variable locally.

In order to do that in Windows, go to Edit the system environment variables, click on Evironment Variables... and add a new one in System variables section. Call this variable TEAMCITY_VERSION and give it some value, for example 2021.1:

If you’re on another operating system, add the environment variable according to your system’s requirements.

Next, restart Visual Studio Code or the console tool you are using and execute npm test again. Now you should see the tests results output in TeamCity format:

Jest tests in TeamCity – summary

I hope you found this piece of advice useful. At Yumasoft, we value tests a lot, both backend and frontend ones. However, the tests which are not run automatically are worth nothing. That’s why we always have all of our tests run automatically as part of our CI process (TeamCity in the sample project’s case).

The post How To Configure Jest Tests on TeamCity appeared first on Yumasoft.