Unit testing in ApexSQL tools / add-ins

What is unit testing

Unit testing is a software testing method where individual functional source code units are tested to determine if each unit (i.e. procedure, function) works the way it was intended to.

A unit test is a software method that instantiates a small portion of the final software product functionality to check and verify its behavior.

A common unit test contains three phases:

1. It initializes one piece of the software product. This is also known as SUT (system under test)
2. It applies data or an event which invokes specific functionality of the SUT
3. Finally, it observes the results

If the code being tested provides expected results then the unit test will pass, otherwise it will fail and will provide information that there is a problem inside the tested code.

Those three phases listed above are known as triple A in programming (Arrange, Act and Assert).

Why write unit tests

Writing unit tests enables agile software development as developers are now required to deliver new software releases frequently – we are aiming for a monthly and eventually weekly release cycles.

By having good unit test code coverage, we can support agile software development by automatically preventing software product quality regression and ensuring that each subsequent release is overall better than the previous one.

Overall benefits for the end product are:

1. Less bugs on start for all new functionality
2. Lower chances of introducing functionality regression in subsequent releases when new product code is written
3. Testers can invest their time on acceptance and real-life testing instead of manual regression testing

ApexSQL tools unit test coverage initiative

Our goal is to have over 80% of code covered with unit tests and currently majority of our shared code between products (shared engines, components) complies with this requirement.

All our newly written shared components and engines have a mandatory requirement to be at least 80% covered with unit tests.

There are still some older components and products that are behind the 80% goal and are now receiving needed attention.

ApexSQL Source Control is one of the products where the code base is retroactively being covered with unit tests as it shares basic functionality with all ApexSQL source control-aware tools: ApexSQL Build, ApexSQL Data Diff, ApexSQL DevOps toolkit, ApexSQL Diff, ApexSQL Model, ApexSQL Script

Various 3^rd party source control systems are being frequently updated and some of those updates can break backwards compatibility. By having unit test code coverage we are able to quickly identify new issues, to isolate them in the code and to quickly deliver a fix to production.

I.e. recently Team Foundation Server (TFS) and Visual Studio Team System (VSTS) were rebranded as Azure DevOps Server and Azure DevOps Services which caused a couple of new bugs. By having unit test code coverage of these source control systems we can now quickly catch future compatibility issues and quickly release a patch.

ApexSQL Source Control unit test code coverage phases and goals

In order to reach the 80%-unit test code coverage goal for ApexSQL Source Control we’ve divided the work into several milestones to fully cover logical parts of the product:

1. Fix existing obsolete unit tests and make them all actual – by doing so we clean up technical debt and ensure that all existing unit tests are current and functional: done
2. Subversion: done
3. Azure DevOps Server and Azure DevOps Services (formerly Team Foundation Server and Visual Studio Team Services): done
4. Git: in progress
5. Mercurial
6. Perforce
7. Shared functionality with all ApexSQL source control-aware tools
8. Static data
9. Working folder

Currently we have over 170 unit tests written and we estimate that we’ll have over 800 when everything is finished which is currently estimated for 2019 Q2.