Development Tasks / Testing

Automating software testing is an important software development task because it helps ensure that software performs as intended. It is particularly important to ensure that a change in one part of the code does not break another part of the code, and that changes in the computer environment don't break the software. Automated testing has not been a large part of StateCU software development in the past and needs to be added.

This documentation contains the following sections:

Test Code Using Unit Tests

Unit tests are compiled code the test an atomic unit of code. Unit tests are a tool for software developers that require access to the develop environment, unlike functional tests, that could be run by users without the full developer environment.

There does not seem to be extensive unit test frameworks for Fortran. See:

A place-holder folder src/test has been created in the repository and src/test/fortran could be used for unit tests.

Need to evaluate Fortran unit tests for automated testing - how does it fit into this project?

Test StateCU by Running Reference Dataset

Need to work with WWG to help explain which dataset should be run and how to check results. Need to put the dataset somewhere on the web for developers to download. Perhaps also explain how to use tools like KDiff3.

Automated Functional Testing Using pytest

A place-holder folder src/test has been created in the repository and src/test/pytest could be used for pytest tests.

pytest may be a way to run functional tests, for example run StateCU from the command line on simple datasets and compare output with expected output. Need to decide whether to prototype automated testing with pytest or try some other automation tool.

Automated Functional Testing Using TSTool

TSTool software can be used to implement automated tests and this approach is used to test TSTool itself. The benefit of using TSTool is that it can read StateCU input and output time series files and can do time series comparison using a precision. Therefore if small, allowable, differences are seen, TSTool can treat as a pass, whereas a simple file comparison would show a fail.

A place-holder folder src/test has been created in the repository and src/test/tstool could be used for TSTool tests.

Need to decide whether to prototype this approach.