wiki:DevQualityAssurance

Version 1 (modified by Christian Beer, 10 years ago) (diff)

--

Proposal: Quality Assurance workflow for BOINC

Motivation

To increase the quality and stability of all BOINC components it is useful to run automated tests and validation tools on a regular basis. This way one can quickly see if a specific change breaks another component/part of the system or introduces a possible security risk. Overall this should help to identify weak spots in the BOINC source code. Using codesniffers (like the PHP CodeSniffer) could help enforcing the BOINC coding guidelines. This has the side-effect of a syntax check and can also generate ToDo/Wish lists from comments in the code.

Requirements

This is a first shot and open for discussion:

  • automated system that runs every night
  • automatically run unit tests that are in VCS
  • automatically run third-party tools on portions of the codebase (e.g. cppcheck, doxygen)
  • compile a report of the automated runs (shown via web)
  • support multiple nodes to test on different operating systems
  • support multiple git branches (e.g. run the same tests on master as on client_release/7/7.2)
  • support testing multiple language on portions of the VCS (C++, PHP, Java)

Tools

Jenkins is a continuous integration server that is extended using plugins:

Those plugins only compile a report from each of those tools. The tools need to be installed and configured separately.

Using JUnit, PHPUnit and CppUnit should simplify the creation and maintenance of test cases.

Jenkins support distributed workers as build nodes.

problems

  • Jenkins has no built-in user authentication
  • How does Jenkins handle multi-language code bases?

Workflow

On creating test cases

To create a test case one has to know what the function/class/component is supposed to do. Especially for white box tests this is crucial. Documenting the preconditions, postconditions and invariants of functions and classes is important. The documentation should be unified in regard to the used languages (C++, PHP, Java) and the used documentation tool (doxygen). It should be possible to distinguish between public and internal APIs.