In order to validate code generators, the code generator testing framework provides an extensible set of tests, with currently 140 tests. Those are ThingML models of programs from which expected outputs (oracle) are provided. As much as possible those model demonstrate one specific aspect of the ThingML language. Therefore, the framework is able to test a compiler, and check that these models are indeed transformed into code that produces the expected outputs. By comparing the outputs of different compilers, it is also possible to detect possible misalignment in the semantics implemented by these different compilers. The testing framework can be simply extended by adding new test models.

To accelerate the execution of the test suite 140 x number of compilers, the execution of the tests is parallelized so as to use all the available core on your local PC. An alternative setup leverages Docker so as to deploy tests on any number of nodes e.g. in the cloud.