[Scons-users] best practices: unit tests

Mon Feb 19 12:06:00 EST 2018

Sorry for coming in late in this.
I had made a UnitTest() functions for Scons via an extension I made. It defined a run_utest and utest targets. it would build some code as a test map it to an alias. it would also create a run sandbox to run the test in so it was easy to map custom data as input for the test run. What was nice about this was that with Scons it was a side effect that running the unit tests only the test that had source changes would rebuild and run. this was very nice for larger projects.
It works well and I have someone using it in very clever ways. I think the core part of any "test" API added to SCons needs to be that it is generic. there are many different test frameworks out there. I think that making sure people can use gtest, tut, catch, my custom setup...  is a critical component to having a good unit test API for a build system. I think it is going too far and making built-in dashboards, etc. They are nice.. but that should be independent systems from a build system.
Jason
________________________________
From: Scons-users <scons-users-bounces at scons.org> on behalf of Bill Deegan <bill at baddogconsulting.com>
Sent: Monday, February 19, 2018 9:42 AM
To: SCons users mailing list
Subject: Re: [Scons-users] best practices: unit tests

Would someone like to summarize this email thread an publish in the wiki?

On Mon, Feb 19, 2018 at 3:29 AM, Hill, Steve (FP COM) <Steve.Hill at cobham.com<mailto:Steve.Hill at cobham.com>> wrote:

I’ve fully integrated our unit-tests (and module tests) into SCons, though as a separate build. The test suites get built and installed into a well-known directory and they are run (by SCons) from there. This means that a) only those unit-tests affected by a given change-set get rerun – but all those affected do get rebuilt and rerun, and b) the test-suites are easy to find if someone wants to run one manually for debugging, for instance.

S.

From: Scons-users [mailto:scons-users-bounces at scons.org<mailto:scons-users-bounces at scons.org>] On Behalf Of Gary Oberbrunner
Sent: 18 February 2018 00:19
To: SCons users mailing list
Subject: Re: [Scons-users] best practices: unit tests

I've been busy and just saw this.

 > What do people do here? Integrate the running of tests into the scons

 > builds? Keep them external?  If integrated, how?

I create a UnitTests alias that builds my unit tests, and that's part of my default build. I usually run them manually after building them with SCons; trying to have SCons run them works but then sometimes you want to re-run them without doing anything else and that gets trickier.

-- Gary

On Thu, Feb 1, 2018 at 5:05 PM, Mats Wichmann <mats at wichmann.us<mailto:mats at wichmann.us>> wrote:

I have a few "is there a Best Known Method" type questions that I'll
dribble out over a few messages (that is, if I can keep my focus, I keep
getting dragged off onto other things).

This one has several interesting hits on the internet, to my
recollection - wiki pages, blogs, stackexchange, etc. The ones I've
looked at over several months didn't seem terribly comprehensive, though.

Unit tests are a key part of many software projects these days, and
often a part of acceptance criteria in code review systems. scons itself
follows this model - I see tests running every time you submit a pull
request on github.

What do people do here? Integrate the running of tests into the scons
builds? Keep them external?  If integrated, how?  On a smaller project I
suspect it could make sense to integrate the test tightly into the
build, such that a builder doesn't even report success until the
matching unit test has also passed - does anyone actually do that?  In a
more complex project, the ordering can get confusing: e.g. if several
different directories each build a shared library and a unit test
program for one shared library may require use of others of the shared
libraries, you can't bind it that simply.  What then - e.g. if you ought
to wait for the whole project to finish building before running the
various tests?  In the case of the project I'm working on, tests are
built with googletest, which is told on invocation to generate a result
file and the CI system swings through and collects the various result
files - thus the pass/fail decision is external to scons - but the
launching of the tests is not.  Indeed, it's big and complex enough that
running the unit tests is optional, which several of the CI builds do,
but a developer might not when testing a patch initially, as it could
slow them down too much.

Again... I realize that if it works, it's not wrong, I'm just curious
for thoughts on best practices, I assume others have had to think
through this question?

_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7C0ea0f6deeb074c73b22808d577af7b00%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636546517915221310&sdata=kMLTfyND4BsLigJlR%2F%2Bh8XEkTx7JHfW7R6t72obwdcM%3D&reserved=0>

--

Gary

_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7C0ea0f6deeb074c73b22808d577af7b00%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636546517915221310&sdata=kMLTfyND4BsLigJlR%2F%2Bh8XEkTx7JHfW7R6t72obwdcM%3D&reserved=0>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20180219/d6e5b429/attachment.html>