[Scons-users] SCons Tools maintenance

[Paweł Tomulik]

W dniu 09.11.2017 o 12:33, Nacho Piqueras pisze:
> Hello all,
> 
> Even if it is a not very much documented feature, I find SCons' Tools
> one of the most powerful concepts to arrange a build that involves
> several code transformations. In our projects we use a lot of 3rd party
> code generators and we wrap them into SCons tools to bring their
> functionality to the build environment.
> 
> The problem we face is that tools packages are distributed per project.
> Currently all SCons tools that are required for a specific project are
> included into a site-tools folder inside that project. Being this way,
> we have the source code distribution problem: sometimes the code for a
> tool drifts from one project to another and it is difficult to track
> bugs/improvements through projects.
> 
> Because scons tools are really coupled with the tool they wrap we could
> try to distribute the python package with the tool (the code generator),
> but this is not always possible. Fresh install of a tool does not bring
> the scons tool with it.
> 
> Also, we would like to unit test the code of a tool. This involves
> mocking a scons environment somehow... for at least testing that the
> action would be correctly called, or the sequence of actions correctly
> arranged (scons tools' code often define scanners, emitters...). I still
> have not figured out how to do it.
> 
> I would like to know how people using SCons distribute their SCons'
> tools code, how do you deal with this situation.
> 
> For example, maybe we could package scons tools and install them with
> pip into a virtualenv created for a project...
> 
> Many thanks for your thoughts on this question and for keeping SCons
> evolving.
> 
> 
A simple approach I use is to keep each tool in its own online repo
(github). Next, my projects, which require these custom tools, are
equipped with a kind of "bootstrap" scripts. The bootstrap downloads all
required tools from github to the site_tools directory. These
downloadable tools are .gitignored from the project (sometimes I
.gitignore whole site_tools/ dir), so they're prevented from being
committed into project.

The bootstrap is usually ran by developer just after cloning project's
repository.

In my experience placing custom tools in project's site_tools is useful
and convenient. At some time it may happen that two projects will
require same tool but in different version, in which case installing
tools "system wide" becomes quite useless. Tool versioning may be easily
handled by using git tags.

Downloading tools per-project also enables new developers to jump in
quickly into new project, as they don't have to prepare their
environments manually by searching and installing tools one by one. It's
just about clone + bootstrap. Furthermore, they don't have to clutter
their environments globally with SCons tools just because of a single
project they started to work on, and there is no need for cleaning it up
when a dev stops working on the project (tools disappear together with
the deleted project's source tree).

Tool projects (as opposed to projects that use tools) can also have a
kind of downloader scripts. In my tool projects I used to automate
downloading SCons testing frameworks for end-to-end testing. Unit tests
(in the strict sense) are rarely seen for SCons tools, but if there is a
need, the standard python 'unittest' module may be used without any
special effort.


Finally, a similar approach (automated downloading of dependencies
per-project) became popular in PHP world (composer), JavaScript/NodeJS
(npm), ruby (bundler). Virtually all dependencies (runtime as well as
development) are downloaded locally into a project. They just have nice
and powerful machinery for doing this.

Best regards!

-- 
Pawel Tomulik