[Scons-users] SCons Tools maintenance

Luke Tunmer luke.tunmer at gmail.com
Thu Nov 9 11:17:52 EST 2017 

Sadly, no. Getting software out of our organization into Open Source can be
quite challenging. However I can describe the general idea.

I've created a package called SConsVariants which downloadable from our
devpi server. At the top of the SConstruct file, we expect the boilerplate
code:

from SConsVariants import basic_config
env = basic_config(env_vars_to_import=['LM_LICENSE_FILE', 'USERPROFILE',
'ARMLMD_LICENSE_FILE',
                                       'OS', 'PYPI_MIRROR'],
                   variants=dict(style=('dbg', 'rel'),
                                 ctoolchain=('msvc', 'mingw'),
                                 ...
                                 )

and then the returned env is the root env from which all others are cloned.
It gets handed down into the next level of SConscript.py files.

Within the basic_config function I:

   - Create a default env without any of the built-in tools (see the
   previous discussion wrt not looking for the MSVC tools):

   SCons.Defaults.DefaultEnvironment(tools=[])

   - Create an env with only the listed env-vars imported from shell
   environment, no others.
   - Use the following utility to find all the installed packages that are
   relevant to scons:

   def _find_site_scons_from_plugins(env):
       """This function scans all installed packages looking for those that
       have declared an entry point ``site_scons``. The entry point is
       called, and this should return a path to the site_scons folder in
       that plugin.
       """
       site_scons_folders = []
       for entry_point in
   pkg_resources.iter_entry_points(group='site_scons'):
           entry_point = entry_point.load()
           logging.info("Calling %s to find site_scons",
   entry_point.__name__)
           site_scons_folders.append(entry_point(env))
       return site_scons_folders

   - Modify the SCons.Tool.DefaultToolpath with these returned paths.
   - Loads a bunch of scons fragments based on the variants list defined in
   the call to basic_config. It's in these files that tools are loaded if
   needed. In this example there should be a file on one of the returned paths
   called "ctoolchain-mingw-env.py" that will get loaded, and in there the
   "ming" tool is installed into that top-level environment.

I wanted to reduce the amount of boilerplate in each SConstruct file (and
we have a lot of them in each of our numerous components), and put that
boilerplate into somewhere common.

There are a few other things that I do in this package, but this is the key
benefit. Apart from this call to basic_config, the only other thing that
the top-level SConstruct file has to do is call SConscript:

SConscript('SConscript.py',
           exports=['env'],
           variant_dir=os.path.join('builds', env['VARIANTS']),
           duplicate=0)

(I like separating object files from source files and not copying source
files to the build folder because if confuses IDEs).

How that explanation helps.

Regards,

Luke







On 9 November 2017 at 14:37, Bill Deegan <bill at baddogconsulting.com> wrote:

> Luke,
>
> That sounds very interesting!
> Is the code in a public repo so we can take a look at it?
>
> -Bill
> SCons Project Co-Manager
>
> On Thu, Nov 9, 2017 at 7:34 AM, Luke Tunmer <luke.tunmer at gmail.com> wrote:
>
>> I have developed a regular Python package that when imported into an
>> SConstuct file, looks for other SCons Python packages using the
>> entry_points mechanism of setup tools. It then calls each of these entry
>> points to get the Tools path that will allow tools to be found when they
>> are added to an environment. This means that these extensions just have to
>> be installed into the virtualenv from which the scons is running. This is
>> done by having a requirements.txt file that lists all the SCons extensions
>> tools needed for the product or component.
>>
>> Then we have an internal devpi server which allows these SCons extension
>> packages to be found.
>>
>> I would like one day to see SCons directly support the idea of
>> entry_point extensions, which seems to used increasingly as a means to
>> introduce plugins into larger frameworks.
>>
>> Regards
>> Luke
>>
>>
>> On 9 November 2017 at 11:33, Nacho Piqueras <natxo.piq at gmail.com> wrote:
>>
>>> Hello all,
>>>
>>> Even if it is a not very much documented feature, I find SCons' Tools
>>> one of the most powerful concepts to arrange a build that involves several
>>> code transformations. In our projects we use a lot of 3rd party code
>>> generators and we wrap them into SCons tools to bring their functionality
>>> to the build environment.
>>>
>>> The problem we face is that tools packages are distributed per project.
>>> Currently all SCons tools that are required for a specific project are
>>> included into a site-tools folder inside that project. Being this way, we
>>> have the source code distribution problem: sometimes the code for a tool
>>> drifts from one project to another and it is difficult to track
>>> bugs/improvements through projects.
>>>
>>> Because scons tools are really coupled with the tool they wrap we could
>>> try to distribute the python package with the tool (the code generator),
>>> but this is not always possible. Fresh install of a tool does not bring the
>>> scons tool with it.
>>>
>>> Also, we would like to unit test the code of a tool. This involves
>>> mocking a scons environment somehow... for at least testing that the action
>>> would be correctly called, or the sequence of actions correctly arranged
>>> (scons tools' code often define scanners, emitters...). I still have not
>>> figured out how to do it.
>>>
>>> I would like to know how people using SCons distribute their SCons'
>>> tools code, how do you deal with this situation.
>>>
>>> For example, maybe we could package scons tools and install them with
>>> pip into a virtualenv created for a project...
>>>
>>> Many thanks for your thoughts on this question and for keeping SCons
>>> evolving.
>>>
>>>
>>> _______________________________________________
>>> Scons-users mailing list
>>> Scons-users at scons.org
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20171109/31587aa9/attachment.html>

More information about the Scons-users mailing list