[Scons-users] Changing VariantDir in an SConscript file

[Bill Deegan]

What is likely to fail when you set up possible infinite recursion by your
use of VariantDir()?
It's possible a simple error could yield N deep levels of
build/B/B/B/B/..... with build results being scattered throughout that
hierarchy. (I've seen this happen).
That's why it's not advised to have a VariantDir be a child of it's source.
Note the word advised.
If you get into trouble here it is generally painful to figure it out and
resolve, so best to not use it in this way.


Nothing prevents lower level SConscript() calls from using the variant_dir
arg.
There's nothing unique about your usage or desire to control variants from
command line.

Generally I suggest using as plain SCons as possible, as opposed to
building lots of abstractions (if possible).

It's pretty much guaranteed that the next engineer after you leave will not
be as skilled as you, and the instant they start changing things will land
in a world of hurt..
(I've often been the engineer who cleans up after the second engineer's
"Fixes") ;)

On Wed, Nov 17, 2021 at 4:24 AM Luke Tunmer <luke.tunmer at gmail.com> wrote:

> Bill,
>
> I'm not sure I understand the risk here of binding that variant dir in
> this way. What is likely to fail?
>
> The problem I have with your suggested approach is it's pushing up to the
> parent the information that only needs to be known by this SConscript file.
> It's this SConscript file that knows that two envs need to be created and
> adorned with all kinds of env flag changes to achieve the variant switch,
> so it's this SConscript file that I want to encode the location of
> generated targets rather than leaking part of this information to the
> parent.
>
> What I have done so far is create an env method CloneWithVariants:
>
> env_A = env.CloneWithVariants(('arm', 'arm4f'), ('ctoolchain', 'gcc'))
>
> which means that the targets introduced into this env_A need to be
> compiled with gcc and with the armf4 options.
>
> This method will clone a new environment, set the VariantDir to be a
> collated sorted list of the variant values ("arm4f-gcc" in this example),
> and define $VIDR to be this same value in the env. Then the env is updated
> by numerous SCons files in a known location that individually define what
> each of these variants means.
>
> We also formalize the idea of a set of *top level* variants, which are
> bound by arguments on the command line, which means a single call to SCons
> will just build one of them at a time. That is what we call one particular
> *flavour* of the entire product. I can't bind to gcc or to arm4f at the
> top level because I need to compile other parts of the product with llvm,
> or for a different ARM core, which is why we need a neat formalism to
> describe *local variants *in SConscript files deep in the tree.
>
> Thanks again for your help.
>
> Cheers
> Luke
>
>
> On Tue, 16 Nov 2021 at 14:15, Bill Deegan <bill at baddogconsulting.com>
> wrote:
>
>> Luke,
>>
>> One important note, it's generally not a good idea to have a variant dir
>> as a subdir of the source dir.
>> env_B.VariantDir('B', '.', duplicate=0)
>>
>> In theory then:
>> B/B/B/B/B/B would refer to .
>>
>> The preferred method is to use the variant_dir argument to SConscript.
>> In your example all your sources are in src, so you could place your
>> SConscript in your src dir and then just call SConscript twice with
>> 'src/SConscript', variant_dir='A', and 'src/SConscript', variant_dir='B'
>>
>> -Bill
>>
>> On Tue, Nov 16, 2021 at 5:19 AM Luke Tunmer <luke.tunmer at gmail.com>
>> wrote:
>>
>>> Thanks Bill, that's quite helpful.
>>>
>>> I have done it this way (with some attempts to minimise typing), which
>>> seems to work:
>>>
>>> Import('env')
>>>
>>> env_A = env.Clone()
>>> env_A.Append(VDIR='A')
>>> env_A.Append(CCFLAGS=' -DVAR_A')
>>> env_A.VariantDir('A', '.', duplicate=0)
>>>
>>> env_B = env.Clone()
>>> env_B.Append(VDIR='B')
>>> env_B.Append(CCFLAGS=' -DVAR_B')
>>> env_B.VariantDir('B', '.', duplicate=0)
>>>
>>> subsystem1_A = env_A.Library('$VDIR/subsystem1_A',
>>> ['$VDIR/src/common.c', 'src/impl_A.c'])
>>> subsystem1_B = env_B.Library('$VDIR/subsystem1_B',
>>> ['$VDIR/src/common.c', 'src/impl_B.c'])
>>>
>>> I use an env var to make it more straightforward. The files that do not
>>> need to be built two ways can just be listed by their source location. This
>>> is actually a good practice - engineers should know which of their source
>>> files are affected by the variant (i.e. use the VAR_A or VAR_B definitions
>>> in this idealized example), and be able to mark them in the SConscript file.
>>>
>>> I will wrap some more abstractions around this to make the idea clear to
>>> my engineers how they manage local variants within their subsystems and
>>> libraries. The bigger picture here is that these two libraries are linked
>>> into two different elf files which are targeted at two different cores
>>> within the SOC, and they should both be built in a single run of SCons.
>>> This is done in a parent SConscript.
>>>
>>> Seems to all work, so thanks again.
>>>
>>> Regards,
>>> Luke
>>>
>>>
>>> On Mon, 15 Nov 2021 at 16:38, Bill Deegan <bill at baddogconsulting.com>
>>> wrote:
>>>
>>>> Luke,
>>>>
>>>> VariantDir()'s seem to be one of the harder to grasp concepts in SCons
>>>> so you're in good company.
>>>> Assuming your Library() statements are in the same SConscript, you'd
>>>> need to change them as follows
>>>>
>>>> lib_A = env_A.Library('Build/A/lib_A', ['Build/A/src/common.c', '
>>>> Build/A/src/impl_A.c'])
>>>> lib_B = env_B.Library('Build/B/lib_B', ['Build/A/src/common.c', '
>>>> Build/A/src/impl_B.c'])
>>>>
>>>> Just adding a VariantDir() to an Environment() doesn't do anything to
>>>> any builders you use with that environment.
>>>>
>>>> All you're doing with any VariantDir() statement is telling SCons that
>>>> for example Build/A should be treated as if it was '.'.
>>>> So any files you reference relative to Build/A, if they don't exist in
>>>> Build/A, SCons should also look in .
>>>>
>>>> So for Build/A/src/common.c SCons should first look at that location,
>>>> then it should look at ./src/commmon.c
>>>>
>>>> Is that any clearer?
>>>> -Bill
>>>>
>>>>
>>>> On Mon, Nov 15, 2021 at 9:54 AM Luke Tunmer <luke.tunmer at gmail.com>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I'm trying to understand why calling VariantDir on my environment
>>>>> within an SConstript file doesn't do what I thought it would. I suspect
>>>>> it's my understanding that is broken, and if so, I'm looking for advice for
>>>>> how to do this properly.
>>>>>
>>>>> My top level SConstruct file started the ball rolling with a
>>>>> variant_dir that encapsulates all the different configurations that this
>>>>> system can be built to ensure each configuration is built into a unique
>>>>> build folder location (I turn duplicate off).
>>>>>
>>>>> Deep down in the tree in a particular SConscript file I need to
>>>>> compile some C files two different ways which target the C compiler
>>>>> optimizer for the particular CPU on which it will run (the embedded system
>>>>> is a multicore one with at least 5 different CPU types). Some of these C
>>>>> files will be running on different cores.
>>>>>
>>>>> I thought I could clone the environment two ways and modify the
>>>>> VariantDir of each:
>>>>>
>>>>> Import('env')
>>>>>
>>>>> env_A = env.Clone()
>>>>> env_A.VariantDir('build/A', '.', duplicate=0)
>>>>> # add various CC flags to env_A
>>>>>
>>>>> env_B = env.Clone()
>>>>> env_B.VariantDir('build/B', '.', duplicate=0)
>>>>> # add different CC flags to env_B
>>>>>
>>>>> And then I would like to be able to specify the two different
>>>>> libraries that need to be built at this level in the tree:
>>>>>
>>>>> lib_A = env_A.Library('lib_A', ['src/common.c', 'src/impl_A.c'])
>>>>> lib_B = env_B.Library('lib_B', ['src/common.c', 'src/impl_B.c'])
>>>>>
>>>>> and then common.c will be compiled two ways into different build
>>>>> folders, and the appropriate one included in each library. However, the
>>>>> change to the VariantDir of each environment seems to make no difference at
>>>>> all: it uses the build folder specified right at the top of the tree for
>>>>> both the .o files and for the .lib files that it makes.
>>>>>
>>>>> Any help in my understanding of how VariantDir is supposed to work,
>>>>> and what it is actually doing is appreciated. Or, of course, any
>>>>> information on the proper way to achieve this result would be great.
>>>>>
>>>>> Regards,
>>>>> Luke
>>>>>
>>>>> _______________________________________________
>>>>> Scons-users mailing list
>>>>> Scons-users at scons.org
>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>
>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>> _______________________________________________
>>> Scons-users mailing list
>>> Scons-users at scons.org
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20211118/1c6f416c/attachment.htm>