[Scons-users] Changing VariantDir in an SConscript file

Thu Nov 18 14:17:08 EST 2021

As a note.

I see this when you have a set of recursive SConscript calls. I find as a practice to use ‘#’ in the variant directory. This has helped remove many cases in which the variantdir gets this massive recursive stricture

Jason

From: Bill Deegan<mailto:bill at baddogconsulting.com>
Sent: Thursday, November 18, 2021 1:13 PM
To: SCons users mailing list<mailto:scons-users at scons.org>
Subject: Re: [Scons-users] Changing VariantDir in an SConscript file

What is likely to fail when you set up possible infinite recursion by your use of VariantDir()?
It's possible a simple error could yield N deep levels of  build/B/B/B/B/..... with build results being scattered throughout that hierarchy. (I've seen this happen).
That's why it's not advised to have a VariantDir be a child of it's source.
Note the word advised.
If you get into trouble here it is generally painful to figure it out and resolve, so best to not use it in this way.

Nothing prevents lower level SConscript() calls from using the variant_dir arg.
There's nothing unique about your usage or desire to control variants from command line.

Generally I suggest using as plain SCons as possible, as opposed to building lots of abstractions (if possible).

It's pretty much guaranteed that the next engineer after you leave will not be as skilled as you, and the instant they start changing things will land in a world of hurt..
(I've often been the engineer who cleans up after the second engineer's "Fixes") ;)

On Wed, Nov 17, 2021 at 4:24 AM Luke Tunmer <luke.tunmer at gmail.com<mailto:luke.tunmer at gmail.com>> wrote:
Bill,

I'm not sure I understand the risk here of binding that variant dir in this way. What is likely to fail?

The problem I have with your suggested approach is it's pushing up to the parent the information that only needs to be known by this SConscript file. It's this SConscript file that knows that two envs need to be created and adorned with all kinds of env flag changes to achieve the variant switch, so it's this SConscript file that I want to encode the location of generated targets rather than leaking part of this information to the parent.

What I have done so far is create an env method CloneWithVariants:

env_A = env.CloneWithVariants(('arm', 'arm4f'), ('ctoolchain', 'gcc'))

which means that the targets introduced into this env_A need to be compiled with gcc and with the armf4 options.

This method will clone a new environment, set the VariantDir to be a collated sorted list of the variant values ("arm4f-gcc" in this example), and define $VIDR to be this same value in the env. Then the env is updated by numerous SCons files in a known location that individually define what each of these variants means.

We also formalize the idea of a set of top level variants, which are bound by arguments on the command line, which means a single call to SCons will just build one of them at a time. That is what we call one particular flavour of the entire product. I can't bind to gcc or to arm4f at the top level because I need to compile other parts of the product with llvm, or for a different ARM core, which is why we need a neat formalism to describe local variants in SConscript files deep in the tree.

Thanks again for your help.

Cheers
Luke

On Tue, 16 Nov 2021 at 14:15, Bill Deegan <bill at baddogconsulting.com<mailto:bill at baddogconsulting.com>> wrote:
Luke,

One important note, it's generally not a good idea to have a variant dir as a subdir of the source dir.
env_B.VariantDir('B', '.', duplicate=0)

In theory then:
B/B/B/B/B/B would refer to .

The preferred method is to use the variant_dir argument to SConscript.
In your example all your sources are in src, so you could place your SConscript in your src dir and then just call SConscript twice with 'src/SConscript', variant_dir='A', and 'src/SConscript', variant_dir='B'

-Bill

On Tue, Nov 16, 2021 at 5:19 AM Luke Tunmer <luke.tunmer at gmail.com<mailto:luke.tunmer at gmail.com>> wrote:
Thanks Bill, that's quite helpful.

I have done it this way (with some attempts to minimise typing), which seems to work:

Import('env')

env_A = env.Clone()
env_A.Append(VDIR='A')
env_A.Append(CCFLAGS=' -DVAR_A')
env_A.VariantDir('A', '.', duplicate=0)

env_B = env.Clone()
env_B.Append(VDIR='B')
env_B.Append(CCFLAGS=' -DVAR_B')
env_B.VariantDir('B', '.', duplicate=0)

subsystem1_A = env_A.Library('$VDIR/subsystem1_A', ['$VDIR/src/common.c', 'src/impl_A.c'])
subsystem1_B = env_B.Library('$VDIR/subsystem1_B', ['$VDIR/src/common.c', 'src/impl_B.c'])

I use an env var to make it more straightforward. The files that do not need to be built two ways can just be listed by their source location. This is actually a good practice - engineers should know which of their source files are affected by the variant (i.e. use the VAR_A or VAR_B definitions in this idealized example), and be able to mark them in the SConscript file.

I will wrap some more abstractions around this to make the idea clear to my engineers how they manage local variants within their subsystems and libraries. The bigger picture here is that these two libraries are linked into two different elf files which are targeted at two different cores within the SOC, and they should both be built in a single run of SCons. This is done in a parent SConscript.

Seems to all work, so thanks again.

Regards,
Luke

On Mon, 15 Nov 2021 at 16:38, Bill Deegan <bill at baddogconsulting.com<mailto:bill at baddogconsulting.com>> wrote:
Luke,

VariantDir()'s seem to be one of the harder to grasp concepts in SCons so you're in good company.
Assuming your Library() statements are in the same SConscript, you'd need to change them as follows

lib_A = env_A.Library('Build/A/lib_A', ['Build/A/src/common.c', 'Build/A/src/impl_A.c'])
lib_B = env_B.Library('Build/B/lib_B', ['Build/A/src/common.c', 'Build/A/src/impl_B.c'])

Just adding a VariantDir() to an Environment() doesn't do anything to any builders you use with that environment.

All you're doing with any VariantDir() statement is telling SCons that for example Build/A should be treated as if it was '.'.
So any files you reference relative to Build/A, if they don't exist in Build/A, SCons should also look in .

So for Build/A/src/common.c SCons should first look at that location, then it should look at ./src/commmon.c

Is that any clearer?
-Bill

On Mon, Nov 15, 2021 at 9:54 AM Luke Tunmer <luke.tunmer at gmail.com<mailto:luke.tunmer at gmail.com>> wrote:
Hi all,

I'm trying to understand why calling VariantDir on my environment within an SConstript file doesn't do what I thought it would. I suspect it's my understanding that is broken, and if so, I'm looking for advice for how to do this properly.

My top level SConstruct file started the ball rolling with a variant_dir that encapsulates all the different configurations that this system can be built to ensure each configuration is built into a unique build folder location (I turn duplicate off).

Deep down in the tree in a particular SConscript file I need to compile some C files two different ways which target the C compiler optimizer for the particular CPU on which it will run (the embedded system is a multicore one with at least 5 different CPU types). Some of these C files will be running on different cores.

I thought I could clone the environment two ways and modify the VariantDir of each:

Import('env')

env_A = env.Clone()
env_A.VariantDir('build/A', '.', duplicate=0)
# add various CC flags to env_A

env_B = env.Clone()
env_B.VariantDir('build/B', '.', duplicate=0)
# add different CC flags to env_B

And then I would like to be able to specify the two different libraries that need to be built at this level in the tree:

lib_A = env_A.Library('lib_A', ['src/common.c', 'src/impl_A.c'])
lib_B = env_B.Library('lib_B', ['src/common.c', 'src/impl_B.c'])

and then common.c will be compiled two ways into different build folders, and the appropriate one included in each library. However, the change to the VariantDir of each environment seems to make no difference at all: it uses the build folder specified right at the top of the tree for both the .o files and for the .lib files that it makes.

Any help in my understanding of how VariantDir is supposed to work, and what it is actually doing is appreciated. Or, of course, any information on the proper way to achieve this result would be great.

Regards,
Luke

_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=04%7C01%7C%7C2c68a0ee3a994b1df25d08d9aac7788a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637728595972524124%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=uKCagSP35l6%2F%2F8eOGzfr%2FClWdFH5FDUhPV6purj5U0o%3D&reserved=0>
_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=04%7C01%7C%7C2c68a0ee3a994b1df25d08d9aac7788a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637728595972534081%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=vUNX%2F4jKArnTuK7iLSZ8vopqPCVlTGGR1HsIbpZr3EU%3D&reserved=0>
_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=04%7C01%7C%7C2c68a0ee3a994b1df25d08d9aac7788a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637728595972544039%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=334i%2F9HIhTw7EQnpytUUGdnmHa0npaRC0jQNO%2FfXczg%3D&reserved=0>
_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=04%7C01%7C%7C2c68a0ee3a994b1df25d08d9aac7788a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637728595972553999%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=erHqW3TxcRQDVpOJaLKXUeHjtzaCQeDb9mf4ZQE288g%3D&reserved=0>
_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=04%7C01%7C%7C2c68a0ee3a994b1df25d08d9aac7788a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637728595972553999%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=erHqW3TxcRQDVpOJaLKXUeHjtzaCQeDb9mf4ZQE288g%3D&reserved=0>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20211118/575ca173/attachment-0001.htm>