[Scons-users] persistent but local tweaks to a node's environment

[Bill Deegan]

Your solution sounds unnecessarily overcomplicated.

Let me see if I understand what you're trying to do. (It's been a few days
since I last engaged in your issue).
1.  Some target files require individually different CCFLAGS (in addition
to a base set of flags) and/or OBJSUFFIX.
2. You build various flavors (debug, optimized,etc)

For #1, how do you keep track of what flags/OBJSUFFIX each file requires?
Also are there any other SCons variables which need to vary per file?

For #2, are you using variant dirs? (If not, why?)

BTW. You should not use CCFLAGS for -Dxyz=1, that should be in CPPDEFINES.



On Fri, Jul 30, 2021 at 5:59 PM Gabe Black <gabe.black at gmail.com> wrote:

> Hello again! I've had a chance to try implementing the idea in this toy
> example into our actual build scripts, and while it got me closer, I'm
> still running into problems.
>
> The way our scripts are structured is something roughly like this:
>
> main script:
> find SConscripts in subdirectories and parse them
>
>     sub script(s):
>     declare sources, mark some as part of special categories, or needing
> environment tweaks,
>     declare some unit tests which may include sources (by category) that
> haven't been declared yet
>
> after processing subdirectories...
> build environments for debug, optimized, profiled, etc, builds
> in each of those environments, set up all top level targets (main
> binaries, unit tests, library version of project, etc)
>
>
> To set up each of those build environments, we use Clone(), and then
> customize them with different flags, different object file or binary
> suffixes, etc. Unfortunately it looks like this Clone()ing step breaks the
> mechanism that worked in the toy example.
>
> A possible solution I've considered is that we might want to use
> Override() and not Clone(), because as somebody mentioned, that creates an
> overlay and does not completely copy and detach from the underlying
> environment. This is not ideal given our current structure, since
> Override() is not really a public mechanism, and if something down the line
> building up an environment either globally, or (for instance) to use for
> each of the unit tests tries to use Clone() itself, things will blow up.
>
> So then I thought about how I might use Override(), but without reaching
> down into SCons's guts myself and using it directly.
>
> My thought thus far is that a builder can call other builders implicitly,
> if it needs to build one of its sources, and it was set up with that other
> builder as a source builder. Would it work to have, say, a pseudo top level
> builder which builds some generic, internal only node (an Alias with a
> weird name maybe), and then for each of its sources to imply a sub builder?
> Then that sub builder would (I think) inherit the environment from the top
> level builder, including any overrides. Those sources could be another
> internal only node like "unit tests" or "libraries" or "main binaries", and
> then each of those could in turn customize the environment further and get
> down to the actual "Program" or "StaticLibrary" or ... builder which
> actually builds an output.
>
> fast_environment.EveryThingBuilder(fast_override1="$BLAH $BLAH",
> fast_override2="$FOO $BAR")
> opt_environment.EveryThingBuilder(opt_override1=...)
> debug_environment.EverythingBuilder(debug_override1=...)
>
> Importantly, this imaginary top level target would never be something the
> user would actually build. It would just teach scons about a bunch of
> targets by implying them as sources (would this actually work?)
>
> Another nice benefit of this approach is that the source files could be
> built up in some sort of construction variable as the SConscripts are read
> in, and since these are not actually consumed until all these implicit
> builders get called, we wouldn't have to worry about the case where a
> binary says they need all sources having to do with the built in python
> interpreter, but those haven't all be declared yet.
>
> I feel like there's a solution in there, but I'm not sure if all my
> assumptions are sound, and I've gotten a bit lost in the details. Can
> someone please confirm whether this is something that even *could* work,
> and if so help me rough out what pieces to use?
>
> Thanks so much for your help!
>
> Gabe
>
> On Sun, Jul 25, 2021 at 8:05 AM Gabe Black <gabe.black at gmail.com> wrote:
>
>> Sure. Here is an example SConstruct I was playing with:
>>
>> '''
>> env = Environment()
>>
>> foo_o = env.Object('foo.c', CCFLAGS='${CCFLAGS} -DFOO=foo')
>>
>> env.Append(CCFLAGS=['-DBAR=bar'])
>> foo = env.Program('foo', foo_o)
>>
>> Default(foo)
>> '''
>>
>> When that builds foo.o, it always uses -DFOO=foo, and never -DFOO=bar, so
>> the env.Append line seems to be clobbered by the override in env.Object().
>>
>> Thinking about what you said though, I tried a little experiment which
>> seemed to work more like what I would expect
>>
>> '''
>> env = Environment()
>>
>> env.Append(CCFLAGS=['${CCFLAGS_extra}'])
>> foo_o = env.Object('foo.c', CCFLAGS_extra='-DFOO=foo')
>>
>> env.Append(CCFLAGS=['-DBAR=bar'])
>> foo = env.Program('foo', foo_o)
>>
>> Default(foo)
>> '''
>>
>> With that I get both -DFOO=bar and -DFOO=foo. I can probably work with
>> that since everything I'd need to add is local to env.Object, and I might
>> even be able to wrap it in something to make it a little less cumbersome.
>>
>> Related to this, I know you can override a variable using FOO='bar' in a
>> builder, but is there a way to Append? I usually want to, for instance, add
>> extra compiler flags without clobbering everything that's already there.
>> That's what I was *hoping* to achieve with the CCFLAGS='${CCFLAGS} ...',
>> but I don't think that's what it ended up doing!
>>
>> Gabe
>>
>> On Sun, Jul 25, 2021 at 6:52 AM Mats Wichmann <mats at wichmann.us> wrote:
>>
>>> On 7/24/21 4:18 AM, Gabe Black wrote:
>>> > Hi! The project I work on builds several different versions of its
>>> > outputs, like a debug version, an optimized version, etc. Also, some
>>> > files in the project, for instance automatically generated files, need
>>> > to have slightly customized build flags, like disabling certain
>>> warnings
>>> > that are false positives and/or are in source we can't change.
>>> >
>>> > Right now we handle that by creating an environment for each build,
>>> > which I think is the standard approach. Unfortunately, we then have to
>>> > have a separate step which goes through and generates new environments
>>> > based on those for each file that needs its slightly customized build
>>> flags.
>>> >
>>> > What I would *like* to do, is to be able to specify for any given node
>>> > that it should build using the flags that would be implied by the
>>> > environment that's pulling it in, except that they should be tweaked
>>> > just a little bit. Something like:
>>> >
>>> > Object('foo.cc', '-Wno-deprecated-copy')
>>> >
>>> > and then:
>>> >
>>> > opt.Program('foo.opt', 'foo.cc', CCFLAGS='${CCFLAGS} -O3')
>>> > debug.Program('foo.debug', 'foo.cc', CCFLAGS='${CCFLAGS} -g',
>>> > OBJSUFFIX='.do')
>>> > etc
>>> >
>>> > and then have foo.do build with "-Wno-deprecated-copy -g", and foo.o
>>> > build with "-Wno-deprecated-copy -O3"
>>> >
>>> > The problem is, that as soon as I tell SCons to use custom variables,
>>> it
>>> > Clone-s the underlying environment and disassociates from the
>>> underlying
>>> > environment.
>>>
>>> could you explain what you're seeing a little further?
>>>
>>> In the case of construction variable overrides in a builder call, it
>>> doesn't Clone, as that's considered expensive, it just builds an
>>> override dictionary, and it doesn't disassociate - it retains the
>>> underlying construction environment and only applies the override dict
>>> if there were any overrides - so it's supposed to be a kind of
>>> copy-on-write setup.  Since this doesn't seem to match what you're
>>> seeing it would be good to figure out what's actually happening.
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Scons-users mailing list
>>> Scons-users at scons.org
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20210731/788735af/attachment.htm>