[Scons-users] Problems with SConscript files using variant_dir and duplicate=False
Bill Deegan
bill at baddogconsulting.com
Fri May 22 17:15:21 EDT 2020
I'm 90% sure you've overcomplicated your solution.
I'll take a look over the weekend and get back to you.
Generally.. the SConstruct should be at the top, and sub projects should be
below it.
You can cd into any sub dir and run "scons -u ." which will walk up the
tree find the SConstruct initialize all the logic including reading
subordinate SConscript, and then only build the current directories targets.
Barring that you can create aliases to limit what is built.
As I said before.. if you're using srcnode() in a SConscript/SConstruct and
it's not in the guts of a complicated builder, you're doing something wrong.
Also doing a variant dir in such a way the variant can include the current
dir (think recursion), is a bad idea.
Also this:
build_env = Environment(tools=['gcc', 'as', 'ar', 'gnulink'])
build_env.PrependENVPath('PATH', 'D:/Tools/Gcc-7.3.0-32/bin')
Is probably not ideal. You should set the PATH before you initialize the
tools.
build_env = Environment(tools=[])
build_env.PrependENVPath('PATH', 'D:/Tools/Gcc-7.3.0-32/bin')
for t in ['gcc', 'as', 'ar', 'gnulink']:
build_env.Tool(t)
That way if there's something specific about the binary used by the tool
which changes the way SCons is initialized when you load the tool, it will
be referencing the correct one. (and in some cases will actually configure
the tool if it's not findable via the default env['ENV']['PATH'] or some
other defaulted path)
-Bill
On Fri, May 22, 2020 at 1:29 PM Hans-Christian Wild <hchr.wild at gmail.com>
wrote:
> Hi, to show off what I am actually after I have created a second example
> showing a different and maybe easier to understand usecase. But the exposed
> 'problems' and the style to structure the build system using SConscript
> files is exactly the same in both repos.
>
> https://github.com/hchrwild/project01/tree/v0.1
>
> However I the second usecase, I hope that it is much easier to understand
> why this top-level SConscript file exists in the first place. The two
> examples are closely related to each other: The scons01 project shows a
> possible structure how 'mylib' is developed in a dedicated repo, together
> with some examples for testing and unit tests (not included in the demo)
> etc.. . The second example shows mylib being used in a microcontroller
> project.
>
> The second example has a SConscruct at top-level, "composing" the build.
> There are two secondary SConscript files, one knowing how a "bootloader" is
> sonstructed, one building a "firmware". These two SConscript relate
> conceptually 1:1 to the "example" SConstruct from the first example
> project. There is a ternary SConscript file library/mylib/SConscript which
> 1:1 corresponds to the top-level SConscript of the first example.
>
> I hope that this makes clearer why there is a top-level SConscript in the
> first example in the first place: When the library is used (think of
> library/mylib being a subtree/submodule copy of the first example, with
> examples etc. being stripped/there but not used in any fashion). When
> somebody uses the library he can immediately use it in the projects for it
> comes preconfigured in a SConscript file, allowing it to be included
> seamlessly in the project.
>
> Back to the example: All three software partitions (bootloader0/1 and
> firmware) use the same mylib, but compile it each in a different variant.
> As example they just inject a global string via a cppdefine APP_NAME, which
> is really the stupidest example I could come up with...
>
> *Summary:*
> /SConscructs calls /{bootloader,firmware}/SConscripts to generate some
> executables.
> /{bootloader,firmware}/SConscripts call /library/mylib/SConscript because
> they all need their dedicated version of mylib (eg. different defines,
> target architecture, cflags, ....).
> All outputs artifacts are to be placed in /output-*
> /library/mylib/SConscript is there as convinience, hiding most of the
> details how mylib is sconstructed.
>
> *The example again should work out of the box, provided there is a gcc in
> path:*
>
> $ scons run-all
> scons: Reading SConscript files ...
> scons: done reading SConscript files.
> scons: Building targets ...
> gcc -o output-bl0\src\main.obj -c -Ilibrary\mylib\include
> output-bl0\src\main.c
> gcc -o output-bl0\mylib\src\AppName.obj -c -DAPP_NAME="Bootloader0"
> -Ioutput-bl0\mylib\include -Ioutput-bl0\mylib\src\include
> output-bl0\mylib\src\AppName.c
> ar rc output-bl0\mylib\libmylib.a output-bl0\mylib\src\AppName.obj
> gcc -o output-bl0\bootloader.exe output-bl0\src\main.obj
> output-bl0\mylib\libmylib.a
> gcc -o output-bl1\src\main.obj -c -Ilibrary\mylib\include
> output-bl1\src\main.c
> gcc -o output-bl1\mylib\src\AppName.obj -c -DAPP_NAME="Bootloader1"
> -Ioutput-bl1\mylib\include -Ioutput-bl1\mylib\src\include
> output-bl1\mylib\src\AppName.c
> ar rc output-bl1\mylib\libmylib.a output-bl1\mylib\src\AppName.obj
> gcc -o output-bl1\bootloader.exe output-bl1\src\main.obj
> output-bl1\mylib\libmylib.a
> gcc -o output-fw\src\main.obj -c -DAPP_NAME=Firmware
> -Ilibrary\mylib\include output-fw\src\main.c
> gcc -o output-fw\mylib\src\AppName.obj -c -DAPP_NAME=Firmware
> -Ioutput-fw\mylib\include -Ioutput-fw\mylib\src\include
> output-fw\mylib\src\AppName.c
> ar rc output-fw\mylib\libmylib.a output-fw\mylib\src\AppName.obj
> gcc -o output-fw\firmware.exe output-fw\src\main.obj
> output-fw\mylib\libmylib.a
> output-bl0\bootloader.exe
> Hello, this is Bootloader0!
> output-bl1\bootloader.exe
> Hello, this is Bootloader1!
> output-fw\firmware.exe
> Hello, this is Firmware!
> scons: done building targets.
>
> Now how to selectively produce the problems:
>
> *1) If you set duplicate=0 in /SConstruct in line 7 but remove srcnode()
> in line 6 of /library/mylib/SConscript the build breaks.*
>
> $ scons run-all
> scons: Reading SConscript files ...
> scons: done reading SConscript files.
> scons: Building targets ...
> gcc -o output-bl0\src\main.obj -c -Ioutput-bl0\mylib\include
> bootloader\src\main.c
> bootloader\src\main.c:1:10: fatal error: AppName.h: No such file or
> directory
> #include <AppName.h>
> ^~~~~~~~~~~
> compilation terminated.
> scons: *** [output-bl0\src\main.obj] Error 1
> scons: building terminated because of errors.
>
> *2) If you set duplicate=0 in /SConstruct in line 7 but keep srcnode() in
> line 6 of /library/mylib/SConscript the build succeeds, but the include
> paths are duplicated: *
>
> $ scons run-all
> scons: Reading SConscript files ...
> scons: done reading SConscript files.
> scons: Building targets ...
> gcc -o output-bl0\src\main.obj -c -Ilibrary\mylib\include
> bootloader\src\main.c
> gcc -o output-bl0\mylib\src\AppName.obj -c -DAPP_NAME="Bootloader0" -Ioutput-bl0\mylib\include
> -Ilibrary\mylib\include -Ibootloader\mylib\include
> -Ioutput-bl0\mylib\src\include -Ilibrary\mylib\src\include
> -Ibootloader\mylib\src\include library\mylib\src\AppName.c
> ar rc output-bl0\mylib\libmylib.a output-bl0\mylib\src\AppName.obj
> gcc -o output-bl0\bootloader.exe output-bl0\src\main.obj
> output-bl0\mylib\libmylib.a
> gcc -o output-bl1\src\main.obj -c -Ilibrary\mylib\include
> bootloader\src\main.c
> gcc -o output-bl1\mylib\src\AppName.obj -c -DAPP_NAME="Bootloader1" -Ioutput-bl1\mylib\include
> -Ilibrary\mylib\include -Ibootloader\mylib\include
> -Ioutput-bl1\mylib\src\include -Ilibrary\mylib\src\include
> -Ibootloader\mylib\src\include library\mylib\src\AppName.c
> ar rc output-bl1\mylib\libmylib.a output-bl1\mylib\src\AppName.obj
> gcc -o output-bl1\bootloader.exe output-bl1\src\main.obj
> output-bl1\mylib\libmylib.a
> gcc -o output-fw\src\main.obj -c -DAPP_NAME=Firmware
> -Ilibrary\mylib\include firmware\src\main.c
> gcc -o output-fw\mylib\src\AppName.obj -c -DAPP_NAME=Firmware -Ioutput-fw\mylib\include
> -Ilibrary\mylib\include -Ifirmware\mylib\include
> -Ioutput-fw\mylib\src\include -Ilibrary\mylib\src\include
> -Ifirmware\mylib\src\include library\mylib\src\AppName.c
> ar rc output-fw\mylib\libmylib.a output-fw\mylib\src\AppName.obj
> gcc -o output-fw\firmware.exe output-fw\src\main.obj
> output-fw\mylib\libmylib.a
> output-bl0\bootloader.exe
> Hello, this is Bootloader0!
> output-bl1\bootloader.exe
> Hello, this is Bootloader1!
> output-fw\firmware.exe
> Hello, this is Firmware!
> scons: done building targets
>
> *3) (This one is new) If you exchange all occurrences (6 total) of
> `output-` in /SConstruct with `output/`, example `output-bl0` becomes
> `output/bl0`, scons fails with a stacktrace:*
>
> /w/Projects/myproject2
> $ scons run-all
> scons: Reading SConscript files ...
>
> scons: warning: Calling missing SConscript without error is deprecated.
> Transition by adding must_exist=0 to SConscript calls.
> Missing SConscript 'output\fw\mylib\SConscript'
> File "W:\Projects\myproject2\output\fw\SConscript", line 5, in <module>
> TypeError: cannot unpack non-iterable NoneType object:
> File "W:\Projects\myproject2\SConstruct", line 10:
> firmware = SConscript('firmware/SConscript', exports=exports,
> variant_dir='output/fw/', duplicate=duplicate)
> File
> "w:\projects\myproject2\.venv\lib\site-packages\scons\SCons\Script\SConscript.py",
> line 660:
> return method(*args, **kw)
> File
> "w:\projects\myproject2\.venv\lib\site-packages\scons\SCons\Script\SConscript.py",
> line 597:
> return _SConscript(self.fs, *files, **subst_kw)
> File
> "w:\projects\myproject2\.venv\lib\site-packages\scons\SCons\Script\SConscript.py",
> line 286:
> exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
> File "W:\Projects\myproject2\output\fw\SConscript", line 5:
> mylib_a, mylib_h = env.SConscript('../library/mylib/SConscript',
> exports={'env': mylib_env}, variant_dir='mylib', duplicate=duplicate)
>
> Regards
>
> On Fri, May 22, 2020 at 6:27 PM Bill Deegan <bill at baddogconsulting.com>
> wrote:
>
>> If you have to go that route.. I can pretty much guarantee you've done
>> something wrong.
>> I've cloned your repo and I'll take a look at it this weekend.
>> Pretty sure it's the combo of SConscript variant dir and partial separate
>> VariantDir() for other dirs which is causing an issue.
>>
>> On Fri, May 22, 2020 at 4:20 AM Hans-Christian Wild <hchr.wild at gmail.com>
>> wrote:
>>
>>> OK, thank you for the clarification!
>>>
>>> On Fri, May 22, 2020 at 12:32 AM Daniel Moody <dmoody256 at gmail.com>
>>> wrote:
>>>
>>>> Ah my mistake, I ran scons first with duplication on, then modified the
>>>> SConstruct to disable it and ran scons -c, which scons then did not cleaned
>>>> the duplicated files because of my change, so they were still there when I
>>>> built again. So that is not working for me either.
>>>>
>>>> On Thu, May 21, 2020 at 4:51 PM Hans-Christian Wild <
>>>> hchr.wild at gmail.com> wrote:
>>>>
>>>>> Hi Daniel,
>>>>>
>>>>> thank you for taking the time to answer! Unfortunately, there are
>>>>> still problems.
>>>>>
>>>>> Firstly, your suggestion does not work for me?! I am puzzled,,
>>>>>
>>>>> When I just change the following line, the output still reads:
>>>>>
>>>>> mylib_a, mylib_h = SConscript('../../SConscript', exports='env',
>>>>> variant_dir='../../target/mylib', duplicate=False)
>>>>>
>>>>> /w/Projects/mylib/example/mingw
>>>>> $ scons
>>>>> scons: Reading SConscript files ...
>>>>> scons: done reading SConscript files.
>>>>> scons: Building targets ...
>>>>> scons: building associated VariantDir targets: .
>>>>> gcc -o target\src\main.obj -c -IW:\Projects\mylib\target\mylib\include
>>>>> target\src\main.c
>>>>> target\src\main.c:4:10: fatal error: Types.h: No such file or directory
>>>>> #include <Types.h>
>>>>> ^~~~~~~~~
>>>>> compilation terminated.
>>>>> scons: *** [target\src\main.obj] Error 1
>>>>> scons: building terminated because of errors.
>>>>>
>>>>> So only the effect was that SCons has decided to put a absolute path
>>>>> to the wrong location, instead of the relative one.
>>>>>
>>>>> Secondly, I am aware of that sentence from the docs and this is what I
>>>>> would like! I need X examples building X different version of mylib.
>>>>>
>>>>> Also, I assumed that when I do a (in /SConscript)
>>>>>
>>>>> headers = env.Dir('include/')
>>>>>
>>>>> by the docs:
>>>>>
>>>>> Dir(name, [directory]) , env.Dir(name, [directory])
>>>>> [...] If no directory is specified, the current script's directory is
>>>>> used as the parent.
>>>>>
>>>>> .. that this would mean that scons remembers a relative path and the
>>>>> current SConscript file as parent and is able to compute and abs or
>>>>> relative path to if from where ever scons is called our what variant dirs
>>>>> you specify.
>>>>>
>>>>> SCons does so correctly when you have duplication on, if not, it
>>>>> doesn't.
>>>>>
>>>>> -hans
>>>>>
>>>>> On Thu, May 21, 2020 at 10:22 PM Daniel Moody <dmoody256 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Also should note, I am not saying its "not a bug", just what worked
>>>>>> when I was messing with your example. It seems like its "not a bug" and if
>>>>>> you really want your variant dir in the location of the SConstruct you
>>>>>> should use duplicated tree. So probably more of a feature request if you
>>>>>> were to submit an issue.
>>>>>>
>>>>>> On Thu, May 21, 2020 at 2:12 PM Daniel Moody <dmoody256 at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> the docs say:
>>>>>>> https://scons.org/doc/3.1.2/HTML/scons-man.html#f-SConscript
>>>>>>> "The variant_dir argument is interpreted relative to the directory
>>>>>>> of the calling SConscript file."
>>>>>>>
>>>>>>> So in the case the SConstruct is the "calling SConscript" and you
>>>>>>> are saying the variant dir for the SConscript should be target/mylib, but
>>>>>>> its not, its ../../target/mylib relative to the calling SConstruct.
>>>>>>>
>>>>>>> So you need to call it like this:
>>>>>>> mylib_a, mylib_h = SConscript('../../SConscript', exports='env',
>>>>>>> variant_dir='../../target/mylib', duplicate=False)
>>>>>>>
>>>>>>> For a non duplicated variant dir, the variant dir will be
>>>>>>> substituted with the source dir when resolving paths for the build, which
>>>>>>> means substituting "target/mylib/include" to "./include" where
>>>>>>> 'target/mylib' is the variant dir and '.' is the source dir, will not work
>>>>>>> from the calling SConstructs location. It should be substituted with
>>>>>>> "../../target/mylib" so the variant dir resolved to the correct location.
>>>>>>>
>>>>>>> On Thu, May 21, 2020 at 12:54 PM Hans-Christian Wild <
>>>>>>> hchr.wild at gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I have two questions about SCons, I have uploaded a *working*
>>>>>>>> example as base where SCons does everything nearly as I want it. The
>>>>>>>> problems start to occur only when we do slight modifications to this
>>>>>>>> example.
>>>>>>>>
>>>>>>>> https://github.com/hchrwild/scons01/tree/master
>>>>>>>>
>>>>>>>> In the project we have a main SConstruct file in `/example/mingw/`
>>>>>>>> which imports a library SConscript file from `/`. The SConscript file
>>>>>>>> creates its little library for the given environment and returns it
>>>>>>>> together with its public header include path. The SConstruct file in turn
>>>>>>>> builds an executable linking the static library and using the public header
>>>>>>>> include path.
>>>>>>>>
>>>>>>>> Output:
>>>>>>>>
>>>>>>>> scons: Reading SConscript files ...
>>>>>>>> scons: done reading SConscript files.
>>>>>>>> scons: Building targets ...
>>>>>>>> scons: building associated VariantDir targets:
>>>>>>>> target\mylib\example\mingw
>>>>>>>> gcc -o target\src\main.obj -c -Itarget\mylib\include
>>>>>>>> target\src\main.c
>>>>>>>> gcc -o target\mylib\src\Assert.obj -c -Itarget\mylib\include
>>>>>>>> -Itarget\mylib\src\include target\mylib\src\Assert.c
>>>>>>>> ar rc target\mylib\libmylib.a target\mylib\src\Assert.obj
>>>>>>>> gcc -o target\mingw-example.exe target\src\main.obj
>>>>>>>> target\mylib\libmylib.a
>>>>>>>> scons: done building targets.
>>>>>>>>
>>>>>>>> *Perfect!*
>>>>>>>> (Although I don't understand why it prints `building associated
>>>>>>>> VariantDir targets: *target\mylib\example\mingw*` which to me does
>>>>>>>> not make sense, IMO it should be *`example/mingw/target/mylib`* if
>>>>>>>> anything, but let's put that aside..)
>>>>>>>>
>>>>>>>> *Now the actual problems:*
>>>>>>>>
>>>>>>>> 1) I don't like that SCons duplicates everything. What I really
>>>>>>>> want is an out-of-source variant build. When I set `duplicate=False`, the
>>>>>>>> build fails, however:
>>>>>>>>
>>>>>>>> mylib_a, mylib_h = SConscript('../../SConscript', exports='env',
>>>>>>>> variant_dir='target/mylib', duplicate=False)
>>>>>>>>
>>>>>>>> scons: Reading SConscript files ...
>>>>>>>> scons: done reading SConscript files.
>>>>>>>> scons: Building targets ...
>>>>>>>> scons: building associated VariantDir targets: .
>>>>>>>> gcc -o target\src\main.obj -c *-Itarget\mylib\include*
>>>>>>>> target\src\main.c
>>>>>>>> target\src\main.c:4:10: fatal error: Types.h: No such file or
>>>>>>>> directory
>>>>>>>> #include <Types.h>
>>>>>>>> ^~~~~~~~~
>>>>>>>> compilation terminated.
>>>>>>>> scons: *** [target\src\main.obj] Error 1
>>>>>>>> scons: building terminated because of errors.
>>>>>>>>
>>>>>>>> As marked in above output, the include path is not updated. The
>>>>>>>> SConscript file is at `../../` relative to SConstruct, and the include
>>>>>>>> folder is ./include relative to SConscript. Therefore, without duplication,
>>>>>>>> the correct include path relative to SConstruct would be `../../include` ,
>>>>>>>> which SCons does not seem to compute however. Is this expected or a missing
>>>>>>>> feature/bug? I really like the idea, that the SConstruct file really only
>>>>>>>> needs to know where the SConscript file is located and having everything
>>>>>>>> else being figured out by the build system...
>>>>>>>>
>>>>>>>> 2) Let's put that aside and keep above change, but hardcode the
>>>>>>>> expected include path in SConstruct file:
>>>>>>>>
>>>>>>>> program = env.Program('target/mingw-example', sources, CPPPATH=[
>>>>>>>> '../../include'], LIBS=[mylib_a])
>>>>>>>>
>>>>>>>> scons: Reading SConscript files ...
>>>>>>>> scons: done reading SConscript files.
>>>>>>>> scons: Building targets ...
>>>>>>>> scons: building associated VariantDir targets: .
>>>>>>>> gcc -o target\src\main.obj -c -IW:\Projects\mylib\include
>>>>>>>> target\src\main.c
>>>>>>>> gcc -o target\mylib\src\Assert.obj -c *-Itarget\mylib\include
>>>>>>>> -IW:\Projects\mylib\include -Itarget\mylib\src\include
>>>>>>>> -IW:\Projects\mylib\src\include* W:\Projects\mylib\src\Assert.c
>>>>>>>> ar rc target\mylib\libmylib.a target\mylib\src\Assert.obj
>>>>>>>> gcc -o target\mingw-example.exe target\src\main.obj
>>>>>>>> target\mylib\libmylib.a
>>>>>>>> scons: done building targets.
>>>>>>>>
>>>>>>>> Now the compilation succeeds. However, scons has now duplicated the
>>>>>>>> include paths for the static library! One time with the invalid
>>>>>>>> variant-with-duplication path, and one time with the correct (absolute
>>>>>>>> path). Why does it do that? Is this a bug?
>>>>>>>>
>>>>>>>> Sorry for the long post. Any feedback appreciated!
>>>>>>>>
>>>>>>>> -hans
>>>>>>>> _______________________________________________
>>>>>>>> Scons-users mailing list
>>>>>>>> Scons-users at scons.org
>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>> Scons-users mailing list
>>>>>> Scons-users at scons.org
>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>
>>>>> _______________________________________________
>>>>> Scons-users mailing list
>>>>> Scons-users at scons.org
>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>
>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>> _______________________________________________
>>> Scons-users mailing list
>>> Scons-users at scons.org
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20200522/f63ee4af/attachment-0001.html>
More information about the Scons-users
mailing list