[Scons-users] Determining source of scons crash
Luke Robison
lukerobison at gmail.com
Tue Apr 28 16:52:46 EDT 2020
Submitted #3630 https://github.com/SCons/scons/issues/3630
Thanks Bill.
On Sat, Apr 25, 2020 at 5:48 PM Bill Deegan <bill at baddogconsulting.com>
wrote:
> Shouldn't crash.
> Normally you'd see a warning/error about file generated by two different
> builders...
>
> Go ahead and file a github issue with a simple reproducer?
>
> On Fri, Apr 24, 2020 at 3:01 PM Luke Robison <lukerobison at gmail.com>
> wrote:
>
>> I guess the question is: should this be a valid Sconstruct? If not,
>> should Scons issue a error/warning to the user, or should it crash?
>>
>> env = Environment()
>> src = ['test1.f90']
>> env.SharedLibrary('testlib-shared',src)
>> env.StaticLibrary('testlib-static',src)
>>
>> On Fri, Apr 24, 2020 at 4:04 PM Bill Deegan <bill at baddogconsulting.com>
>> wrote:
>>
>>> Luke,
>>>
>>> Not all platforms differentiate the object suffixes between shared and
>>> not shared so you could have similar issue with c code..
>>> Specify a different target name for one or the other and I think your
>>> issue should go away.
>>> -Bill
>>>
>>> On Fri, Apr 24, 2020 at 1:54 PM Luke Robison <lukerobison at gmail.com>
>>> wrote:
>>>
>>>> Correct. For example, I'm sure with C code you could make a
>>>> StaticLibrary and a SharedLibrary, those would require .o and .os files
>>>> respectively (compiled from the same source code). The problem is unique
>>>> to the fact that gfortran (and ifort, and most other fortran compilers)
>>>> emit a .mod file each time they compile a source file which includes a
>>>> "module" statement, and while the .o/.os files deconflict, the .mod files
>>>> would be named the same.
>>>>
>>>> To make matters a little worse: My experience is that ifort produces a
>>>> unique .mod file each time you re-compile the same source (I suspect they
>>>> have a timestamp in there for some reason -- this forces scons to rebuild
>>>> more than necessary!). Luckily gfortran re-produces the exact same if you
>>>> re-compile the same source.
>>>>
>>>>
>>>> On Fri, Apr 24, 2020 at 3:22 PM Bill Deegan <bill at baddogconsulting.com>
>>>> wrote:
>>>>
>>>>> Hmm..
>>>>> so .o and.os but they're both generating the same .mod filename?
>>>>>
>>>>> On Fri, Apr 24, 2020 at 1:21 PM Bill Deegan <bill at baddogconsulting.com>
>>>>> wrote:
>>>>>
>>>>>> So you're building both an object and a shared object from the same
>>>>>> source?
>>>>>> Without specifying a target to create a different named file?
>>>>>>
>>>>>>
>>>>>> On Fri, Apr 24, 2020 at 12:26 PM Luke Robison <lukerobison at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Minor Correction: In my case my library was making a StaticLibrary
>>>>>>> and I was calling env.SharedObject manually.
>>>>>>>
>>>>>>> Additionally, on Linux you can remove the Object() builds and
>>>>>>> replace them with both SharedLibrary and StaticLibrary, and get the same
>>>>>>> error. For some reason windows/MSYS didn't seem to crash on that
>>>>>>> configuration.
>>>>>>>
>>>>>>> On Fri, Apr 24, 2020 at 2:20 PM Luke Robison <lukerobison at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> It seems the problem was that I was making my Library as:
>>>>>>>> env.SharedLibrary( src_glob ), but that I had also called env.SharedObject(
>>>>>>>> src_glob ). it seems the SharedLibrary was implicitly building .o files
>>>>>>>> and I had requested .os files as well, and probably generating those
>>>>>>>> compiled headers (.mod files) twice. When I changed to calling env.Objects
>>>>>>>> rather than env.SharedObjects (or, turns out I wasn't using them anyways,
>>>>>>>> get rid of the call entirely) then the build is happy.
>>>>>>>>
>>>>>>>> Here is minimum to reproduce:
>>>>>>>>
>>>>>>>> test1.f90
>>>>>>>> module mod1
>>>>>>>> integer :: mod1_int
>>>>>>>> end module
>>>>>>>>
>>>>>>>> Sconstruct
>>>>>>>> env = Environment()
>>>>>>>> src = ['test1.f90']
>>>>>>>> env.Object(src)
>>>>>>>> env.SharedObject(src)
>>>>>>>>
>>>>>>>> Execution:
>>>>>>>> $ scons
>>>>>>>> scons: Reading SConscript files ...
>>>>>>>> scons: done reading SConscript files.
>>>>>>>> scons: Building targets ...
>>>>>>>> gfortran -o test1.os -c test1.f90
>>>>>>>> gfortran -o test1.o -c test1.f90
>>>>>>>> scons: done building targets.
>>>>>>>> AttributeError: 'NoneType' object has no attribute 'get_build_env':
>>>>>>>> File "/usr/lib/python3.7/site-packages/SCons/Script/Main.py",
>>>>>>>> line 1381:
>>>>>>>> _exec_main(parser, values)
>>>>>>>> File "/usr/lib/python3.7/site-packages/SCons/Script/Main.py",
>>>>>>>> line 1344:
>>>>>>>> _main(parser)
>>>>>>>> File "/usr/lib/python3.7/site-packages/SCons/Script/Main.py",
>>>>>>>> line 1119:
>>>>>>>> nodes = _build_targets(fs, options, targets, target_top)
>>>>>>>> File "/usr/lib/python3.7/site-packages/SCons/Script/Main.py",
>>>>>>>> line 1318:
>>>>>>>> jobs.run(postfunc = jobs_postfunc)
>>>>>>>> File "/usr/lib/python3.7/site-packages/SCons/Job.py", line 111:
>>>>>>>> self.job.start()
>>>>>>>> File "/usr/lib/python3.7/site-packages/SCons/Job.py", line 216:
>>>>>>>> task.executed()
>>>>>>>> File "/usr/lib/python3.7/site-packages/SCons/Script/Main.py",
>>>>>>>> line 256:
>>>>>>>> SCons.Taskmaster.OutOfDateTask.executed(self)
>>>>>>>> File "/usr/lib/python3.7/site-packages/SCons/Taskmaster.py", line
>>>>>>>> 312:
>>>>>>>> t.push_to_cache()
>>>>>>>> File "/usr/lib/python3.7/site-packages/SCons/Node/FS.py", line
>>>>>>>> 2967:
>>>>>>>> self.get_build_env().get_CacheDir().push(self)
>>>>>>>> File "/usr/lib/python3.7/site-packages/SCons/Node/__init__.py",
>>>>>>>> line 654:
>>>>>>>> result = self.get_executor().get_build_env()
>>>>>>>>
>>>>>>>> On Fri, Apr 24, 2020 at 11:59 AM Bill Deegan <
>>>>>>>> bill at baddogconsulting.com> wrote:
>>>>>>>>
>>>>>>>>> Can you repro with a small example?
>>>>>>>>> --taskmastertrace=trace.log can provide some useful information.
>>>>>>>>>
>>>>>>>>> You could wrap the line 624 in a try except and have it pop into
>>>>>>>>> the debugger in the except.
>>>>>>>>>
>>>>>>>>> On Fri, Apr 24, 2020 at 9:31 AM Luke Robison <
>>>>>>>>> lukerobison at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> I'm trying to track down a crash I have triggered in scons (both
>>>>>>>>>> 3.0.1 and 3.1.2). This crash only happens when I ask scons to build my
>>>>>>>>>> entire output directory (it will happily compile if my command-line target
>>>>>>>>>> is build-variant/my_prog.exe, but giving it the target build-variant causes
>>>>>>>>>> the crash).
>>>>>>>>>>
>>>>>>>>>> AttributeError: 'NoneType' object has no attribute
>>>>>>>>>> 'get_build_env':
>>>>>>>>>> File "/usr/lib/scons/SCons/Script/Main.py", line 1376:
>>>>>>>>>> _exec_main(parser, values)
>>>>>>>>>> File "/usr/lib/scons/SCons/Script/Main.py", line 1339:
>>>>>>>>>> _main(parser)
>>>>>>>>>> File "/usr/lib/scons/SCons/Script/Main.py", line 1103:
>>>>>>>>>> nodes = _build_targets(fs, options, targets, target_top)
>>>>>>>>>> File "/usr/lib/scons/SCons/Script/Main.py", line 1313:
>>>>>>>>>> jobs.run(postfunc = jobs_postfunc)
>>>>>>>>>> File "/usr/lib/scons/SCons/Job.py", line 111:
>>>>>>>>>> self.job.start()
>>>>>>>>>> File "/usr/lib/scons/SCons/Job.py", line 229:
>>>>>>>>>> task.executed()
>>>>>>>>>> File "/usr/lib/scons/SCons/Script/Main.py", line 237:
>>>>>>>>>> SCons.Taskmaster.OutOfDateTask.executed(self)
>>>>>>>>>> File "/usr/lib/scons/SCons/Taskmaster.py", line 312:
>>>>>>>>>> t.push_to_cache()
>>>>>>>>>> File "/usr/lib/scons/SCons/Node/FS.py", line 2910:
>>>>>>>>>> self.get_build_env().get_CacheDir().push(self)
>>>>>>>>>> File "/usr/lib/scons/SCons/Node/__init__.py", line 624:
>>>>>>>>>> result = self.get_executor().get_build_env()
>>>>>>>>>>
>>>>>>>>>> I have two questions:
>>>>>>>>>> 1) How can I go about debugging this? I found --debug=pdb to be
>>>>>>>>>> usless here, as I end up in Main.py calling _build_targets rather than
>>>>>>>>>> where the error occurs. Similarly, I can't do --tree, since python crashes
>>>>>>>>>> before it gets to that point. Modifying the scons source with print
>>>>>>>>>> statements has been my way forward for now.
>>>>>>>>>> 2) Is it considered a bug to have scons crash like this (even if
>>>>>>>>>> I may have fed it something bad?)
>>>>>>>>>>
>>>>>>>>>> After adding some prints to the scons code, It seems one of my
>>>>>>>>>> compiled header outputs (code is Fortran, so the compiled header is
>>>>>>>>>> foo.mod) is the "self" here, and self.get_executor() returns "None", and
>>>>>>>>>> then python fails on trying to call None.get_build_env(). This crash
>>>>>>>>>> happens immediately after building the relevant SharedObject file (foo.os)
>>>>>>>>>> from the source (foo.f90). It doesn't happen for all files, but it happens
>>>>>>>>>> predictably on some, and those tend to have no dependencies themselves.
>>>>>>>>>> Re-running the build enough times will eventually let it complete.
>>>>>>>>>>
>>>>>>>>>> Any suggestions greatly appreciated.
>>>>>>>>>>
>>>>>>>>>> Luke
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Scons-users mailing list
>>>>>>>>>> Scons-users at scons.org
>>>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Scons-users mailing list
>>>>>>>>> Scons-users at scons.org
>>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>> Scons-users mailing list
>>>>>>> Scons-users at scons.org
>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>
>>>>>> _______________________________________________
>>>>> Scons-users mailing list
>>>>> Scons-users at scons.org
>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>
>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>> _______________________________________________
>>> Scons-users mailing list
>>> Scons-users at scons.org
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20200428/34fa6745/attachment-0001.html>
More information about the Scons-users
mailing list