[Scons-users] Determining source of scons crash
Bill Deegan
bill at baddogconsulting.com
Tue Apr 28 17:14:12 EDT 2020
Thanks!
On Tue, Apr 28, 2020 at 1:53 PM Luke Robison <lukerobison at gmail.com> wrote:
> Submitted #3630 https://github.com/SCons/scons/issues/3630
>
> Thanks Bill.
>
> On Sat, Apr 25, 2020 at 5:48 PM Bill Deegan <bill at baddogconsulting.com>
> wrote:
>
>> Shouldn't crash.
>> Normally you'd see a warning/error about file generated by two different
>> builders...
>>
>> Go ahead and file a github issue with a simple reproducer?
>>
>> On Fri, Apr 24, 2020 at 3:01 PM Luke Robison <lukerobison at gmail.com>
>> wrote:
>>
>>> I guess the question is: should this be a valid Sconstruct? If not,
>>> should Scons issue a error/warning to the user, or should it crash?
>>>
>>> env = Environment()
>>> src = ['test1.f90']
>>> env.SharedLibrary('testlib-shared',src)
>>> env.StaticLibrary('testlib-static',src)
>>>
>>> On Fri, Apr 24, 2020 at 4:04 PM Bill Deegan <bill at baddogconsulting.com>
>>> wrote:
>>>
>>>> Luke,
>>>>
>>>> Not all platforms differentiate the object suffixes between shared and
>>>> not shared so you could have similar issue with c code..
>>>> Specify a different target name for one or the other and I think your
>>>> issue should go away.
>>>> -Bill
>>>>
>>>> On Fri, Apr 24, 2020 at 1:54 PM Luke Robison <lukerobison at gmail.com>
>>>> wrote:
>>>>
>>>>> Correct. For example, I'm sure with C code you could make a
>>>>> StaticLibrary and a SharedLibrary, those would require .o and .os files
>>>>> respectively (compiled from the same source code). The problem is unique
>>>>> to the fact that gfortran (and ifort, and most other fortran compilers)
>>>>> emit a .mod file each time they compile a source file which includes a
>>>>> "module" statement, and while the .o/.os files deconflict, the .mod files
>>>>> would be named the same.
>>>>>
>>>>> To make matters a little worse: My experience is that ifort produces a
>>>>> unique .mod file each time you re-compile the same source (I suspect they
>>>>> have a timestamp in there for some reason -- this forces scons to rebuild
>>>>> more than necessary!). Luckily gfortran re-produces the exact same if you
>>>>> re-compile the same source.
>>>>>
>>>>>
>>>>> On Fri, Apr 24, 2020 at 3:22 PM Bill Deegan <bill at baddogconsulting.com>
>>>>> wrote:
>>>>>
>>>>>> Hmm..
>>>>>> so .o and.os but they're both generating the same .mod filename?
>>>>>>
>>>>>> On Fri, Apr 24, 2020 at 1:21 PM Bill Deegan <
>>>>>> bill at baddogconsulting.com> wrote:
>>>>>>
>>>>>>> So you're building both an object and a shared object from the same
>>>>>>> source?
>>>>>>> Without specifying a target to create a different named file?
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Apr 24, 2020 at 12:26 PM Luke Robison <lukerobison at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Minor Correction: In my case my library was making a StaticLibrary
>>>>>>>> and I was calling env.SharedObject manually.
>>>>>>>>
>>>>>>>> Additionally, on Linux you can remove the Object() builds and
>>>>>>>> replace them with both SharedLibrary and StaticLibrary, and get the same
>>>>>>>> error. For some reason windows/MSYS didn't seem to crash on that
>>>>>>>> configuration.
>>>>>>>>
>>>>>>>> On Fri, Apr 24, 2020 at 2:20 PM Luke Robison <lukerobison at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> It seems the problem was that I was making my Library as:
>>>>>>>>> env.SharedLibrary( src_glob ), but that I had also called env.SharedObject(
>>>>>>>>> src_glob ). it seems the SharedLibrary was implicitly building .o files
>>>>>>>>> and I had requested .os files as well, and probably generating those
>>>>>>>>> compiled headers (.mod files) twice. When I changed to calling env.Objects
>>>>>>>>> rather than env.SharedObjects (or, turns out I wasn't using them anyways,
>>>>>>>>> get rid of the call entirely) then the build is happy.
>>>>>>>>>
>>>>>>>>> Here is minimum to reproduce:
>>>>>>>>>
>>>>>>>>> test1.f90
>>>>>>>>> module mod1
>>>>>>>>> integer :: mod1_int
>>>>>>>>> end module
>>>>>>>>>
>>>>>>>>> Sconstruct
>>>>>>>>> env = Environment()
>>>>>>>>> src = ['test1.f90']
>>>>>>>>> env.Object(src)
>>>>>>>>> env.SharedObject(src)
>>>>>>>>>
>>>>>>>>> Execution:
>>>>>>>>> $ scons
>>>>>>>>> scons: Reading SConscript files ...
>>>>>>>>> scons: done reading SConscript files.
>>>>>>>>> scons: Building targets ...
>>>>>>>>> gfortran -o test1.os -c test1.f90
>>>>>>>>> gfortran -o test1.o -c test1.f90
>>>>>>>>> scons: done building targets.
>>>>>>>>> AttributeError: 'NoneType' object has no attribute 'get_build_env':
>>>>>>>>> File "/usr/lib/python3.7/site-packages/SCons/Script/Main.py",
>>>>>>>>> line 1381:
>>>>>>>>> _exec_main(parser, values)
>>>>>>>>> File "/usr/lib/python3.7/site-packages/SCons/Script/Main.py",
>>>>>>>>> line 1344:
>>>>>>>>> _main(parser)
>>>>>>>>> File "/usr/lib/python3.7/site-packages/SCons/Script/Main.py",
>>>>>>>>> line 1119:
>>>>>>>>> nodes = _build_targets(fs, options, targets, target_top)
>>>>>>>>> File "/usr/lib/python3.7/site-packages/SCons/Script/Main.py",
>>>>>>>>> line 1318:
>>>>>>>>> jobs.run(postfunc = jobs_postfunc)
>>>>>>>>> File "/usr/lib/python3.7/site-packages/SCons/Job.py", line 111:
>>>>>>>>> self.job.start()
>>>>>>>>> File "/usr/lib/python3.7/site-packages/SCons/Job.py", line 216:
>>>>>>>>> task.executed()
>>>>>>>>> File "/usr/lib/python3.7/site-packages/SCons/Script/Main.py",
>>>>>>>>> line 256:
>>>>>>>>> SCons.Taskmaster.OutOfDateTask.executed(self)
>>>>>>>>> File "/usr/lib/python3.7/site-packages/SCons/Taskmaster.py",
>>>>>>>>> line 312:
>>>>>>>>> t.push_to_cache()
>>>>>>>>> File "/usr/lib/python3.7/site-packages/SCons/Node/FS.py", line
>>>>>>>>> 2967:
>>>>>>>>> self.get_build_env().get_CacheDir().push(self)
>>>>>>>>> File "/usr/lib/python3.7/site-packages/SCons/Node/__init__.py",
>>>>>>>>> line 654:
>>>>>>>>> result = self.get_executor().get_build_env()
>>>>>>>>>
>>>>>>>>> On Fri, Apr 24, 2020 at 11:59 AM Bill Deegan <
>>>>>>>>> bill at baddogconsulting.com> wrote:
>>>>>>>>>
>>>>>>>>>> Can you repro with a small example?
>>>>>>>>>> --taskmastertrace=trace.log can provide some useful information.
>>>>>>>>>>
>>>>>>>>>> You could wrap the line 624 in a try except and have it pop into
>>>>>>>>>> the debugger in the except.
>>>>>>>>>>
>>>>>>>>>> On Fri, Apr 24, 2020 at 9:31 AM Luke Robison <
>>>>>>>>>> lukerobison at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> I'm trying to track down a crash I have triggered in scons (both
>>>>>>>>>>> 3.0.1 and 3.1.2). This crash only happens when I ask scons to build my
>>>>>>>>>>> entire output directory (it will happily compile if my command-line target
>>>>>>>>>>> is build-variant/my_prog.exe, but giving it the target build-variant causes
>>>>>>>>>>> the crash).
>>>>>>>>>>>
>>>>>>>>>>> AttributeError: 'NoneType' object has no attribute
>>>>>>>>>>> 'get_build_env':
>>>>>>>>>>> File "/usr/lib/scons/SCons/Script/Main.py", line 1376:
>>>>>>>>>>> _exec_main(parser, values)
>>>>>>>>>>> File "/usr/lib/scons/SCons/Script/Main.py", line 1339:
>>>>>>>>>>> _main(parser)
>>>>>>>>>>> File "/usr/lib/scons/SCons/Script/Main.py", line 1103:
>>>>>>>>>>> nodes = _build_targets(fs, options, targets, target_top)
>>>>>>>>>>> File "/usr/lib/scons/SCons/Script/Main.py", line 1313:
>>>>>>>>>>> jobs.run(postfunc = jobs_postfunc)
>>>>>>>>>>> File "/usr/lib/scons/SCons/Job.py", line 111:
>>>>>>>>>>> self.job.start()
>>>>>>>>>>> File "/usr/lib/scons/SCons/Job.py", line 229:
>>>>>>>>>>> task.executed()
>>>>>>>>>>> File "/usr/lib/scons/SCons/Script/Main.py", line 237:
>>>>>>>>>>> SCons.Taskmaster.OutOfDateTask.executed(self)
>>>>>>>>>>> File "/usr/lib/scons/SCons/Taskmaster.py", line 312:
>>>>>>>>>>> t.push_to_cache()
>>>>>>>>>>> File "/usr/lib/scons/SCons/Node/FS.py", line 2910:
>>>>>>>>>>> self.get_build_env().get_CacheDir().push(self)
>>>>>>>>>>> File "/usr/lib/scons/SCons/Node/__init__.py", line 624:
>>>>>>>>>>> result = self.get_executor().get_build_env()
>>>>>>>>>>>
>>>>>>>>>>> I have two questions:
>>>>>>>>>>> 1) How can I go about debugging this? I found --debug=pdb to be
>>>>>>>>>>> usless here, as I end up in Main.py calling _build_targets rather than
>>>>>>>>>>> where the error occurs. Similarly, I can't do --tree, since python crashes
>>>>>>>>>>> before it gets to that point. Modifying the scons source with print
>>>>>>>>>>> statements has been my way forward for now.
>>>>>>>>>>> 2) Is it considered a bug to have scons crash like this (even if
>>>>>>>>>>> I may have fed it something bad?)
>>>>>>>>>>>
>>>>>>>>>>> After adding some prints to the scons code, It seems one of my
>>>>>>>>>>> compiled header outputs (code is Fortran, so the compiled header is
>>>>>>>>>>> foo.mod) is the "self" here, and self.get_executor() returns "None", and
>>>>>>>>>>> then python fails on trying to call None.get_build_env(). This crash
>>>>>>>>>>> happens immediately after building the relevant SharedObject file (foo.os)
>>>>>>>>>>> from the source (foo.f90). It doesn't happen for all files, but it happens
>>>>>>>>>>> predictably on some, and those tend to have no dependencies themselves.
>>>>>>>>>>> Re-running the build enough times will eventually let it complete.
>>>>>>>>>>>
>>>>>>>>>>> Any suggestions greatly appreciated.
>>>>>>>>>>>
>>>>>>>>>>> Luke
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Scons-users mailing list
>>>>>>>>>>> Scons-users at scons.org
>>>>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Scons-users mailing list
>>>>>>>>>> Scons-users at scons.org
>>>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>> Scons-users mailing list
>>>>>>>> Scons-users at scons.org
>>>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>> Scons-users mailing list
>>>>>> Scons-users at scons.org
>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>
>>>>> _______________________________________________
>>>>> Scons-users mailing list
>>>>> Scons-users at scons.org
>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>
>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>> _______________________________________________
>>> Scons-users mailing list
>>> Scons-users at scons.org
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20200428/d15e8ab9/attachment-0001.html>
More information about the Scons-users
mailing list