[Scons-users] Wacky crash in Clone()

[Mats Wichmann]

On 1/21/22 14:39, Marc Branchaud via Scons-users wrote:
> (Possibly should send this to scons-dev@?)

here is fine.

> 
> Hi all,
> 
> I'm updating a complex build from 2.2.0 (Python 2.7.18) to 4.2.0 (Python
> 3.10.1), and I've run into this crash.  Unfortunately I have been unable
> to craft a simple reproduction example, and the code cannot be
> published.  The crash happens inside an env.Clone() call in a function
> being used as an Action's output, where the Action is a custom Builder.
> 
> I'm hoping someone can point out something I'm doing that became "wrong"
> since SCons 2.2.0.  Has there been a change in how custom Builders work?
> 
> Or, maybe someone can point me at someplace that might help me figure
> out the bug.
> 
> The code is basically structured as follows.  The custom builder uses
> functions from a "BaseBuilder" module defined in site_scons.  (I've
> elided a lot of setup work in the real code, which is sprinkled among
> several different files, many of which are exec()'d by the SConstruct.
> I'm trying not to suspect the reliance on exec() to split up what would
> otherwise be a giant SConstruct file.)

that has a bit of a smell to it... why would you need to exec things
manually? things that need to do setup can go in site_scons, and SCons
will arrange to pull them in.

> 
> SConstruct
>     e1 = Environment(
>         tools = [ 'MyBuilder' ],
>     )
>     e2 = e1.Clone(
>         CCFLAGS = ['-g']
>     )
>     e2.MyBuilder(
>         target="out",
>         source="in",
>     )
> 
> site_scons/BaseBuilder.py
>     def message(target, source, env):
>         myenv = env.Clone()           # <-- This crashes
>         print("MESSAGE")
> 
>     def builder(target, source, env):
>         print("BUILDER")
> 
> site_scons/site_tools/MyBuilder.py
>     import BaseBuilder
> 
>     def exists(env):
>         return True
> 
>     def generate(env, **kwargs):
>         env['BUILDERS']['MyBuilder'] = env.Builder(
>             action = env.Action(
>                 BaseBuilder.builder,
>                 BaseBuilder.message,
>             )
>         )
> 
> Sadly the above code works just fine.  But here's the crash from the
> real code:
> 
> Traceback (most recent call last):
>   File
> "/usr/local/lib/python3.10/site-packages/SCons-4.2.0-py3.10.egg/SCons/Taskmaster.py",
> line 235, in execute
>     self.targets[0].build()
>   File
> "/usr/local/lib/python3.10/site-packages/SCons-4.2.0-py3.10.egg/SCons/Node/__init__.py",
> line 755, in build
>     self.get_executor()(self, **kw)
>   File
> "/usr/local/lib/python3.10/site-packages/SCons-4.2.0-py3.10.egg/SCons/Executor.py",
> line 384, in __call__
>     return _do_execute_map[self._do_execute](self, target, kw)
>   File
> "/usr/local/lib/python3.10/site-packages/SCons-4.2.0-py3.10.egg/SCons/Executor.py",
> line 117, in execute_action_list
>     status = act(*args, **kw)
>   File
> "/usr/local/lib/python3.10/site-packages/SCons-4.2.0-py3.10.egg/SCons/Action.py",
> line 686, in __call__
>     cmd = self.strfunction(target, source, env)
>   File
> "/usr/home/marcnarc/Code/Worktrees/python3-build/site_scons/BaseBuilder.py",
> line 77, in message
>     myenv = env.Clone()
>   File
> "/usr/local/lib/python3.10/site-packages/SCons-4.2.0-py3.10.egg/SCons/Environment.py",
> line 1450, in Clone
>     clone = copy.copy(self)
>   File "/usr/local/lib/python3.10/copy.py", line 102, in copy
>     return _reconstruct(x, None, *rv)
>   File "/usr/local/lib/python3.10/copy.py", line 272, in _reconstruct
>     if hasattr(y, '__setstate__'):
>   File
> "/usr/local/lib/python3.10/site-packages/SCons-4.2.0-py3.10.egg/SCons/Environment.py",
> line 2384, in __getattr__
>     attr = getattr(self.__dict__['__subject'], name)
> KeyError: '__subject'
> 
> At the offending line, `self` is an OverrideEnvironment that somehow has
> a completely empty __dict__.
> 
> One frame up, the `y` that's being checked with hasattr() was created
> inside copy.copy()'s _reconstruct() via a __newobj__() call.  This `y`
> is an OverrideEnvironment but it's __dict__ is empty.
> 
> If I set a breakpoint at the dying getattr() line, it gets called twice
> before the crash, both with proper OverrideEnvironment objects.
> 
> Any ideas?  Could something in the build's code have possibly mucked
> with the OverrideEnvironment created by __newobj__()?  (What would such
> a thing even look like?)

My faulty memory is telling me we heard about something like this fairly
recently, where something doesn't get set up right in an
OverrideEnvironment. Might take a little bit of prospecting to find
that, if indeed it was related (which is hard to tell from your example,
since the example works).