[Scons-users] Possible threading bug in scons

[Bill Deegan]

Tom,

I'd say it's not advised.
What are they trying to do in the Command in a scanner?

-Bill

On Tue, Jan 19, 2016 at 5:34 AM, Tom Tanner (BLOOMBERG/ LONDON) <
ttanner2 at bloomberg.net> wrote:

> We've narrowed it down to some code he changed in a scanner which appears
> to be now adding an env.Command()
>
> Is that supported?
>
> Traceback (most recent call last):
> File "scons/lib/scons-2.2.0/SCons/Taskmaster.py", line 233, in execute
> if not t.retrieve_from_cache():
> File "scons/lib/scons-2.2.0/SCons/Node/FS.py", line 2744, in
> retrieve_from_cache
> return self.get_build_env().get_CacheDir().retrieve(self)
> File "scons/lib/scons-2.2.0/SCons/CacheDir.py", line 208, in retrieve
> if CacheRetrieve(node, [], env, execute=1) == 0:
> File "scons/lib/scons-2.2.0/SCons/Action.py", line 566, in __call__
> cmd = self.strfunction(target, source, env)
> File "scons/lib/scons-2.2.0/SCons/CacheDir.py", line 65, in
> CacheRetrieveString
> cachedir, cachefile = cd.cachepath(t)
> File "scons/lib/scons-2.2.0/SCons/CacheDir.py", line 169, in cachepath
> sig = node.get_cachedir_bsig()
> File "scons/lib/scons-2.2.0/SCons/Node/FS.py", line 3127, in
> get_cachedir_bsig
> sigs = [n.get_cachedir_csig() for n in children]
> File "scons/lib/scons-2.2.0/SCons/Node/FS.py", line 3107, in
> get_cachedir_csig
> cachedir, cachefile = self.get_build_env().get_CacheDir().cachepath(self)
> File "scons/lib/scons-2.2.0/SCons/CacheDir.py", line 169, in cachepath
> sig = node.get_cachedir_bsig()
> File "scons/lib/scons-2.2.0/SCons/Node/FS.py", line 3127, in
> get_cachedir_bsig
> sigs = [n.get_cachedir_csig() for n in children]
> File "scons/lib/scons-2.2.0/SCons/Node/__init__.py", line 787, in
> get_cachedir_csig
> return self.get_csig()
> File "scons/lib/scons-2.2.0/SCons/Node/FS.py", line 1778, in get_csig
> contents = self.get_contents()
> File "scons/lib/scons-2.2.0/SCons/Node/FS.py", line 1769, in get_contents
> contents.append('%s %s\n' % (node.get_csig(), node.name))
> File "scons/lib/scons-2.2.0/SCons/Node/__init__.py", line 784, in get_csig
> return self.ninfo.csig
> AttributeError: 'FileNodeInfo' object has no attribute 'csig'
>
> From: wblevins001 at gmail.com At: Jan 18 2016 22:20:44
> To: Tom Tanner (BLOOMBERG/ LONDON) <ttanner2 at bloomberg.net>,
> scons-users at scons.org
> Subject: Re: [Scons-users] Possible threading bug in scons
>
> I looked through TaskMaster::execute in the latest and didn't see anything
> that jumped out at me. Do you have a full stack trace or just that snippet?
>
> On Mon, Jan 18, 2016 at 9:51 PM, William Blevins <wblevins001 at gmail.com>
> wrote:
>
>> I'll take a brief look, but I am pretty busy during exam season. I have
>> ran heterogeneous SCons builds on 40-core servers without every running
>> into this, but it is still possible...
>>
>> On Mon, Jan 18, 2016 at 9:46 PM, William Blevins <wblevins001 at gmail.com>
>> wrote:
>>
>>>
>>>
>>> On Mon, Jan 18, 2016 at 2:48 PM, Tom Tanner (BLOOMBERG/ LONDON) <
>>> ttanner2 at bloomberg.net> wrote:
>>>
>>>> Someone here has been doing a lot of parallel builds and come across
>>>> what looks like a bug though I'm not sure why:
>>>>
>>>> The problem is with this bit in Taskmaster:
>>>>
>>>> def execute(self):
>>>> """
>>>> Called to execute the task.
>>>>
>>>> This method is called from multiple threads in a parallel build,
>>>> so only do thread safe stuff here. Do thread unsafe stuff in
>>>> prepare(), executed() or failed().
>>>> """
>>>> T = self.tm.trace
>>>> if T: T.write(self.trace_message(u'Task.execute()', self.node))
>>>>
>>>> try:
>>>> cached_targets = []
>>>> for t in self.targets:
>>>> if not t.retrieve_from_cache():
>>>> break
>>>>
>>>> The t.retrieve_from_cache ends up doing this:
>>>>
>>>> File ".../SCons/Node/FS.py", line 1778, in get_csig
>>>> contents = self.get_contents()
>>>> File ".../SCons/Node/FS.py", line 1769, in get_contents
>>>> contents.append('%s %s\n' % (node.get_csig(), node.name))
>>>> File ".../SCons/Node/__init__.py", line 784, in get_csig
>>>> return self.ninfo.csig
>>>> AttributeError: 'FileNodeInfo' object has no attribute 'csig'
>>>>
>>>> which goes away if he puts a lock in Node.get_ninfo.
>>>>
>>>> Now, the only way I can think of this triggering is if he's managed to
>>>> get two rules with the same target. Is that possible? and if so how can you
>>>> detect it? I don't like the idea of having a lock in Node.get_ninfo.
>>>>
>>>
>>> Two rules with the same target generate a SCons Error at SConscript
>>> parser time.
>>>
>>>
>>>>
>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>>>
>>>
>>
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20160119/fe05254e/attachment-0001.html>