[Scons-users] Intermittent Install() failure

Bill Deegan bill at baddogconsulting.com
Tue Jul 19 17:04:55 EDT 2016 

William,

Indeed..
-Bill

On Tue, Jul 19, 2016 at 1:48 PM, William Blevins <wblevins001 at gmail.com>
wrote:

> Bill,
>
> I assume you think this may be a problem with not explicitly closing the
> file? Relying on python garbage collection in this case would certainly be
> dangerous.
>
> V/R,
> William
>
> On Tue, Jul 19, 2016 at 6:22 PM, Bill Deegan <bill at baddogconsulting.com>
> wrote:
>
>> Steve,
>>
>> Do you have any python logic creating the files in the SCons process? (a
>> builder where the builder actuall does open('file','w').. to create the
>> file?
>>
>> -Bill
>>
>> On Tue, Jul 19, 2016 at 7:55 AM, William Blevins <wblevins001 at gmail.com>
>> wrote:
>>
>>> Steve,
>>>
>>> As a final thought, I have used SCons to build HT 24-core server blades
>>> (so 48 w/ HT) that RAM boot (no harddrive at RT). If it was specific to the
>>> core package, I imagine I would have seen this issue tons.
>>>
>>> V/R,
>>> William
>>>
>>> On Tue, Jul 19, 2016 at 12:52 PM, William Blevins <wblevins001 at gmail.com
>>> > wrote:
>>>
>>>> Vasily,
>>>>
>>>> I think you are referring to Precious.
>>>>
>>>> V/R,
>>>> William
>>>>
>>>> On Tue, Jul 19, 2016 at 9:51 AM, Vasily <just.one.man at yandex.ru> wrote:
>>>>
>>>>> Hi Steve,
>>>>>
>>>>> What does your build do with this file except installing it? Is it
>>>>> used by a compiler or some other tool?
>>>>>
>>>>> Also, if I'm not mistaken, default SCons behavior is to remove the
>>>>> target before performing any action to regenerate it, so this might be the
>>>>> source of the exception you're seeing. There seems to be a way to turn off
>>>>> the removal part of the action, so you may want to check the manual and see
>>>>> if it helps you.
>>>>>
>>>>> P.S. Based on file path I assume you're working on Windows, is this
>>>>> path a network one?
>>>>>
>>>>> Thanks,
>>>>> Vasily
>>>>> 19 июля 2016 г. 11:02 пользователь "Hill, Steve (FP COM)" <
>>>>> Steve.Hill at cobham.com> написал:
>>>>>
>>>>> Thanks William.
>>>>>>
>>>>>>
>>>>>>
>>>>>> I’ve checked the dependency tree and, as far as I can tell, it looks
>>>>>> OK.
>>>>>>
>>>>>> We do have custom scanners – using env.Scanner(_scan_domain_header) –
>>>>>> but not for the files that are affected.
>>>>>>
>>>>>> As far as I can see, the fix for that Python bug never made it into
>>>>>> 2.6.x but it appears always to result in a “No child processes” OSError so
>>>>>> I don’t believe that is what I am seeing.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Note that I have (temporarily) changed the decider below to simply
>>>>>> return True (without doing anything with the file) and I still see the
>>>>>> issue so the decider doesn’t seem to be relevant to the issue.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Does anyone have any more ideas? This is becoming a major issue for
>>>>>> our automated builds…
>>>>>>
>>>>>>
>>>>>>
>>>>>> S.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Steve,
>>>>>>
>>>>>> I of course should have asked the obvious question, do you know if
>>>>>> you dependency tree has missing dependencies? This tends to be a common
>>>>>> issue.
>>>>>>
>>>>>> V/R,
>>>>>>
>>>>>> William
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Jul 15, 2016 at 5:14 PM, William Blevins <
>>>>>> wblevins001 at gmail.com> wrote:
>>>>>>
>>>>>> Steve,
>>>>>>
>>>>>> I'm not aware of any specific issue with install, but there are some
>>>>>> possible issues that I am aware:
>>>>>>
>>>>>>    1. If you have custom scanners, make sure they implement from
>>>>>>    SCons.Scanner.Current and not SCons.Scanner.Base; otherwise, you might have
>>>>>>    concurrent file access between implicit scanning operations and other
>>>>>>    processes.
>>>>>>    2. There was a big subprocess bug in python 2.6 that carried
>>>>>>    through several other major versions:
>>>>>>    https://bugs.python.org/issue1731717. I would check to see that
>>>>>>    your version of 2.6 contains the patch for this issue.
>>>>>>
>>>>>> V/R,
>>>>>>
>>>>>> William
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Jul 15, 2016 at 9:30 AM, Hill, Steve (FP COM) <
>>>>>> Steve.Hill at cobham.com> wrote:
>>>>>>
>>>>>> I’m having a problem where, with parallel builds (most people use 8,
>>>>>> 12 or 16 threads), we occasionally get failures like the following:
>>>>>>
>>>>>>
>>>>>>
>>>>>> F:\<directory path>\hw_cfgs\1Server_1TM_6C66_2U.cfg: The process cannot access the file because it is being used by another process
>>>>>> scons: building terminated because of errors.
>>>>>>
>>>>>>
>>>>>>
>>>>>> We are running Python 2.6.5 (with pywin32) and SCons 2.3.6. This file
>>>>>> is being copied due to a Install() call. Note that, for various historical
>>>>>> reasons, we have the following decider for these Installs:
>>>>>>
>>>>>>
>>>>>>
>>>>>> def _copy_decider(dependency, target, prev_ni):
>>>>>>
>>>>>>     target = str(target.abspath)
>>>>>>
>>>>>>     dependency = str(dependency.abspath)
>>>>>>
>>>>>>     if os.path.isfile(target) and os.path.isfile(dependency):
>>>>>>
>>>>>> *        # By default, filecmp.cmp assumes that files with identical
>>>>>> os.stat signatures*
>>>>>>
>>>>>> *        # (which includes the inode) are the same file and, hence,
>>>>>> must be the same.*
>>>>>>
>>>>>> *        # However, on Windows, there is no inode - it appears to be
>>>>>> set to zero - so*
>>>>>>
>>>>>> *        # any two files with the same size and
>>>>>> access/creation/modification times*
>>>>>>
>>>>>> *        # will have the same os.stat signature, leading to a false
>>>>>> positive. For this*
>>>>>>
>>>>>> *        # reason, we must force it to do an actual file comparison
>>>>>> by setting shallow*
>>>>>>
>>>>>> *        # to False*
>>>>>>
>>>>>>         return not filecmp.cmp(target, dependency, shallow = False)
>>>>>>
>>>>>>     else:
>>>>>>
>>>>>> *        # Either one of the dependency or target isn't a file or one
>>>>>> of the files*
>>>>>>
>>>>>> *        # (presumably the target) isn't there so do the copy*
>>>>>>
>>>>>>         return True
>>>>>>
>>>>>>
>>>>>>
>>>>>> Note also that our IT department claims that virus checkers are
>>>>>> disabled within the directory where the build is being performed (and we
>>>>>> certainly have not seen any indication in the virus checker console to
>>>>>> suggest that to be incorrect).
>>>>>>
>>>>>>
>>>>>>
>>>>>> Does anyone have any thoughts as to what the problem might be?
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks in advance,
>>>>>>
>>>>>>
>>>>>>
>>>>>> S.
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Scons-users mailing list
>>>>>> Scons-users at scons.org
>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Scons-users mailing list
>>>>>> Scons-users at scons.org
>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> Scons-users mailing list
>>>>> Scons-users at scons.org
>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>
>>>>>
>>>>
>>>
>>> _______________________________________________
>>> Scons-users mailing list
>>> Scons-users at scons.org
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20160719/03bf3b6e/attachment-0001.html>

More information about the Scons-users mailing list