[Scons-users] Intermittent Install() failure

Tue Jul 19 13:48:11 EDT 2016

Bill,

I assume you think this may be a problem with not explicitly closing the
file? Relying on python garbage collection in this case would certainly be
dangerous.

V/R,
William

On Tue, Jul 19, 2016 at 6:22 PM, Bill Deegan <bill at baddogconsulting.com>
wrote:

> Steve,
>
> Do you have any python logic creating the files in the SCons process? (a
> builder where the builder actuall does open('file','w').. to create the
> file?
>
> -Bill
>
> On Tue, Jul 19, 2016 at 7:55 AM, William Blevins <wblevins001 at gmail.com>
> wrote:
>
>> Steve,
>>
>> As a final thought, I have used SCons to build HT 24-core server blades
>> (so 48 w/ HT) that RAM boot (no harddrive at RT). If it was specific to the
>> core package, I imagine I would have seen this issue tons.
>>
>> V/R,
>> William
>>
>> On Tue, Jul 19, 2016 at 12:52 PM, William Blevins <wblevins001 at gmail.com>
>> wrote:
>>
>>> Vasily,
>>>
>>> I think you are referring to Precious.
>>>
>>> V/R,
>>> William
>>>
>>> On Tue, Jul 19, 2016 at 9:51 AM, Vasily <just.one.man at yandex.ru> wrote:
>>>
>>>> Hi Steve,
>>>>
>>>> What does your build do with this file except installing it? Is it used
>>>> by a compiler or some other tool?
>>>>
>>>> Also, if I'm not mistaken, default SCons behavior is to remove the
>>>> target before performing any action to regenerate it, so this might be the
>>>> source of the exception you're seeing. There seems to be a way to turn off
>>>> the removal part of the action, so you may want to check the manual and see
>>>> if it helps you.
>>>>
>>>> P.S. Based on file path I assume you're working on Windows, is this
>>>> path a network one?
>>>>
>>>> Thanks,
>>>> Vasily
>>>> 19 июля 2016 г. 11:02 пользователь "Hill, Steve (FP COM)" <
>>>> Steve.Hill at cobham.com> написал:
>>>>
>>>> Thanks William.
>>>>>
>>>>>
>>>>>
>>>>> I’ve checked the dependency tree and, as far as I can tell, it looks
>>>>> OK.
>>>>>
>>>>> We do have custom scanners – using env.Scanner(_scan_domain_header) –
>>>>> but not for the files that are affected.
>>>>>
>>>>> As far as I can see, the fix for that Python bug never made it into
>>>>> 2.6.x but it appears always to result in a “No child processes” OSError so
>>>>> I don’t believe that is what I am seeing.
>>>>>
>>>>>
>>>>>
>>>>> Note that I have (temporarily) changed the decider below to simply
>>>>> return True (without doing anything with the file) and I still see the
>>>>> issue so the decider doesn’t seem to be relevant to the issue.
>>>>>
>>>>>
>>>>>
>>>>> Does anyone have any more ideas? This is becoming a major issue for
>>>>> our automated builds…
>>>>>
>>>>>
>>>>>
>>>>> S.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Steve,
>>>>>
>>>>> I of course should have asked the obvious question, do you know if you
>>>>> dependency tree has missing dependencies? This tends to be a common issue.
>>>>>
>>>>> V/R,
>>>>>
>>>>> William
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Jul 15, 2016 at 5:14 PM, William Blevins <
>>>>> wblevins001 at gmail.com> wrote:
>>>>>
>>>>> Steve,
>>>>>
>>>>> I'm not aware of any specific issue with install, but there are some
>>>>> possible issues that I am aware:
>>>>>
>>>>>    1. If you have custom scanners, make sure they implement from
>>>>>    SCons.Scanner.Current and not SCons.Scanner.Base; otherwise, you might have
>>>>>    concurrent file access between implicit scanning operations and other
>>>>>    processes.
>>>>>    2. There was a big subprocess bug in python 2.6 that carried
>>>>>    through several other major versions:
>>>>>    https://bugs.python.org/issue1731717. I would check to see that
>>>>>    your version of 2.6 contains the patch for this issue.
>>>>>
>>>>> V/R,
>>>>>
>>>>> William
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Jul 15, 2016 at 9:30 AM, Hill, Steve (FP COM) <
>>>>> Steve.Hill at cobham.com> wrote:
>>>>>
>>>>> I’m having a problem where, with parallel builds (most people use 8,
>>>>> 12 or 16 threads), we occasionally get failures like the following:
>>>>>
>>>>>
>>>>>
>>>>> F:\<directory path>\hw_cfgs\1Server_1TM_6C66_2U.cfg: The process cannot access the file because it is being used by another process
>>>>> scons: building terminated because of errors.
>>>>>
>>>>>
>>>>>
>>>>> We are running Python 2.6.5 (with pywin32) and SCons 2.3.6. This file
>>>>> is being copied due to a Install() call. Note that, for various historical
>>>>> reasons, we have the following decider for these Installs:
>>>>>
>>>>>
>>>>>
>>>>> def _copy_decider(dependency, target, prev_ni):
>>>>>
>>>>>     target = str(target.abspath)
>>>>>
>>>>>     dependency = str(dependency.abspath)
>>>>>
>>>>>     if os.path.isfile(target) and os.path.isfile(dependency):
>>>>>
>>>>> *        # By default, filecmp.cmp assumes that files with identical
>>>>> os.stat signatures*
>>>>>
>>>>> *        # (which includes the inode) are the same file and, hence,
>>>>> must be the same.*
>>>>>
>>>>> *        # However, on Windows, there is no inode - it appears to be
>>>>> set to zero - so*
>>>>>
>>>>> *        # any two files with the same size and
>>>>> access/creation/modification times*
>>>>>
>>>>> *        # will have the same os.stat signature, leading to a false
>>>>> positive. For this*
>>>>>
>>>>> *        # reason, we must force it to do an actual file comparison by
>>>>> setting shallow*
>>>>>
>>>>> *        # to False*
>>>>>
>>>>>         return not filecmp.cmp(target, dependency, shallow = False)
>>>>>
>>>>>     else:
>>>>>
>>>>> *        # Either one of the dependency or target isn't a file or one
>>>>> of the files*
>>>>>
>>>>> *        # (presumably the target) isn't there so do the copy*
>>>>>
>>>>>         return True
>>>>>
>>>>>
>>>>>
>>>>> Note also that our IT department claims that virus checkers are
>>>>> disabled within the directory where the build is being performed (and we
>>>>> certainly have not seen any indication in the virus checker console to
>>>>> suggest that to be incorrect).
>>>>>
>>>>>
>>>>>
>>>>> Does anyone have any thoughts as to what the problem might be?
>>>>>
>>>>>
>>>>>
>>>>> Thanks in advance,
>>>>>
>>>>>
>>>>>
>>>>> S.
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Scons-users mailing list
>>>>> Scons-users at scons.org
>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Scons-users mailing list
>>>>> Scons-users at scons.org
>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>
>>>>>
>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>>>
>>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20160719/ef433094/attachment-0001.html>