[Scons-users] CacheDir race during parallel Windows builds?

[Bill Deegan]

Jason,

If you open the file "w" and leave it open and then try the link step,
that's likely a good simulation of the current problem.


On Sun, Aug 7, 2016 at 6:35 PM, Jason Kenny <dragon512 at live.com> wrote:

>
>
> Hi,
>
>
>
> So let me go over what we know:
>
> 1) no cache and serial build -> worked
>
> 2) no cache and -j build -> Worked
>
> 3) cache and serial build -> Worked
>
> 4) cache and -j build -> Fail constantly
>
>
>
> From this it would seems to be having the cache on and a parallel build.
> My guess is that a thread was doing something with the file and the main
> thread was doing something else to have this happen.
>
> Then I did a simple test.
>
>
>
> Basically I opened an object file I just built manually in different
> python interactive shell I opened it only as “r” and left it open.
>
> I could link the program in a different shell.
>
> If it opened the file with “a” or “r+” ( anything with implied write), the
> program would not link with a “LINK : fatal error LNK1104: cannot open file
> 'hello.obj'”.
>
>
>
> I am guessing that the linker has some “exclusive” read mode set that
> fails is the object file is opened with a write mode. If I try to do this
> on Linux it looks like it works fine even is python has an open handle
> Write handle open. Also if I do this with different processes on windows it
> seem to be fine as well. I think the linker is locking the file while it
> does some work to prevent it from changing while it is busy making the PE
> format of the finial output.
>
>
>
> Based on this I would suggest we have a race in SCons with cacheDir set in
> which python has a write mode handle open on the object file that was not
> closed yet. I did this test on Windows 10 with VS 2015 ( I tested linux on
> the bash shell feature on windows 10 and doubled checked on Ubuntu in a
> VM). The race I would assume to be something with the actions running a
> link command while the main thread is doing something with that file. Or
> there is something else touching that file.
>
>
>
> I don’t know enough of the pathways with cacheDir at the moment to say
> want would be going on.
>
>
>
> I don’t think Parts File tweaks would help much with solving this problem
> at the moment. Given 4) is the only time this happen, this *seems* to be
> a SCons issue.
>
>
>
> Jason
>
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20160808/b1e78ec4/attachment.html>