[Scons-users] CacheDir race during parallel Windows builds?

[Jason Kenny]

That is basically the test. If you open with “w” the file will be sized to 0 bytes. That is why I used “a”. However “w” given the same result. If done correctly I seemed to confuse scons into thinking that the obj file is now 0 bytes and will not rebuild the file correctly until you delete it manually or with a clean. That is a different issue however

 

Jason

 

From: Scons-users [mailto:scons-users-bounces at scons.org] On Behalf Of Bill Deegan
Sent: Monday, August 8, 2016 7:52 AM
To: SCons users mailing list <scons-users at scons.org>
Subject: Re: [Scons-users] CacheDir race during parallel Windows builds?

 

Jason,

If you open the file "w" and leave it open and then try the link step, that's likely a good simulation of the current problem.

 

On Sun, Aug 7, 2016 at 6:35 PM, Jason Kenny <dragon512 at live.com <mailto:dragon512 at live.com> > wrote:

 

Hi,

 

So let me go over what we know:

1) no cache and serial build -> worked

2) no cache and -j build -> Worked

3) cache and serial build -> Worked

4) cache and -j build -> Fail constantly

 

>From this it would seems to be having the cache on and a parallel build. My guess is that a thread was doing something with the file and the main thread was doing something else to have this happen. 

Then I did a simple test. 

 

Basically I opened an object file I just built manually in different python interactive shell I opened it only as “r” and left it open. 

I could link the program in a different shell.

If it opened the file with “a” or “r+” ( anything with implied write), the program would not link with a “LINK : fatal error LNK1104: cannot open file 'hello.obj'”. 

 

I am guessing that the linker has some “exclusive” read mode set that fails is the object file is opened with a write mode. If I try to do this on Linux it looks like it works fine even is python has an open handle Write handle open. Also if I do this with different processes on windows it seem to be fine as well. I think the linker is locking the file while it does some work to prevent it from changing while it is busy making the PE format of the finial output.

 

Based on this I would suggest we have a race in SCons with cacheDir set in which python has a write mode handle open on the object file that was not closed yet. I did this test on Windows 10 with VS 2015 ( I tested linux on the bash shell feature on windows 10 and doubled checked on Ubuntu in a VM). The race I would assume to be something with the actions running a link command while the main thread is doing something with that file. Or there is something else touching that file. 

 

I don’t know enough of the pathways with cacheDir at the moment to say want would be going on. 

 

I don’t think Parts File tweaks would help much with solving this problem at the moment. Given 4) is the only time this happen, this seems to be a SCons issue.

 

Jason

 


_______________________________________________
Scons-users mailing list
Scons-users at scons.org <mailto:Scons-users at scons.org> 
https://pairlist4.pair.net/mailman/listinfo/scons-users

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20160808/7f845cde/attachment-0001.html>