[Scons-users] Scanner / Builder Race Condition?

[Plunket, Tom]

I have the same behavior in SCons 2.3.5, unfortunately I haven’t been able to take the time to strip it down to something that can be shared easily.

I have an input file that is processed to generate another file. This output file is then scanned for dependencies to put into the final output file. Let’s call this file the intermediate file below…

When starting from clean, everything works fine.

When incrementally building, the soon-to-be-replaced intermediate file is scanned for dependencies. Maybe some of the dependencies are changed, and they are then processed while the intermediate file is regenerated (which only happens when necessary and it takes a long time so I’ve never had the experience of the file being changed while scanning).

Where this causes a real problem is when the list of dependencies includes a target that doesn’t have a source file available. Of course SCons complains that it can’t build that target, but it aborts the build even if the target isn’t going to be in the intermediate file when it’s regenerated. This also halts the clean operation, I get an error that the source for this other target doesn’t exist and the clean is aborted!

I’ve recently downloaded the SCons source but I’m not really sure where to start (and of course I have other things that need to be done!).

To put this into a programming context for example purposes, the primary input would be like a project file, like a a VCPROJ from Visual Studio. That would be processed to generate a text file containing a list of files to compile along with per-file compilation options. Then I link that all together into an executable. The list of files is scanned even before the vcproj would be re-exported (and then scanned again after export) but any missing source files cause an error that is unrecoverable short of just deleting the file list/intermediate file manually. NOTE: this is for example purposes only that demonstrate the flow, my actual processing involves binary files, animation programs and their data files, and lots of graphics data processing. I am not trying to reinvent the Visual Studio build package.

From: Scons-users [mailto:scons-users-bounces at scons.org] On Behalf Of Brian Cody
Sent: Monday, February 22, 2016 12:23 PM
To: scons-users at scons.org
Subject: [Scons-users] Scanner / Builder Race Condition?

Hey folks,
We have a command builder that creates a file, let's call it project.prj. The project.prj file is a '.prj' file, which we have a scanner for.

create_project_target = env.Command(project_prj, [project_file_sources], create_project_function)
Then I have two builds that take it as a source:

version_file = env.GenerateVersionFile(bin_file_name, create_project_target)
binary = env.Compile(bin_file_name, [create_project_target, version_file])
If the project.prj doesn't already exist, everything works as expected. The prj file gets created and then scanned, and then the builds proceed.
The problem is when I change a file in project_file_sources. SCons will invoke the scanner on the project.prj file before it runs any build. That's OK, it's just a potential waste of time. The problem is that (with multiple jobs) it will start running re-building the project.prj file, and before waiting for it to be done, it'll start scanning the file again! The scanner will now throw an exception.
Simplified log:
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
Start Scanning
Done Scanning
scons: rebuilding `project.prj' because `project_prj_source' changed
create_project_function(["project.prj"], [project_prj_source_list])
start of create_project_function
Start Scanning
PRJ SCANNER EXPLODE!!! <--- helpful try/except block
end of create_project_function
scons: *** [bin_file_name] IndexError : list index out of range <--- The exact error varies, it's just what happens when the file you're scanning is suddenly truncated.
I would rather not disable the multiple jobs, and multiple invocations of SCons really isn't acceptable either. Is there any proper way to get SCons to wait to scan a file that it has already determined needs to be rebuilt?
Also if we perform an Execute(Delete(project_prj)) at the top of the SConstruct it "works", however due to how our version file works it will always perform the build of the bin_file_name. This is extremely costly in this case! (It's a Xilinx build).
Thank you!
-Brian






IMPORTANT CONFIDENTIALITY NOTICE:

This E-mail(including any documents referred to in, or attached, to this E-mail) may contain information that is personal, confidential or the subject of copyright or other proprietary rights in favor of Aristocrat, its affiliates or third parties. This E-mail is intended only for the named addressee. Any privacy, confidence, copyright or other proprietary rights in favor of Aristocrat, its affiliates or third parties, is not lost because this E-mail was sent to you by mistake.

If you received this E-mail by mistake you should: (i) not copy, disclose, distribute or otherwise use it, or its contents, without the consent of Aristocrat or the owner of the relevant rights; (ii) let us know of the mistake by reply E-mail or by telephone (US 1-877-274-9661, or  AU +61 2 9013 6000); and (iii) delete it from your system and destroy all copies.

Any personal information contained in this E-mail must be handled in accordance with applicable privacy laws.

Electronic and internet communications can be interfered with or affected by viruses and other defects. As a result, such communications may not be successfully received or, if received, may cause interference with the integrity of receiving, processing or related systems (including hardware, software and data or information on, or using, that hardware or software). Aristocrat gives no assurances in relation to these matters.

If you have any doubts about the veracity or integrity of any electronic communication we appear to have sent you, please call (US 1-877-274-9661, or  AU +61 2 9013 6000) for clarification.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20160227/617797ae/attachment-0001.html>