[Scons-users] How to restart a scanner if it cannot provide all dependencies.

[Michael Hasling]

Hello,

I've written a somewhat lengthy description of the issue I'm trying to solve, but the essence of the problem is that I need to be able to run a scanner potentially in several iterations with the SCons engine building dependencies that have been reported between each iteration before restarting the scanner on the original source file.

We build some of our software libraries using a custom tool which has a Tcl interpreter at its heart. In its non-GUI mode the tool executes a top-level Tcl script to transform the input files and create the outputs. When we started using SCons we created a SCons builder and scanner for the tool. The builder is relatively straightforward, but the scanner has turned out to be less so. Some of our Tcl scripts are generated, so they may not exist when the top-level Tcl script file is scanned.

The scanner needs to implement a Tcl interpreter, since determining dependencies can require interpreter state data only available in the Tcl engine. We use the python Tkinter module, and we register command callbacks for those commands which would reveal an implicit dependency or affect the interpreter state. The built-in source command is also re-mapped. Tcl fragments that only have tool-related side-effects are just ignored. If the scanner detects a Tcl file that should be sourced, and if this file has not yet been created then the scanner cannot source this script, so it cannot detect dependencies in this file and the Tcl engine state will no longer match the one in the builder tool, potentially making other dependencies inaccurate.

My understanding of the architecture for SCons scanners is that each file is only scanned once, and if this scan reveals dependencies that are not yet built then the hierarchical mode should be used so that these dependencies will be scanned after they have been built. In this case each file can be scanned statically and independently.

That approach does not seem to work for our custom tool since the scanning cannot really be done statically on individual files, but rather we need to always start with the top-level script and execute the script to determine the dependency tree. I haven't found an obvious way to get SCons to consider that the top-level script needs rescanning.

Currently we're solving these issues when they occur by adding explicit dependencies, but I'd prefer to have a more robust generalised solution if that's possible.

Before I try out some of my more wild ideas (which I'm happy to share in a follow-up!) I wondered if I'm missing something obvious?

Regards,
Michael

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 476 bytes
Desc: not available
Url : http://four.pairlist.net/pipermail/scons-users/attachments/20140404/7a654d9f/attachment-0001.pgp