[Scons-users] Mark existing file as up-to-date

[William Blevins]

Pedro,

Extension to previous comments (now that I'm not sending a response via
smartphone).

@William, the default build action is something like

> "foo $IN > $OUT"

> I can think of three situations where SCons flags $OUT to be rebuilt,

> 1. I might change the default build action for something like,

> "echo Building $OUT; foo $IN > $OUT"

> 2. I have recompiled foo, but I know that nothing changes in the output.

> 3. I may have changed input $IN, but I know that nothing changed in the

> output.



*1. For sake of argument, I am going to take this example exactly as is
(that's all I have to go on).*

Execute scons with the "-n" argument.  See full documentation here:
http://www.scons.org/doc/production/HTML/scons-man.html


> -n, --just-print, --dry-run, --recon




No execute. Print the commands that would be executed to build any

> out-of-date target files, but do not execute the commands.


You can use this in conjunction with any of the following arguments without
changing the build action and probably get everything you wanted and more:


> --debug=explain

>



Print an explanation of precisely why *scons* is deciding to (re-)build any

> targets. (Note: this does not print anything for targets that are *not*

>  rebuilt.)


--debug=prepare

>



Print a line each time any target (internal or external) is prepared for

> building. *scons* prints this for each target it considers, even if that

> target is up to date (see also --debug=explain). This can help debug

> problems with targets that aren't being built; it shows whether *scons* is

> at least considering them or not.

>




> --debug=presub

>



Print the raw command line used to build each target before the

> construction environment variables are substituted. Also shows which

> targets are being built by this command. Output looks something like this:


 $ scons --debug=presub

>

> Building myprog.o with action(s):

>

>  $SHCC $SHCFLAGS $SHCCFLAGS $CPPFLAGS $_CPPINCFLAGS -c -o $TARGET $SOURCES

>

> ...

>

> Without taking your example literally, I assume there is still a better

way to accomplish your goal because I cannot think of a reason to change
the build commands like that (I may have a frail imagination though).
*2. I imagine you are using a code generation tool or using SCons to
pipeline multi-stage data processing.*

Unless you are changing the default config, then updating foo in "foo
$INPUT > $OUTPUT" shouldn't cause a rebuild unless you are doing something
like *Execute( 'foo $INPUT > $OUTPUT' )* without using a builder.
 Hopefully, a developer with knowledge in this area will chime in.

See full documentation here:
http://www.scons.org/doc/production/HTML/scons-man.html


> --config=

> *mode*




This specifies how the *Configure* call should use or generate the results

> of configuration tests. The option should be specified from among the

> following choices:

>



--config=auto

>



scons will use its normal dependency mechanisms to decide if a test must be

> rebuilt or not. This saves time by not running the same configuration tests

> every time you invoke scons, but will overlook changes in system header

> files or external commands (such as compilers) if you don't specify those

> dependecies explicitly. This is the default behavior.

>



--config=force

>



If this option is specified, all configuration tests will be re-run

> regardless of whether the cached results are out of date. This can be used

> to explicitly force the configuration tests to be updated in response to an

> otherwise unconfigured change in a system header file or compiler.

>



--config=cache

>



If this option is specified, no configuration tests will be rerun and all

> results will be taken from cache. Note that scons will still consider it an

> error if --config=cache is specified and a necessary test does not yet have

> any results in the cache.



*3.A. You should be able to resolve this by changing from the default
decider function.*

See documentation:
http://www.scons.org/doc/production/HTML/scons-user.html#chap-depends

By default SCons does MD5SUM comparison, I misspoke before (or maybe the
default changed in a recent version).  Since SCons still attempts a
rebuild, I can determine that your $INPUT file(s) creation process is
non-deterministic.

Since you *know* that the $OUTPUT will not be updated, maybe you can make a
custom decider that will let SCons *know*.  This may not be possible (or at
least easy).

*3.B. I don't know anything about the scanner you are using, so this may
(or may not) be helpful.*

You may be able to turn off rebuilding $OUTPUT if the $INPUT is implicit
based on scanning.  You didn't mention anything about your scanner, so it
may not be helpful.

 --implicit-cache

>



Cache implicit dependencies. This causes *scons* to use the implicit

> (scanned) dependencies from the last time it was run instead of scanning

> the files for implicit dependencies. This can significantly speed up SCons,

> but with the following limitations:


*scons* will not detect changes to implicit dependency search paths (e.g.

> *CPPPATH*, *LIBPATH*) that would ordinarily cause different versions of

> same-named files to be used.

> *scons* will miss changes in the implicit dependencies in cases where a

> new implicit dependency is added earlier in the implicit dependency search

> path (e.g. *CPPPATH*, *LIBPATH*) than a current implicit dependency with

> the same name.

>



--implicit-deps-changed

>



Forces SCons to ignore the cached implicit dependencies. This causes the

> implicit dependencies to be rescanned and recached. This implies

> *--implicit-cache*.

> --implicit-deps-unchanged

>



Force SCons to ignore changes in the implicit dependencies. This causes

> cached implicit dependencies to always be used. This implies

> *--implicit-cache*.



*Without more information*, this is about the best I can do.  The more
detail you give, the better the responses will be: mock dependency trees,
your custom scanners/builders, etc.

V/R,
William


On Fri, May 9, 2014 at 10:36 AM, William Blevins <wblevins001 at gmail.com>wrote:


> Would a custom decider help in this case?  The default is timestamp but

> base SCons package also provides MD5 sum and you can make your own.

>

> V/R,

> William

> On May 9, 2014 4:22 AM, "Pedro Inácio" <pedromiragaia at gmail.com> wrote:

>

>> @all, Thank you for your feedback.

>>

>> @William, the default build action is something like

>> "foo $IN > $OUT"

>> I can think of three situations where SCons flags $OUT to be rebuilt,

>> 1. I might change the default build action for something like,

>> "echo Building $OUT; foo $IN > $OUT"

>> 2. I have recompiled foo, but I know that nothing changes in the output.

>> 3. I may have changed input $IN, but I know that nothing changed in the

>> output.

>>

>> In all three situations SCons is right. OUT should be rebuilt.

>> However, I would like to have control and be able to tell SCons that in

>> all three situations it does not need to rebuild the file.

>>

>> @Bill, good idea. I have tried this but it doesn't seem to work as

>> expected. Perhaps I am doing something wrong. I believe the most pragmatic

>> approach for me would be to ignore the entire build string.

>> I replaced

>> BarBuilder=Builder(action='''

>> foo $SOURCE > $TARGET

>> ''')

>> with

>> BarBuilder=Builder(action='''$(

>> foo $SOURCE > $TARGET

>> $)''')

>> but for some reason SCons keeps flagging files for rebuild whenever I

>> change the build action, despite the $( $).

>>

>>

>>

>> On 9 May 2014 00:35, Dirk Bächle <tshortik at gmx.de> wrote:

>>

>>> Hi Pedro,

>>>

>>>

>>> On 08.05.2014 23:32, Pedro Inácio wrote:

>>>

>>>> I am using SCons for data processing.

>>>> One of the perks is that, unlike traditional compilation projects, each

>>>> file in my project might have required several hours to compute.

>>>> Re-building existing files is, in general, not acceptable.

>>>>

>>>> [...

>>>> Any ideas?

>>>>

>>>>  I would simply split up your build into two single phases, something

>>> like "SConstruct.preprocess" where you update all your precious files and

>>> "SConstruct.postprocess" for all the build steps that use them as sources.

>>> Then you can manually decide when it's efficient to run the preprocess

>>> step again.

>>>

>>> Hope this helps.

>>>

>>> Best regards,

>>>

>>> Dirk

>>>

>>>

>>> _______________________________________________

>>> Scons-users mailing list

>>> Scons-users at scons.org

>>> http://four.pairlist.net/mailman/listinfo/scons-users

>>>

>>

>>

>> _______________________________________________

>> Scons-users mailing list

>> Scons-users at scons.org

>> http://four.pairlist.net/mailman/listinfo/scons-users

>>

>>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://four.pairlist.net/pipermail/scons-users/attachments/20140509/bd5b800d/attachment-0001.html