[Scons-users] Mark existing file as up-to-date

[Pedro Inácio]

@all, thanks

My original question was whether it was possible to tell SCons to accept
the existing files as up-to-date irregardless of how there were generated.
It seems clear to me now, from the mailing list feedback and reading the
manual back and forth that such feature is not available out-of-the the box.

I have mentioned three items where I would like to control whether files
should be rebuild or not.
#1, it seems that SCons always checks for changes in the build action. Even
when I change the default MD5 decider to the makelike decider, SCons
detects changes in the build action and rebuilds the files.
I believe the best solution for me is to use the $( $) to ignore the whole
build action.
It would be nice to have flag to turn of detecting changes in the build
command.

@Dirk, I'd really like to know what exactly that means "doesn't seem to

> work as expected". Are you getting errors or warnings, if yes how do they

> look like? How did you try (executed commands), which answers did you

> expect and what was returned instead?



My first tests did not work as expected. Apparently the $( and $) must be
on the same line otherwise they do not work as expected. For example

customBuilder=Builder(action='''$(
cat $SOURCE > $TARGET
$)''')

does not work, while

customBuilder=Builder(action='''
$( cat $SOURCE > $TARGET $)
''')

does.

@Dirk, your suggestion of a null build is also a good idea.
[SOLVED]

#2 According to @William this is not an issue.
[SOLVED]

#3 As suggested by several people, this can be solved by changing the
decider function. Using the makelike decider should work fine. Whenever I
want SCons to accept existing files as up-to-date, all I have to do is
touch them.
[SOLVED]

It seems that I have found the solution I was looking for, so, from my part
this issue is closed.

I would like to leave a final thought regarding,


> @Dirk, "Picking up these kind of dependencies is one of SCons' strongest

> points, that's why it takes control for you usually. If you want to be in

> charge, what exactly do you want SCons to do for you (what's left that a

> script couldn't do)? This may seem strange to hear it on the SCons user ML,

> but maybe you're using the wrong tool for the job...just sayin'."



In my original email I started by stating that I was using SCons for data
processing.
I am a researcher and for my work I deal with processing satellite data. I
begin with raw data collected by the satellite and proceed to apply a
series of processing steps until we obtain the final data products.
Until now I did this using traditional bash scripts, but it is obvious that
using a tool like SCons has multiple advantages to ensure the
reproducibility and correctness of the final data products.
This is even more critical taking into account that the processing time
requires several days on large computers.
You are right, and at first sight, it might seem that I am using the wrong
tool for the job.
But only because 99% of SCons (and other build tools) are used for software
compilation.
Fundamentally, compilation is a very specific case of data processing.
SCons is a program which builds a graph and goes through it to build only
what is necessary.
This is exactly what we need for data processing.
We need it as much as software compilation projects do.
Just imagine yourself building your executable using only bash scripts!
To finalize, my point is, SCons is a great tool and has great potential
also outside of software compilation domain where such a tool is much
needed and few people know about it.

P








On 11 May 2014 11:15, Dirk Bächle <tshortik at gmx.de> wrote:


> Hi Pedro,

>

>

> On 09.05.2014 10:21, Pedro Inácio wrote:

>

>> @all, Thank you for your feedback.

>>

>> @William, the default build action is something like

>> "foo $IN > $OUT"

>> I can think of three situations where SCons flags $OUT to be rebuilt,

>> 1. I might change the default build action for something like,

>> "echo Building $OUT; foo $IN > $OUT"

>> 2. I have recompiled foo, but I know that nothing changes in the output.

>> 3. I may have changed input $IN, but I know that nothing changed in the

>> output.

>>

>> In all three situations SCons is right. OUT should be rebuilt.

>> However, I would like to have control and be able to tell SCons that in

>> all three situations it does not need to rebuild the file.

>>

>>

> I just had another idea that might help you out. Instead of preventing

> SCons from trying to rebuild your files, you could make the action a "null"

> build. You could define your action as

>

>   $MYFOO $SOURCE > $TARGET

>

> and then set the variable "MYFOO" in your environment either to "foo" as

> before, or to a new build script "dummy_foo". The latter would obviously be

> doing nothing, so the output files are left untouched.

>

> The drawback of this method would be that you'd get a lot more screen

> output. Another point might be the MD5 hash summing for your output files.

> If they are rather large ( > 100MB for example), you'll lose some time

> since they'd get rehashed on every build attempt.

>

> But apart from that, it might be the solution you're searching for. You

> then just have to link a command-line option to how you set "MYFOO", or set

> this in your SConstruct manually.

>

> Hope this helps.

>

>

> Regards,

>

> Dirk

>

> _______________________________________________

> Scons-users mailing list

> Scons-users at scons.org

> http://four.pairlist.net/mailman/listinfo/scons-users

>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://four.pairlist.net/pipermail/scons-users/attachments/20140512/04778d8d/attachment-0001.html