[Scons-users] Bug in up_to_date for directory targets

Tue Mar 21 12:53:18 EDT 2017

Manish,

Ah, ok. I was confused when your original example seemed so simple. You
should probably create a custom builder then:
https://bitbucket.org/scons/scons/wiki/ToolsForFools

It isn't a bug that SCons doesn't rebuild targets it doesn't have. The
issue is that you need to tell SCons about said targets. Do this via a
custom builder.

I haven't used npm, but most package managers have a way to list the
contents of a package, so just convert that list into a target set. I would
need more information about the source -> target dependency chain in terms
of actual vs expected in order to say much more.

Sorry for the fix response and run, but normally I don't get time to answer
emails this early in the day. Let us know if you think a custom builder
will not solve your problem.

V/R,
William

On Tue, Mar 21, 2017 at 12:22 PM, Manish Vachharajani <
manishv at unbounded.systems> wrote:

> If you look at my original message, the problem is that in my real use
> case, I have no idea what the file targets will be.  The only target I know
> will be built by the builder is a set of directories.  Your example simply
> causes scons to have a file target in the second Command builder which
> works around the bug as I describe in my original email.  In real life, my
> builder looks something like this;
>
> env.Command(['node_modules/minimist', 'node_modules/@types/node, ...],
> #these are directories
>             ['package.json'],
>             ['npm install --global-style'],
>             chdir=Dir('.')) #chdir because npm doesn't have a way to
> install elsewhere
>
> In this case, I have no idea what files npm install will create.  The
> targets array is actually created by reading package.json and looking at
> the dependencies and devDependencies fields of the main object, so those
> aren't even hard-coded in the SConstruct/SConscript.
>
> Again, if I fake out scons by have a 'node_modules/.built' File target in
> the list and add a 'touch node_modules/.built' command to the builder after
> 'npm install', all will work as expected because there is a File target.
> In your example, things work because the second builder has a file target
> and SCons will only run the first builder in your example when foo does not
> exist, regardless of the state of foo-contents.txt in relation to what is
> in .sconsign.  That is the essence of the bug, if the target directory
> exists and it is the only target, the builder will not run regardless of
> the state of the sources.
>
> Manish
>
>
> On Mon, Mar 20, 2017 at 9:09 PM, William Blevins <wblevins001 at gmail.com>
> wrote:
>
>> Manish,
>>
>> * Why does my example not work?
>>
>> SCons Comand Builder parameters are target(s), source(s), command(s):
>> http://scons.org/doc/HTML/scons-user.html#chap-builders-commands
>>
>> Foo is the target parameter, and foo-contents.txt is the source
>> parameter. Targets explicitly depend on sources. In your example, directory
>> foo depends on source file foo-contents.txt; target file copy does not
>> depend on source file.
>>
>> * How do I fix this issue?
>>
>> 1. Call Command Builder correctly: Command('foo', 'foo-contents', 'mkdir
>> $TARGET') which creates file dependency on directory, and then
>> Command('foo/foo-contents.txt', 'foo-contents.txt', 'cp $SOURCE
>> $TARGET') which creates file target dependency on file source.
>>
>> 2: [PREFERRED] Use the SCons FileSystem factory
>> http://scons.org/doc/HTML/scons-user.html#chap-factories
>>
>> V/R,
>> William
>>
>>
>> On Mon, Mar 20, 2017 at 1:47 PM, Manish Vachharajani <
>> manishv at unbounded.systems> wrote:
>>
>>> I believe there is a bug in how up to date is computed for file nodes
>>> and directory targets.  Below is a SConstruct and run output illustrating
>>> the bug.
>>>
>>> ##############
>>> # SConstruct
>>> env = Environment()
>>> targets = env.Command('foo',
>>>                       'foo-contents.txt',
>>>                       ['mkdir foo',
>>>                        'cp foo-contents.txt foo'])
>>> env.Default(targets)
>>>
>>> ##############
>>> # Shell log
>>>
>>> # Create the file that will be installed into foo/foo-contents.txt
>>> $ echo "Hello, " >> foo-contents.txt
>>> # All is well, scons runs the command builder as expected
>>> $ scons
>>>
>>> scons: Reading SConscript files ...
>>> scons: done reading SConscript files.
>>> scons: Building targets ...
>>> mkdir foo
>>> cp foo-contents.txt foo
>>> scons: done building targets.
>>>
>>> # All is well, the target is up to date as expected
>>> $ scons
>>>
>>> scons: Reading SConscript files ...
>>> scons: done reading SConscript files.
>>> scons: Building targets ...
>>> scons: `foo' is up to date.
>>> scons: done building targets.
>>>
>>> # Uh-oh, source is updated but the target is considered up to date!
>>> $ echo "World!" >> foo-contents.txt
>>> $ scons
>>>
>>> scons: Reading SConscript files ...
>>> scons: done reading SConscript files.
>>> scons: Building targets ...
>>> scons: `foo' is up to date.
>>> scons: done building targets.
>>>
>>> For the bug to trigger, all targets for a builder must be directories.
>>> The target directory must exist and the source file must exist and be out
>>> of date due to a content update (resulting in a signature mismatch).
>>>
>>> I think the problem is that is_up_to_date for File nodes only checks the
>>> signatures of its children, which avoids this bug as long as one of the
>>> builder targets is a File and not a Directory.  However, I haven't done
>>> enough analysis to be sure that this is the issue.
>>>
>>> A work around is to make sure that you have a file (or create a dummy
>>> .built file) and include it as a target.
>>>
>>> This is showing up specifically for us when trying to calculate
>>> dependencies for a package manager (npm) where we have no insight into the
>>> contents of the resulting directory.
>>>
>>> Manish
>>>
>>> _______________________________________________
>>> Scons-users mailing list
>>> Scons-users at scons.org
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20170321/9721833f/attachment.html>