[Scons-users] find new targets from other targets?

Tue Feb 25 12:13:20 EST 2020

Scott,

A Scanner processes an input file to look for dependencies.
If your source jinja source files only have simple ifdef type constructs
turning includes on and off the you can write a simple python script to
only process those and look for the include statements. This does NOT
require fully processing your source files.

See: https://scons.org/doc/production/HTML/scons-man.html#scanner_objects
And: https://scons.org/doc/production/HTML/scons-user.html#chap-scanners

Just as SCons doe NOT have to run a full c preprocessor to find it's
include files (recursively) to build up the list of dependencies, neither
should it be necessary to do so for Jinja2.

Also SCons can cache such dependencies (updating only when files are
updated) thus you'll not have to fully process every jinja2 source file
twice on every run. And you should only run full jinja2 process when needed.

If that's not clear enough. I'm not sure how to better explain it.

-Bill

On Mon, Feb 24, 2020 at 10:28 PM Scott Johnson <scott.johnson at arilinc.com>
wrote:

> I’m sure I understand what a scanner is. “Process the jinja source files”
> is, in my case, the exact same thing as building the target file.
>
> So the scanner would have to process the jinja source file once to find
> the “dependencies” (which are not actually dependencies in my case, just
> additional targets), and then Scons would process the jinja source file a
> second time to build the target file.
>
> In addition to having to jinja everything twice, this would also add
> unnecessary dependencies to the graph, such that e.g. if pkg1.F changed,
> Scons would think it needs to rebuild top.F, which is not actually
> necessary.
>
> Thank you for the help but it’s become clear that Scons isn’t designed to
> work the way I need.
>
>
> On Feb 24, 2020, at 6:25 PM, Bill Deegan <bill at baddogconsulting.com>
> wrote:
>
> No. I mean if you write a scanner it can process the jinja source files to
> find the dependencies. Just like SCons does with C files.
>
> Thus a Scanner..
> Did you read the users guide and manpage?
>
> On Mon, Feb 24, 2020 at 2:33 PM Scott Johnson <scott.johnson at arilinc.com>
> wrote:
>
>> If I understand you correctly, that means that every time I run a clean
>> build via Scons I have to run every file through Jinja once to find all the
>> targets, then a second time through Jinja to build the outputs. Even an
>> incremental build will have to run every file through Jinja once before it
>> discovers that there’s nothing to rebuild.
>>
>> At that point what does Scons buy me versus a simple Python script that
>> builds all files every time? With a script, clean builds will be faster and
>> incremental builds will be no slower.
>>
>>
>>
>> On Feb 24, 2020, at 7:33 AM, Bill Deegan <bill at baddogconsulting.com>
>> wrote:
>>
>> Actually if your scanner reading the jinja source files can navigate the
>> jinja if's then you don't need to produce the files before you know what
>> files it depends on.
>>
>> (not that different from c preprocessors #ifdef..)
>>
>> On Sun, Feb 23, 2020 at 10:31 PM Scott Johnson <scott.johnson at arilinc.com>
>> wrote:
>>
>>> Yes, evaluating the if's is what Jinja does, and that's how the source
>>> file gets processed into the target file.
>>>
>>> So I need to build the target, then I can parse the target to find
>>> additional targets that I need to also build.
>>>
>>> I will have some other mechanisms too, not only Jinja, for building
>>> these files, but each of the targets will need to be parsed after being
>>> built.
>>>
>>>
>>> On Fri, Feb 21, 2020 at 2:11 PM Bill Deegan <bill at baddogconsulting.com>
>>> wrote:
>>>
>>>> Scott,
>>>>
>>>> I'm not sure based on your example how all the files which need to be
>>>> processed wouldn't be known before you ran SCons.
>>>>
>>>> if top.F.jinja is the following (and is the top file which ends up
>>>> including all the others)
>>>>
>>>> tgt1.v
>>>> -F pkg1.F
>>>> {% if some_condition %}
>>>> -F pkg2.F
>>>> {% endif %}
>>>>
>>>> And "-F" is the include statement. Then all you'd need is the way to
>>>> evaluate the {% if's %} to know which files it includes.
>>>>
>>>> -Bill
>>>>
>>>> On Thu, Feb 20, 2020 at 10:05 PM Scott Johnson <
>>>> scott.johnson at arilinc.com> wrote:
>>>>
>>>>>
>>>>> On Feb 20, 2020, at 6:46 PM, Bill Deegan <bill at baddogconsulting.com>
>>>>> wrote:
>>>>>
>>>>> Yes. set it as src_scanner.
>>>>>
>>>>> You're scanner doesn't return any nodes...
>>>>>
>>>>>
>>>>> It doesn’t yet, as this is just an experimental toy, but see my
>>>>> original post. I can’t determine what else needs to be built until after
>>>>> the first target is built. And unlike this toy example, the real target
>>>>> isn’t just a simple copy of the source file. So scanning the source file
>>>>> won’t work for me.
>>>>>
>>>>> A traditional scanner looks at the source file and finds dependencies
>>>>> that need to be built before this source file is processed. My use case is
>>>>> different. I need to process the source file, then look at the target file
>>>>> and find new (independent) files to build.
>>>>>
>>>>> I found this question on Stack Overflow that suggests that what I’m
>>>>> trying to do is either tricky or impossible, depending on which answer you
>>>>> believe.
>>>>>
>>>>>
>>>>> https://stackoverflow.com/questions/24671859/scons-how-to-generate-dependencies-after-some-targets-have-been-built/26302284
>>>>>
>>>>> _______________________________________________
>>>>> Scons-users mailing list
>>>>> Scons-users at scons.org
>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>
>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>> _______________________________________________
>>> Scons-users mailing list
>>> Scons-users at scons.org
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20200225/e3037638/attachment.html>