[Scons-users] Help my build get faster... (was SCons and parts)

[Jason Kenny]

In parts I tend to make lots of Clones, However this is limited to having an environment per component, as normally these need an environment that is not shared because of various flags, defines, libs that cannot be shared. Not fulling understanding what you are doing... a Clone for each object ( as in a object file) seem excessive. I think the worse I had seen it with work I have done was a product with about 1200 "Parts"  which meant I had over 2000 clones myself, loading everything first pass took 60+ seconds ( mostly because of large disk scan). Not cloning, but creating new environments meant the time to load everything went to 10+ minutes. Ideally having a lighter Clone would be ideal, and can be done. I have not gone the extra mile for this in Parts as of yet. Ideally building each DLL/SO/EXE/program will want to have a Cloned Environment, If you have a test setup this might want to have a clone as well. only in very advance cases do you need something per object file, and most of the time these can be done correctly in a shared environment. The value you get out of part if you use it here is that on the second pass I can load less stuff based on what is out of date, as I know what builds what. This mean loading less and a smaller node tree in SCons. However if you change a file in the leaf component everything else needs, at the moment I am forced to load everything that depends on that leaf.

Ideally one build with 2000+ Programs/SO is really large

Jason

-----Original Message-----
From: Scons-users [mailto:scons-users-bounces at scons.org] On Behalf Of Pico Geyer
Sent: Tuesday, August 15, 2017 11:09 AM
To: SCons users mailing list <scons-users at scons.org>
Subject: Re: [Scons-users] Help my build get faster... (was SCons and parts)

It's actually our own builder that we are using in this case, each object could have different include paths or options.
But just like if you had a list of C files to build each with their own include paths, wouldn't you need to create a new environment for each?

On Tue, Aug 15, 2017 at 5:51 PM, Bill Deegan <bill at baddogconsulting.com> wrote:
> Why do you need a new Environment for each object?
> Can you paste some code to explain?
>
>
> On Tue, Aug 15, 2017 at 11:34 AM, Pico Geyer <picogeyer at gmail.com> wrote:
>>
>> Bill,
>> Sorry for the delay.
>>
>> Right of course. I was just hoping for something in scons due to the 
>> number of SConscript files that I have (and laziness :) ) So indeed 
>> it was the file I suspected.
>> I seemed to have further narrowed it down to env.Clone(), since I 
>> need a new environment for each object. And their are roughly 2000 of them.
>> This seems to take about 4 seconds.
>> I found this with the combination of print messages (like your
>> example) and the very useful  line_profiler tool
>> (https://github.com/rkern/line_profiler)
>> I could send you the output off list if you're interested.
>>
>> Any suggestion on what I can do to improve things?
>> If there isn't a way to make it faster, I was hoping for a way to 
>> detect early on that I'm not building that component anyway, so I 
>> don't need to process those objects (and therefore do so many 
>> environment clones).
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
_______________________________________________
Scons-users mailing list
Scons-users at scons.org
https://pairlist4.pair.net/mailman/listinfo/scons-users