[Scons-users] Hi, wondering about possible speed ups to scons

[Bill Deegan]

To add to what Dirk said, you also need to test such on a non-local
filesystem (NFS...)

-Bil

On Thu, Mar 5, 2015 at 3:27 AM, Dirk Bächle <tshortik at gmx.de> wrote:

> Hi Ben,
>
> On 05.03.2015 00:52, Schleimer, Ben via Scons-users wrote:
>
>> Hi Y'all,
>>     First of all, I'd like to say that I use SCons all of the time for my
>> C++ projects and it's by far the best build system I've
>> had the pleasure to work with. I especially like the ease of using the
>> Qt4 tools and precompiled header support, yada yada yada...
>>
>>
> that's really nice to hear. Thanks a lot for the positive feedback!
>
>  Secondly, I've been thinking about all of the criticism aimed at scons
>> for not being fast enough.
>> I understand that python is inheritantly slow (ignoring cython, etc..)
>> but it seems like SCons could easily design it's way out of
>> this problem.
>>
>>
> Did you have a look at http://www.scons.org/wiki/WhySconsIsNotSlow yet?
> Especially the numbers at the bottom of the page will be what you have to
> beat with your solution. There also exists a pre-edit version of the talk I
> gave at the PyconFr 2014 in Lyon (http://www.pycon.fr/2014/
> schedule/presentation/25/), the download URL is:
>
>   http://dl.afpy.org/pycon-fr-14/videos/25-10-2014-112947_bd16.mp4
>
>
>  I'm wondering if the design change for SCons I'm about to propose has
>> been proposed before:
>>
>>
> Yes, it has...lots of times. ;) These kind of approaches always seem to
> work well, assuming that SCons has a DAG that is statically created in the
> background, and then simply traversed during the build step. But that's not
> true, SCons detects the dependencies on-the-fly while building targets.
> There is no explicit structure that you can point to and refer to as "the
> DAG". It's somehow implicitly given since each Node stores its children,
> and this is the info that gets updated continuously during the single build
> steps.
> So, having a data structure as "the DAG" and update its nodes, will not
> directly map to what SCons uses internally to figure out which targets are
> up-to-date or need to get rebuilt.
>
>  1) Create a scons daemon (sconsd) that stays resident and watches a
>> particular SConstruct file and all of its generated
>> dependencies. Whenever any of the dependencies change or is deleted,
>> scons updates the dependency graph in the background.
>>
>> 2) When the user actually runs scons to compile, the dependency graph is
>> ready to be immediately traversed and all of the Nodes built.
>>
>> Thoughts? Ideas?
>>
>>
> What about the Nodes that get generated during the build itself? What if I
> want to compile/build only a part of the complete DAG (a single
> subdirectory) and then later on possibly build the whole thing? How to
> ensure that things like build infos for the single Nodes/Targets get
> properly merged/updated between those two separate runs?
>
> And how will such a daemon behave performance-wise on a project with
> 50000+ C files? My guess is that there would be a lot of stat()ing going
> on...and wrapped in a package like watchdog, one would have to make a
> simple testcase first and try to profile some decent numbers out of it.
> Then watch the scaling behaviour...and if that looks okay, we can talk
> about how to integrate such a thing into SCons. ;)
>
> Just a few first thoughts...
>
> Best regards,
>
> Dirk
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20150305/a56ddaee/attachment.html>