[Scons-users] SCons working with Ninja

[Bill Deegan]

The work in progress is generously contributed and upstreaming sponsored
by MongoDB, so the reference to someone's packaging of such would probably
be an earlier version of that code.

As I understand it many of the build steps as possible are exported to the
ninja build file, the remainder will call back into SCons to build.

Don - I'd be curious if you could share what post link steps have to be
implemented in python? Are they generating command lines? or doing file
munging in python?

The plan is to add it to the next (4.1.1 or 4.2.0.. I need to review the
included changes since 4.1.0), it will NOT be enabled by default.
As the code has only been used in only one production environment, I expect
there'll be a lot of bumps in the road before it's ready for general usage.

It's possible the implementation may be significantly changed before it's
enabled by default.

If you're willing to be a PATIENT "beta" user and even contribute fixes.. ,
then we'd be glad to have the help.
If not, then you're better waiting until it's had more usage to try it out.

Further, if you already have an implementation (based on the mongodb work,
or not) and can share any lessoned and code learned, that'd be great.

We don't really have the resources to fully support this at the moment,
thus the "experimental feature" approach was chosen as a way to expose it
to motivated parties who might help us move it forward to production
quality.

-Bill
SCons Project Co-Manager.


On Wed, Apr 14, 2021 at 1:56 PM Don Baldwin <donb at qti.qualcomm.com> wrote:

> Thanks Gary.  However, in our case, we have several post-link steps that
> are carried out as Python functions, so they are an unavoidable part of the
> build process.
>
> -Don
>
> -----Original Message-----
> From: Scons-users <scons-users-bounces at scons.org> On Behalf Of Gary
> Granger
> Sent: Wednesday, April 14, 2021 1:22 PM
> To: SCons users mailing list <scons-users at scons.org>
> Subject: [EXT] Re: [Scons-users] SCons working with Ninja
>
> Hello Don,
>
> For whatever it's worth, we have a tool to generate ninja files from
> SCons, but I did not try to solve the problem of running python functions
> from ninja.  Instead, the tool generates a ninja build file from the nodes,
> and then it lets scons build just the nodes which do not use command line
> actions.  Most of our python function builders generate source files and
> version header files, so they are the leaves of the dependency graph and it
> doesn't matter if ninja cannot build them. Building the few python actions
> establishes a 'build baseline', then ninja builds the rest.  Obviously not
> complete, since someone has to remember to rebuild the ninja file if the
> python dependencies change. Although I suppose the tool could be modified
> to list all the python targets with a rule to re-run scons to build just
> those.  And maybe this does not help you, if most of your actions are not
> command lines.
>
> That said, we don't use this much because in practice it's easier to just
> stick with scons for everything.
>
> https://github.com/NCAR/eol_scons/blob/master/eol_scons/tools/ninja.py
>
> The ninja support mentioned by Mats is surprising news, I'll be interested
> to see how that works.
>
> Gary
>
> On 4/14/21 1:42 PM, Don Baldwin wrote:
> > Thanks Mats.  Will this be implemented in SCons 3.x as well as 4.x?  We
> are planning on moving to 4.x fairly soon, but we aren't there yet, and
> will probably not move our older projects to 4.x.  Also, do you have an ETA
> of when this will be available?
> >
> > Also, what about existing tools that claim to support generating Ninja
> scripts from SCons builds, such as:
> >
> >     * https://github.com/RedBeard0531/mongo_module_ninja
> >
> > Does anyone in this group know if that supports function actions?
> >
> > Lastly, we've been able to implement our own home-grown tools for doing
> a build "play-back" similar to what Ninja does, but we've only gotten it to
> work for command line actions.  I think the main hurdle for function
> actions is reproducing the SCons environment.  We've tried pickling it
> using both the native pickle library and 'dill', but have encountered
> infinite loop errors and errors trying to pickle class objects.  Has anyone
> else attempted this and gotten past these errors?  This is using SCons
> 3.1.2.
> >
> > Thanks,
> > Don
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20210414/ca462d33/attachment-0001.htm>