[Scons-users] build time

[TOM TANNER (BLOOMBERG/ LONDON)]

And just in case it was directed to me,

I too don't have many builders with involved python logic. I do have a couple of scanners, though I don't know at which stages in the process they get invoked (and I've not worked out how to sanely cache the results, as that part of the scanner library appears to be somewhere in the C scanner and is not exposed in a well documented interface)

----- Original Message -----
From: scons-users at scons.org
To: scons-users at scons.org
At:  7/09 08:28:13

Bill,

Sorry for not answering, I thought the question was directed to Tom Tanner.
Personally, I dont have many builders with involved Python logic. Looking
at it from the point of view you just mentioned, it makes sense that the
GIL shouldnt have that much impact, but I guess it would be better to test
it.

Regards,

Brady

On Mon, Jul 9, 2012 at 12:21 AM, William Deegan
<bill at baddogconsulting.com>wrote:


> Russel,

>

> On Jul 7, 2012, at 4:30 AM, Russel Winder wrote:

>

> > On Sat, 2012-07-07 at 12:07 +0200, Brady Johnson wrote:

> >> What would be cool would be for SCons to be able to use the Python

> >> multiprocessing module instead of threads. The API is very similar by

> >> design, so it shouldnt be too difficult to change. Then all available

> >> CPUs/cores could be used without having to deal with the GIL. And maybe

> >> this would be too much, but if it were configurable, that would just be

> >> awesome.

> >>

> >> Thoughts, anyone?

> >

> > This would require setting the SCons Python floor to 2.6 – well it could

> > be done with 2.5 as there is a backport of multiprocessing to 2.5, but I

> > think 2.6 would be a better option.  The problem is that there are a lot

> > of 2.4 users out there who are unwilling and/or unable to upgrade.

> >

> > There are many options.

> >

> > 1. Up the floor anyway and only allow SCons updates to be used by people

> > runing 2.6 or higher.

> >

> > 2. Do nothing and continue to use 2.4 as the floor.

> >

> > 3.  Use the PyPy with STM instead of a GIL.

> >

> > 4. Ensure all actions get spawned via ctypes of some such so they are

> > not running Python bytecodes and thus not constrained by GIL.

> >

> > 5. Rewrite SCons in Python 3 an make some real progress whilst really

> > annoying people stuck on Python 2.4 :-)

>

> As I understand it, the GIL only becomes a problem for SCons if you have

> builders which are python logic. If you change such that they are scripts

> run from the command line, then the GIL no longer comes into play.

>

> Also important to keep in mind that depending on the specifics of a build

> there may be only so much parallelism which would be possible.

> (Think 100 sources files each build 2 libraries which then build a

> program, which then is run to generate source files...)

>

> Thus my question (unanswered) to Brady if he had any python builder

> logic...

>

> -Bill

> _______________________________________________

> Scons-users mailing list

> Scons-users at scons.org

> http://four.pairlist.net/mailman/listinfo/scons-users

>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://four.pairlist.net/pipermail/scons-users/attachments/20120709/1e741732/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 7517695304669002.txt
Type: application/octet-stream
Size: 149 bytes
Desc: not available
Url : <http://four.pairlist.net/pipermail/scons-users/attachments/20120709/1e741732/attachment-0001.obj>