[Scons-users] Nuitka and Scons, questions, potential bugs, no Windows parallel build

[Kay Hayen]

Hello everybody,



> . You might want to take a peek at the SConstruct for MongoDB (or Mapnik)

>> to get some ideas about how to support different options...

>> If you plan to create several builds (native +  cross-compiling) on one

>> system, VariantDirs are your friend.

>

>

> I am using "VariantDir" for the static source code that typically also

> lives in a place not writable.

>

>

>>

>> Second, for MSVC, it does outputs that I cannot prevent it seems. I would

>>> appreciate warnings, but it does some information that I cannot disable.

>>> Therefore "setupSpawn", where I basically filter the output myself.

>>>

>>>  I can't say anything about it, unless you give an example of how

>> exactly (copy-paste) the output looks like that you're referring to.

>

>

> CompiledCodeHelpers.cpp

> cl /FoAsserts.build\static\win32_ucontext_src\fibers_win32.obj /c

> "nuitka\build\static_src\win32_ucontext_src\fibers_win32.cpp" /TP /nologo

> /EHsc /J /Gd /Ox /MT /D_NUITKA_FULL_COMPAT /D__NUITKA_NO_ASSERT__

> /D_NUITKA_EXE /IC:\Python27_32\include /IAsserts.build

> /Inuitka\build\include

>

> I can't get "cl.exe" to not say the filename and the command line. So I

> throw away the first two lines of stdout. I think it also does a spurious

> empty line at the end.

>


Nuitka is a bit special in that I want to strip such information that
everybody else will consider progress information. I would at least need
have these things optional. I gather nobody cares about MSVC making these
outputs, but rather wants them in any case.



> Have you tried "-j 4" instead of the "--jobs 4" in your SconsInterface.py?

>> Apart from this, there shouldn't be any restrictions...can you point me to

>> the older posts that you mentioned?

>>

>

> I just checked, and indeed, my spawn is called 6 times, but still only one

> "cl.exe" is running at a time. I have code like that:

>

> proc = subprocess.Popen(

>

>     cmdline,

>

>     stdin = subprocess.PIPE,

>

>     stdout = subprocess.PIPE,

>

>     stderr = subprocess.PIPE,

>

>     startupinfo = startupinfo,

>

>     shell = False,

>

>     env = env

>

> )

>

> data, err = proc.communicate()

> rv = proc.wait()

>

> Since I can observe 6 spawns at once, this somehow must end up handling

> only one process at once. I am going to debug a bit around, but it's

> probably about not using the Python APIs properly then.

>



Totally my fault. I think it has to do with "MSVC_BATCH = True" and my
false assumptions plus caching. The bigger case I was working on has one
file that takes really long, and smaller ones that are then finished near
immediate in comparison. With Nuitka starting compilation of C++ only after
2 minutes (processing a lot of Python before), it has a scalability issue
in one generated file, which compiles alone at the end. This looked like
the same thing, but it is not.

So this is solved. If I am right (will check too), the MSVC_BATCH mode
should mention that it cannot build in paralell in the docs. Thinking of
it, batch is that, but yeah, not everybody is thinking. :)

Yours,
Kay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://four.pairlist.net/pipermail/scons-users/attachments/20131115/8d0c2318/attachment.html