[Scons-users] Nuitka and Scons, questions, potential bugs, no Windows parallel build

[Kay Hayen]

Hello Dirk,

 First of all, for Windows, I am attempting to support MinGW and MSVC both,

>> which seems to not be allowed in one environment. Therefore I am creating

>> an environment with default tools, and then with mingw to see if they work.

>> Is that the best way to do it.

>>

> each Environment in SCons has only one variable named "CXX", so you can't

> have two different C++ compilers in a single Environment. This is actually

> a good thing because it would otherwise confuse SCons (and the user) about

> which compiler gets selected for the actual build.

> If you want to let the user select which compiler should be used, you

> could add an option (AddOption) like "--compiler" that can get set to

> "msvc|mingw". In your SConstruct you read the value of this option and then

> initialize the build environment with the requested Tool. Note, that you

> don't have to let SCons search automatically for the Tools that exist on

> the current system. You can explicitly tell it which support should get

> included:

>

>   env = Environment(tools=['as','link','ar','mingw'])

>


Thanks for the pointer. I looked at the MongoDB one, which has quite a bit
of interesting stuff. Can't properly paste here, but I create an
environment with "default" and if that doesn't work (env[CXX] does not
exist), I create one with "mingw"

My end user is not supposed to interact at all with Scons, so these flags
are outside, and reach it as "clang_mode", "mingw_mode".

Seems my hope that something exists in Scons core that allows to provide a
list of wanted stuff in an order, and Scons finds what actually exists, is
not supported. I of course want to have working defaults.




> . You might want to take a peek at the SConstruct for MongoDB (or Mapnik)

> to get some ideas about how to support different options...

> If you plan to create several builds (native +  cross-compiling) on one

> system, VariantDirs are your friend.



I am using "VariantDir" for the static source code that typically also
lives in a place not writable.



>

> Second, for MSVC, it does outputs that I cannot prevent it seems. I would

>> appreciate warnings, but it does some information that I cannot disable.

>> Therefore "setupSpawn", where I basically filter the output myself.

>>

>>  I can't say anything about it, unless you give an example of how exactly

> (copy-paste) the output looks like that you're referring to.



CompiledCodeHelpers.cpp
cl /FoAsserts.build\static\win32_ucontext_src\fibers_win32.obj /c
"nuitka\build\static_src\win32_ucontext_src\fibers_win32.cpp" /TP /nologo
/EHsc /J /Gd /Ox /MT /D_NUITKA_FULL_COMPAT /D__NUITKA_NO_ASSERT__
/D_NUITKA_EXE /IC:\Python27_32\include /IAsserts.build
/Inuitka\build\include

I can't get "cl.exe" to not say the filename and the command line. So I
throw away the first two lines of stdout. I think it also does a spurious
empty line at the end.

Or "link.exe" when asked to create the ".pyd", it will say this:

   Creating library Asserts.lib and object Asserts.exp

Can't stop it from doing saying that.



>  Also there seemed to be problems with arguments that end in a backslash,

>> which paths frequently do, and in one case at least, MSVC requires it, or

>> else it won't use it as a directory.

>>

>>  Again, a concrete example would help to understand your problem much

> better.



Now that I think of it this may be more about Nuitka's spawn code than
anything else.

And the issue that makes me post here, is that I noticed that compilation

>> on Windows is not using the available cores. At first I was considering it

>> being related to batch mode or not, but it seems the same for MinGW and

>> with and without batch mode for MSVC. Searching I found older posts, about

>> the same problem.

>>

>>

> Have you tried "-j 4" instead of the "--jobs 4" in your SconsInterface.py?

> Apart from this, there shouldn't be any restrictions...can you point me to

> the older posts that you mentioned?

>


I just checked, and indeed, my spawn is called 6 times, but still only one
"cl.exe" is running at a time. I have code like that:

proc = subprocess.Popen(

    cmdline,

    stdin = subprocess.PIPE,

    stdout = subprocess.PIPE,

    stderr = subprocess.PIPE,

    startupinfo = startupinfo,

    shell = False,

    env = env

)

data, err = proc.communicate()
rv = proc.wait()

Since I can observe 6 spawns at once, this somehow must end up handling
only one process at once. I am going to debug a bit around, but it's
probably about not using the Python APIs properly then.

Yours,
Kay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://four.pairlist.net/pipermail/scons-users/attachments/20131111/4c9a8ea2/attachment.htm