[Scons-users] Godot generated dependency files access issue

[Alexandar Danilovic]

Hello, folks.

Godot game engine has a Scons build issue and I am hoping kind folks here
can offer some help.

Godot has for a long time had an issue with it's parallel builds on windows
with MSVC...

Basically, generated .h files are immediately after creation required by
the next compile order and cl.exe cannot access it immediately after it is
generated...

...\drivers/gles2/rasterizer_gles2.h(55): fatal error C1083: Cannot open
include file: 'drivers/gles2/shaders/canvas.glsl.h': No such file or
directory
1>  scons: *** [platform\windows\os_windows.windows.tools.64.obj] Error 2

Now, i have tested whether they are in the dependency tree, and they are...
Which is why -j1 builds work fine... only -j2+ fail...

Another important bit of info is that parallel builds work fine with Scons
2.4.1 and 2.4.0 when Pywin32 is installed... when pywin isn't installed,
they fail with this error just like 2.5.1...

But 2.5.1, the latest one... does not compile parallel builds at all...
even though i have Pywin32 installed for it (it's using the same python
install)... it errors on the above message regardless of pywin32.

Now, i managed to get compiles to compile on 2.5.1 with a hack...
basically, i sleep the file builders that build the generated .h files for
a few seconds to allow file handles to be closed in time (i believe it's
the handles)...

https://github.com/Griefchief/godot/commit/4c3b9c2a71b69ec5922bdd47a5bde4acfe09a477#diff-45b856a77c4dd4017d6ebe6299a75cb4R1463


I want to ask if any of you kind folks that have read this far have any
idea if there is a "propper", non hackish fix for this?

I did this fix because i needed to be sure what was going on, so that you
and I understand better what is happening...

I will note again what i have noticed:

1) Generated .h files have access issues, my guess is due to non released
file handles
2) File handles should be released by python either manually via close() or
during garbage calls...
3) File handle release is not happening fast enough... for a really huge
file called certs.h that has several MBs i noticed that it seems to take
longer then for the small files used for shaders etc that are kilobytes in
size... i had to increase the wait for this file to 4 seconds instead of 1
or 2 to ensure it will not error out... Writes should not be asynchronous
so i'm not sure what is happening... maybe it was a fluke that certs.h took
longer to release the handle...

4) Pywin32 fixes 2.4.1 and 2.4.0 versions of SCons... however, for 2.5.1
Pywin32 installed seems to have no effect. I am not the only person that
has issues only with 2.5.1 (when pywin is installed). Other folks have also
reported only 2.5.1 being the problem.

The above points more or less cover it. Even though i attempt to close the
handles manually, they simply seem to not be released in time...

It seems that there is some kind of interaction between the
filesystem/processes or something third that keeps the file handles alive
for too long. It's strange that it is actually sleep() that releases the
handles in time... if Python was leaking handles, I'd expect sleep() to not
have an effect but it does... I have zero experience with file handles,
maybe something else is going on that needs time that i don't know about
that is not Python itself (windows related?)...

I hope i have given you enough information to offer your suggestions if you
have any.

Again, 2.5.1 seems to be a hotspot...

Best regards,
Aleksandar Danilovic.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20160616/91c98eb6/attachment.html>