[Scons-users] Problems with SCons parallel build

[Brady Johnson]

Sorry guys, I spoke too soon. It worked with "-j 4", but then I got greedy
and tried with "-j 12" and "-j 8" and it failed again, saying source files
dont exist, even though they do indeed exist, and they are not generated.
Seems like the "cd" in a Command() was one problem, because it wouldnt work
before with -j 2 nor -j 4. Now there's something else.

Regards,

Brady



On Fri, Mar 21, 2014 at 11:30 PM, Brady Johnson <bradyallenjohnson at gmail.com

> wrote:



> Wow, I fixed it!! Actually a few emails ago when I reported that removing

> all the calls to cd from the Command() builders failed, it failed because

> of what I commented out, later parts of the build needed these files. I

> changed all the calls to "cd /some/dest/dir ; jar xf /some/file" with

> "unzip -d /some/dest/dir /some/file" (no cd's anywhere) and now it works

> great with "-j 4"

>

> So then either Command() isnt creating a sub process, or the child process

> is somehow affecting the parent. Either way, calling "cd" in a Command()

> causes parallel building to not work.

>

> Thanks for your help Gary, Dirk, and Evan!!

>

> Regards,

>

> Brady

>

>

>

> On Fri, Mar 21, 2014 at 10:56 PM, Evan Driscoll <driscoll at cs.wisc.edu>wrote:

>

>> On 2014-03-21 16:50, Brady Johnson wrote:

>>

>>> I commented out all the cd's from the Command() builders, and its

>>> still failing.

>>>

>>> Something else that I just considered: I build several RPMs by

>>> calling os.system('rpmbuild...') directly instead of calling the

>>> SCons rpm builder. I wonder if rpmbuild changes directories,

>>> subsequently changing it for the SCons process. Anybody know if this

>>> could cause problems?

>>>

>>

>> A child process can't change the directory of the parent (or, maybe

>> there's a way if you try really hard and jump through hoops or something).

>> That shouldn't be the cause.

>>

>> One thought you could try: change env['ENV']['PATH'] so that

>> x86_64-dx-linux-gnu-g++ is a script you write that outputs the current

>> directory before running. Would at least tell you if this is a wild goose

>> chase. :-)

>>

>> Evan

>>

>>

>>

>>> Thanks,

>>>

>>> Brady

>>>

>>> On Fri, Mar 21, 2014 at 10:30 PM, Brady Johnson

>>> <bradyallenjohnson at gmail.com> wrote:

>>>

>>>  The only cd in the entire project is in the aforementioned call to

>>>> Command(), chdir isn't used anywhere, neither via os nor via

>>>> builders.

>>>>

>>>> Regards,

>>>>

>>>> Brady

>>>>

>>>> On Mar 21, 2014 10:26 PM, "Evan Driscoll" <driscoll at cs.wisc.edu>

>>>> wrote:

>>>> On 2014-03-21 16:04, Gary Oberbrunner wrote:

>>>> "cd" in a command is OK, since that runs in a subprocess.  But a

>>>> python chdir() would be very bad, since the whole process has only

>>>> one

>>>> current dir, so you're changing it for all threads if you do that.

>>>>

>>>> Don't forget chdir arguments to builders too.

>>>>

>>>> In other words, when using a parallel build:

>>>>

>>>> 1) Command("cd blah && whatever") is OK

>>>> 2) AnyBuilder(..., chdir=1) or AnyBuilder(..., chdir='blah') is NOT

>>>> 3) os.chdir() from a Python function called by a builder is NOT

>>>>

>>>> Evan

>>>>

>>>> _______________________________________________

>>>> Scons-users mailing list

>>>> Scons-users at scons.org

>>>> http://four.pairlist.net/mailman/listinfo/scons-users [1]

>>>>

>>>

>>>

>>>

>>> Links:

>>> ------

>>> [1] http://four.pairlist.net/mailman/listinfo/scons-users

>>>

>>>

>>> _______________________________________________

>>> Scons-users mailing list

>>> Scons-users at scons.org

>>> http://four.pairlist.net/mailman/listinfo/scons-users

>>>

>> _______________________________________________

>> Scons-users mailing list

>> Scons-users at scons.org

>> http://four.pairlist.net/mailman/listinfo/scons-users

>>

>

>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://four.pairlist.net/pipermail/scons-users/attachments/20140321/77b7c019/attachment.htm