[Scons-users] SCONS -j12, console output is disordered

[Duane Ellis]

SCONS could –
                Capture the output of each sub process in the standard way.
                Ie: use “POPEN()” and COMMUNICATE with the sub process.
                   Thus the subprocess output goes to SCONS and SCONS collects it in a buffer.
                    You might want to put a timeout on all sub processes, ie: 5 minutes…
                    In case they output an error and want sometype of input from the human to continue

                Then once the subprocess has completed, (Exit success or Exit Error)
               The responsible runner thread would:
                      Lock the console (mutex) so other SCONS threads cannot use it.
                      output stdout and stderr of the sub process
                      Unlock the console so other SCONS threads can use it.

From: Scons-users <scons-users-bounces at scons.org> on behalf of Mats Wichmann <mats at wichmann.us>
Date: Saturday, December 23, 2023 at 8:51 AM
To: scons-users at scons.org <scons-users at scons.org>
Subject: Re: [Scons-users] SCONS -j12, console output is disordered
On 12/23/23 08:39, Yu,Zhiwei (YFI,Shanghai,CN) via Scons-users wrote:
> Hi Guys，
>
>       I add -j12 param, then the console output is disordered.
>       I wonder that the reason is python's print function is not
> thread-safe or atomic operation, right?
>       How could I fix the problem when I use SCONS.

There are two sources of output:

(1) scons-generated (by default, the command line, modifiable by setting
the various COMSTR variables, or setting up a PRINT_CMD_LINE_FUNC).
This text should not have any particular interleave problems.
(2) command-issued output.  This you can't do much about, because SCons
isn't really designed for that.  As jobs are classified as ready to run,
they're put on the queue; meanwhile when a job runner slot opens up,
it's assigned a job from the queue. Those are run as subprocesses, and
run independently; SCons doesn't collect their output or any other
information besides the exit status.  So you're just seeing the effect
of having 12 job runner slots all firing off jobs as they become
available - if those generate their own output that can well interleave.

To not have interleaved command output, you'd need to convince SCons to
collect the output of the subprocesses rather than going to the default
stdout/stderr.  That is possible, but beyond the scope of a first answer.

_______________________________________________
Scons-users mailing list
Scons-users at scons.org
https://pairlist4.pair.net/mailman/listinfo/scons-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20231223/4c7188b9/attachment.htm>