[Scons-users] Memory problem with Integration tests

[Rob Deckers]

Hi,

Thanks for the first comments.

It's nice to hear that the overall size and the basic setup doesn't sound alarming to you.

My machine (Windows 7) has 14GB of RAM available.
Python (being 32bits) seems to run out while coming close to the 2GB boundary.
(watched it grow and grow during the run).
It especially seemed to grow between after finishing one of the integration-test-targets and before starting a second.

What I had noticed already was that this in between time was VERY long (single to several minutes).
Because the tests cannot be run in parallel, and I was trying to optimize, I tried:

scons scripttests --max-drift=1 --implicit-cache -j 1
and
env.Decider('MD5-timestamp')

The "-j 1" really made a HUGE improvement. With this option the memory is around 400-500MB (what you had guessed)
Also the very long waits between the integration-test-targets are almost completely gone.

It seems to work, but I don't know why and I would not like to be obligated to use the "-j 1"

Kind regards,
Rob Deckers


-----Original Message-----
From: Scons-users [mailto:scons-users-bounces at scons.org] On Behalf Of Dirk Bächle
Sent: maandag 11 augustus 2014 17:35
To: scons-users at scons.org
Subject: Re: [Scons-users] Memory problem with Integration tests

Hi Rob,

it's difficult to guess what's wrong without seeing the SConstructs/SConscripts. Please find a few first comments below.

On 11.08.2014 09:21, Rob Deckers wrote:
> Hi All,
>
> I have SCons working including integration tests on one archive and now I'm integrating the integration tests on a second (slightly different) archive.
>
> The previous archive had about 800 scripttests and this one has 1200.
> The 1200 tests are devided over about 100 directories and each directory is it's own target.
>
> Each scripttest uses the same single executable.
> The executable is dependend on about 8.000 header and cpp files
Given this number of input files, a memory consumption of 400-500MB (as your test results indicate), looks quite normal to me. This depends on the project's structure of course, but it's not an alarming value.

> To make a all_scripttests target, each individual scripttest target is added like:
> env.Alias(FscScriptTestTool['scripttest_alias'], target)
>
> When I first do a scons run to build the executable and then an all_scripttests run.
> I see the memory growing with each scripttest (a rate of about 1MB every 4 seconds).
>
> Eventually, I get a memory error triggered from some copy command.
> When I disabled that single line, I got the error on some other line.
>
> Is it possible my alias construction is to blame?
I don't think so.
> What does scons (beneath the surface) do with 100 targets all dependent on one executable with 8K files ?
> Does it create 100 times the entire tree for the executable?
No, it shouldn't do this. The dependencies are internally stored in a graph, which allows several targets to have the same source node(s) as input.

> In the attachment is the output from an up-to-date run for the executable. I used:
> scons executable --debug=memory --debug=count I have a hard time 
> really interpreting those results.
The first thing that catches my eye is the high number of "Override" 
Environments. This means you're using things like

   env.Program(target, source, SET_A_VARIABLE="something_special")

a lot. Try to avoid that, and setup the Environments in your top-level SConstruct for the different cases you need. Then simply pass them around, but don't recreate them in all the places.
If you think you *have* to do it like this, let's discuss the underlying problem and how to solve it. There should be a way around it...

How much memory does your system have?
Have you tried to watch memory consumption with a "top/vmstat" in parallel to running SCons? The "scons --debug=memory" isn't necessarily comprehensive.
By how much does the overall memory consumption of the "scons" process grow when you run it with one test target, versus running with two targets?

Best regards,

Dirk


_______________________________________________
Scons-users mailing list
Scons-users at scons.org
http://four.pairlist.net/mailman/listinfo/scons-users
------------------------------------------------------------------------------

** Disclaimer **

This e-mail, including any attachments, may include proprietary and confidential information of Vanderlande and may only be read by the person or those persons to whom it is addressed.
This document is forwarded to you in such a form (e-mail) that Vanderlande cannot guarantee the completeness and/or correctness of its contents and information.
If you have received this e-mail message in error, please notify us immediately. Please also delete this document from your computer.
This document may not be reproduced, copied, distributed, published, modified, or furnished to third parties, without the prior written consent of Vanderlande.