[Scons-users] Performance of copying from cache

[a.cavallo at cavallinux.eu]

Mmm. I suppose rsyncing to a local mounted dir would be better solution: is there
any special reason for the source being stored on a nfs mount? Are you using
clearcase? If I rember right it does support local snapshots as well.

I hope this help







On Wed 12/06/13 11:16, "Tom Tanner (BLOOMBERG/ LONDON)" ttanner2 at bloomberg.net wrote:

> So, we have a biggish build, which we tend to run with -j 2 or 4, with some

> large files on it and we've noticed that copying files out of our NFS cache

> can take ridiculous amounts of time (and we're talking about 10+ minutes,

> though AIX seems to be a lot worse than solaris or linux)

> Having a hunt round, I discovered that shutil.copy2 copies 16k at a time,

> which doesn't seem terrifically efficient (it would appear I'm not the only

> person who thinks that). So I took a copy of that, used a 1M buffer, and it

> reduced my worst case copy to 11 seconds.

> But the thing that really improved worst performance was replacing that

> with "cp -p". However, that hosed overall performance.

> Has anyone any suggestions? Would it be saner to just read the while file

> at once (although as some of the files are quite large, that might be

> painful).

_______________________________________________

> Scons-users mailing list

> Scons-use

> rs at scons.org

http://four.pairlist.net/mailman/listinfo/scons-users

> 

>