[Scons-users] SCons bug when combining CacheDir and MSVC_BATCH

Thu May 25 02:44:01 EDT 2023

That certainly changes the behaviour and the build now passes:

$ scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
Retrieved `build1\main.obj' from cache
cl /Fobuild1\\ /c src\main.c src\foo.c /nologo
main.c
foo.c
Generating Code...
link /nologo /OUT:build1\a.exe build1\main.obj build1\foo.obj
cl /Fobuild2\foo.obj /c src\foo.c /nologo
foo.c
link /nologo /OUT:build2\a.exe build2\main.obj build2\foo.obj
scons: done building targets.

However, you can see that build1\main.obj is retrieved from the cache but is then also rebuilt, which is missing the point of the cache…

S.

From: Scons-users <scons-users-bounces at scons.org> On Behalf Of Eric Fahlgren
Sent: Wednesday, May 24, 2023 6:11 PM
To: SCons users mailing list <scons-users at scons.org>
Subject: Re: [Scons-users] SCons bug when combining CacheDir and MSVC_BATCH

Edit site-packages/SCons/Tool/msvc.py and change line 203 (in scons 4.5.2) to the following and see if that helps...

    if msvc_batch_key(None, env, target, source) is None:  # Original fails when base names are different.

On Wed, May 24, 2023 at 9:03 AM Bill Deegan <bill at baddogconsulting.com<mailto:bill at baddogconsulting.com>> wrote:
Can you try without MSVC_BATCH enabled?
-Bill

On Wed, May 24, 2023 at 8:50 AM Steve Hill (Wireless) via Scons-users <scons-users at scons.org<mailto:scons-users at scons.org>> wrote:

Thanks for the reply, Mats.

1) Moving the setting of MSVC_BATCH to the constructor doesn't make any difference - I get the same link failure.

2) Adding debug, I get:

$ scons --cache-debug=-

scons: Reading SConscript files ...

scons: done reading SConscript files.

scons: Building targets ...

Retrieved `build1\main.obj' from cache

CacheRetrieve(build1\main.obj):  retrieving from 97427c14b8c8b53e9e7af4d7d8afd524

requests: 1, hits: 1, misses: 0, hit rate: 100.00%

CacheRetrieve(build1\foo.obj):  2c4b62611510a27efcd0dd7066b26b04 not in cache

requests: 2, hits: 1, misses: 1, hit rate: 50.00%

cl /Fobuild1\\ /c src\main.c src\foo.c /nologo

main.c

foo.c

Generating Code...

CachePush(build1\main.obj):  97427c14b8c8b53e9e7af4d7d8afd524 already exists in cache

requests: 2, hits: 1, misses: 1, hit rate: 50.00%

CachePush(build1\foo.obj):  pushing to 2c4b62611510a27efcd0dd7066b26b04

requests: 2, hits: 1, misses: 1, hit rate: 50.00%

CacheRetrieve(build1\a.exe):  bcd159a2ee6e98e239022957d9dfe010 not in cache

requests: 3, hits: 1, misses: 2, hit rate: 33.33%

link /nologo /OUT:build1\a.exe build1\main.obj build1\foo.obj

CachePush(build1\a.exe):  pushing to bcd159a2ee6e98e239022957d9dfe010

requests: 3, hits: 1, misses: 2, hit rate: 33.33%

Retrieved `build2\main.obj' from cache

CacheRetrieve(build2\main.obj):  retrieving from 5619d81a934ee71939605662178f5d54

requests: 1, hits: 1, misses: 0, hit rate: 100.00%

CacheRetrieve(build2\foo.obj):  6d30ba71fee8872f9c2c8541225ead92 not in cache

requests: 2, hits: 1, misses: 1, hit rate: 50.00%

cl /Fobuild2\\ /c src\foo.c /nologo

foo.c

CachePush(build2\foo.obj):  pushing to 6d30ba71fee8872f9c2c8541225ead92

requests: 2, hits: 1, misses: 1, hit rate: 50.00%

CacheRetrieve(build2\a.exe):  e4533b0dc80dcb3d3b1cd2cdd081933c not in cache

requests: 3, hits: 1, misses: 2, hit rate: 33.33%

link /nologo /OUT:build2\a.exe build2\main.obj build2\foo.obj

LINK : fatal error LNK1181: cannot open input file 'build2\main.obj'

scons: *** [build2\a.exe] Error 1181

scons: building terminated because of errors.

The odd thing is that it is retrieving build2/main.obj from the cache when it is already up-to-date but that still doesn’t explain why it gets deleted…

S.

-----Original Message-----
From: Scons-users <scons-users-bounces at scons.org<mailto:scons-users-bounces at scons.org>> On Behalf Of Mats Wichmann
Sent: Wednesday, May 24, 2023 3:49 PM
To: scons-users at scons.org<mailto:scons-users at scons.org>
Subject: Re: [Scons-users] SCons bug when combining CacheDir and MSVC_BATCH

On 5/24/23 06:24, Steve Hill (Wireless) via Scons-users wrote:

> All,

>

> We have recently switched on CacheDir in our CI machine and, while

> this has largely been a positive experience, we have seen the odd link

> failure that we could not explain. I was eventually able to reproduce

> the issue and produce a minimal SConstruct that illustrates the problem.

Something's broken here.  I've tried to trace this a little bit and it looks like things can get removed after they're retrieved.  There are other oddities too, in your rebuild log:

> Retrieved `build1\main.obj' from cache cl /Fobuild1\\ /c src\main.c

> src\foo.c /nologo main.c foo.c Generating Code...

If there are two sources, but one object is up to date, SCons should not calculate a batched build is needed, in other words it should have emitted:

cl /Fobuild1\foo.o /c src\foo.c /nologo

> Would you like me to raise an issue on GitHub for this?

I'd say yes, but not being wise in the ways of MSVC_BATCH maybe there's something I'm missing.

Can you try a couple of tweaks to your reproducer (thanks for taking the time to come up with one!) while you're at things?

1) put the MSVC_BATCH initialization in the Environment call.  The msvc tool is notoriously sensitive to getting its initialization wrong if it doesn't have all its details in hand when it runs.  No idea if that moves the goalposts but worth trying.

2) turn on some debugging - add  --cache-debug=-   (the dash argument

sends the debug to the console)

MSVC_BATCH has several issues already in the tracker about interaction with things, didn't look to me like any of them were likely this issue, though.
_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://urldefense.com/v3/__https:/pairlist4.pair.net/mailman/listinfo/scons-users__;!!Niha4SQ!7Q3eIYcm3nNqidhG9pyPlFM3mt8ZA5ju9c8QoHpGkZi8BL3SzDaYnIiswcGyrAfZQKhBMu-1nYyyRJ_o6WKwv7NtI5mxmXA$>
_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://urldefense.com/v3/__https:/pairlist4.pair.net/mailman/listinfo/scons-users__;!!Niha4SQ!7Q3eIYcm3nNqidhG9pyPlFM3mt8ZA5ju9c8QoHpGkZi8BL3SzDaYnIiswcGyrAfZQKhBMu-1nYyyRJ_o6WKwv7NtI5mxmXA$>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20230525/625fb0d6/attachment-0001.htm>