[Scons-users] SCons bug when combining CacheDir and MSVC_BATCH

Wed May 24 13:15:27 EDT 2023

That is odd.
Please file a GH issue and include your reproducer?

It's possible that no one's tested MSVC_BATCH + CacheDir()..

-Bill

On Wed, May 24, 2023 at 9:56 AM Steve Hill (Wireless) <
Steve.Hill1 at viavisolutions.com> wrote:

> Hi Bill,
>
>
>
> Disabling MSVC_BATCH for both Environments is successful:
>
>
>
> $ scons --cache-debug=-
>
> scons: Reading SConscript files ...
>
> scons: done reading SConscript files.
>
> scons: Building targets ...
>
> Retrieved `build1\main.obj' from cache
>
> CacheRetrieve(build1\main.obj):  retrieving from
> 271f0c07bd36befae43c546754892cb4
>
> requests: 1, hits: 1, misses: 0, hit rate: 100.00%
>
> CacheRetrieve(build1\foo.obj):  065661fe50a5dabcbe97c12c3851870c not in
> cache
>
> requests: 2, hits: 1, misses: 1, hit rate: 50.00%
>
> cl /Fobuild1\foo.obj /c src\foo.c /nologo
>
> foo.c
>
> CachePush(build1\foo.obj):  pushing to 065661fe50a5dabcbe97c12c3851870c
>
> requests: 2, hits: 1, misses: 1, hit rate: 50.00%
>
> CacheRetrieve(build1\a.exe):  53fb57b1111735e2542ad175c63f4401 not in cache
>
> requests: 3, hits: 1, misses: 2, hit rate: 33.33%
>
> link /nologo /OUT:build1\a.exe build1\main.obj build1\foo.obj
>
> CachePush(build1\a.exe):  pushing to 53fb57b1111735e2542ad175c63f4401
>
> requests: 3, hits: 1, misses: 2, hit rate: 33.33%
>
> CacheRetrieve(build2\foo.obj):  a0dbce679b116a4af9f659f102aa6f37 not in
> cache
>
> requests: 1, hits: 0, misses: 1, hit rate: 0.00%
>
> cl /Fobuild2\foo.obj /c src\foo.c /nologo
>
> foo.c
>
> CachePush(build2\foo.obj):  pushing to a0dbce679b116a4af9f659f102aa6f37
>
> requests: 1, hits: 0, misses: 1, hit rate: 0.00%
>
> CacheRetrieve(build2\a.exe):  c12a2901cc6b9bdd3896e188e4e97fe5 not in cache
>
> requests: 2, hits: 0, misses: 2, hit rate: 0.00%
>
> link /nologo /OUT:build2\a.exe build2\main.obj build2\foo.obj
>
> CachePush(build2\a.exe):  pushing to c12a2901cc6b9bdd3896e188e4e97fe5
>
> requests: 2, hits: 0, misses: 2, hit rate: 0.00%
>
> scons: done building targets.
>
>
>
> Possibly more interesting is that I still get the link failure if env1
> does **not** have MSVC_BATCH but env2 does. With the opposite, the build
> is successful.
>
>
>
> Cheers,
>
>
>
> S.
>
>
>
> *From:* Bill Deegan
> *Sent:* Wednesday, May 24, 2023 5:03 PM
> *To:* SCons users mailing list <scons-users at scons.org>
> *Subject:* Re: [Scons-users] SCons bug when combining CacheDir and
> MSVC_BATCH
>
>
>
> Can you try without MSVC_BATCH enabled?
>
> -Bill
>
>
>
> On Wed, May 24, 2023 at 8:50 AM Steve Hill (Wireless) via Scons-users <
> scons-users at scons.org> wrote:
>
> Thanks for the reply, Mats.
>
>
>
> 1) Moving the setting of MSVC_BATCH to the constructor doesn't make any
> difference - I get the same link failure.
>
> 2) Adding debug, I get:
>
>
>
> $ scons --cache-debug=-
>
> scons: Reading SConscript files ...
>
> scons: done reading SConscript files.
>
> scons: Building targets ...
>
> Retrieved `build1\main.obj' from cache
>
> CacheRetrieve(build1\main.obj):  retrieving from
> 97427c14b8c8b53e9e7af4d7d8afd524
>
> requests: 1, hits: 1, misses: 0, hit rate: 100.00%
>
> CacheRetrieve(build1\foo.obj):  2c4b62611510a27efcd0dd7066b26b04 not in
> cache
>
> requests: 2, hits: 1, misses: 1, hit rate: 50.00%
>
> cl /Fobuild1\\ /c src\main.c src\foo.c /nologo
>
> main.c
>
> foo.c
>
> Generating Code...
>
> CachePush(build1\main.obj):  97427c14b8c8b53e9e7af4d7d8afd524 already
> exists in cache
>
> requests: 2, hits: 1, misses: 1, hit rate: 50.00%
>
> CachePush(build1\foo.obj):  pushing to 2c4b62611510a27efcd0dd7066b26b04
>
> requests: 2, hits: 1, misses: 1, hit rate: 50.00%
>
> CacheRetrieve(build1\a.exe):  bcd159a2ee6e98e239022957d9dfe010 not in cache
>
> requests: 3, hits: 1, misses: 2, hit rate: 33.33%
>
> link /nologo /OUT:build1\a.exe build1\main.obj build1\foo.obj
>
> CachePush(build1\a.exe):  pushing to bcd159a2ee6e98e239022957d9dfe010
>
> requests: 3, hits: 1, misses: 2, hit rate: 33.33%
>
> Retrieved `build2\main.obj' from cache
>
> CacheRetrieve(build2\main.obj):  retrieving from
> 5619d81a934ee71939605662178f5d54
>
> requests: 1, hits: 1, misses: 0, hit rate: 100.00%
>
> CacheRetrieve(build2\foo.obj):  6d30ba71fee8872f9c2c8541225ead92 not in
> cache
>
> requests: 2, hits: 1, misses: 1, hit rate: 50.00%
>
> cl /Fobuild2\\ /c src\foo.c /nologo
>
> foo.c
>
> CachePush(build2\foo.obj):  pushing to 6d30ba71fee8872f9c2c8541225ead92
>
> requests: 2, hits: 1, misses: 1, hit rate: 50.00%
>
> CacheRetrieve(build2\a.exe):  e4533b0dc80dcb3d3b1cd2cdd081933c not in cache
>
> requests: 3, hits: 1, misses: 2, hit rate: 33.33%
>
> link /nologo /OUT:build2\a.exe build2\main.obj build2\foo.obj
>
> LINK : fatal error LNK1181: cannot open input file 'build2\main.obj'
>
> scons: *** [build2\a.exe] Error 1181
>
> scons: building terminated because of errors.
>
>
>
> The odd thing is that it is retrieving build2/main.obj from the cache when
> it is already up-to-date but that still doesn’t explain why it gets deleted…
>
>
>
> S.
>
>
>
> -----Original Message-----
> From: Scons-users <scons-users-bounces at scons.org> On Behalf Of Mats
> Wichmann
> Sent: Wednesday, May 24, 2023 3:49 PM
> To: scons-users at scons.org
> Subject: Re: [Scons-users] SCons bug when combining CacheDir and MSVC_BATCH
>
>
>
> On 5/24/23 06:24, Steve Hill (Wireless) via Scons-users wrote:
>
> > All,
>
> >
>
> > We have recently switched on CacheDir in our CI machine and, while
>
> > this has largely been a positive experience, we have seen the odd link
>
> > failure that we could not explain. I was eventually able to reproduce
>
> > the issue and produce a minimal SConstruct that illustrates the problem.
>
>
>
> Something's broken here.  I've tried to trace this a little bit and it
> looks like things can get removed after they're retrieved.  There are other
> oddities too, in your rebuild log:
>
>
>
> > Retrieved `build1\main.obj' from cache cl /Fobuild1\\ /c src\main.c
>
> > src\foo.c /nologo main.c foo.c Generating Code...
>
>
>
> If there are two sources, but one object is up to date, SCons should not
> calculate a batched build is needed, in other words it should have emitted:
>
>
>
> cl /Fobuild1\foo.o /c src\foo.c /nologo
>
>
>
> > Would you like me to raise an issue on GitHub for this?
>
> I'd say yes, but not being wise in the ways of MSVC_BATCH maybe there's
> something I'm missing.
>
>
>
> Can you try a couple of tweaks to your reproducer (thanks for taking the
> time to come up with one!) while you're at things?
>
>
>
> 1) put the MSVC_BATCH initialization in the Environment call.  The msvc
> tool is notoriously sensitive to getting its initialization wrong if it
> doesn't have all its details in hand when it runs.  No idea if that moves
> the goalposts but worth trying.
>
> 2) turn on some debugging - add  --cache-debug=-   (the dash argument
>
> sends the debug to the console)
>
>
>
>
>
> MSVC_BATCH has several issues already in the tracker about interaction
> with things, didn't look to me like any of them were likely this issue,
> though.
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
> <https://urldefense.com/v3/__https:/pairlist4.pair.net/mailman/listinfo/scons-users__;!!Niha4SQ!4unNd_qrV1SXfyxNYK7N3FaGQ8MEV_5m5uGHpBCJ0aEkO7vz4eA36Eo4phi8NG0eH8Gxxy9xBwdTgKJ5jGShaoveWw4lww$>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20230524/0c226c9b/attachment-0001.htm>