[Scons-users] Bug causing corruption of .sconsign.dblite

[Andrew C. Morrow]

Dirk, Bill, William -

Thanks for providing a great deal more context. I certainly understand
the need to prioritize features over bugs that have workarounds, and
it appears that I misconstrued the intent of William's earlier email,
so, apologies for that.

Part of my frustration was that I actually agree with your assessment
that fixing this issue is perhaps very challenging. It did not seem
like something that would be reasonably tackled by anyone outside of
the core group of SCons developers, and I'm not one of those.

As to the impact of the bug, using the MD5-Timestamp provider and
CacheDir each has substantial cost savings for us in terms of
incremental build times, so we do rely on them for developer
productivity. Being unable to use them together is really unfortunate.
While I appreciate Bill's point that spurious rebuilds of files is
'harmless' in a technical sense, a single spuriously recompiled
translation unit early in the graph can cause a substantial amount of
follow-on work, in particular, very expensive re-links.

Finally, I'd actually be personally happy to help the SCons project
more with development, but my work commitments don't realistically
allow it. As you note, MongoDB is heavily dependent on SCons, and we
have a number of changes that we would like to see. As I'm sure is
unsurprising though, we have our development priorities too. We
actually have had some internal discussion recently about whether
MongoDB might sponsor some SCons development work on features or fixes
that are important to us. Would there be any interest from the core
developers?

Thanks,
Andrew


On Tue, Aug 25, 2015 at 3:04 PM, Dirk Bächle <tshortik at gmx.de> wrote:
> Hi Andrew,
>
> I'd like to second Bill's thoughts. The signature subsystem in SCons isn't
> easy to comprehend and fix. It's scattered all over the place and pretty
> much undocumented (at least compared to the rest of our documentation, it's
> far behind).
> So chances are slim to none that some newbie will jump on this issue and
> come up with a valid fix (not breaking any existing behaviour) in a week or
> so.
>
> In fact, I did have a look at the problem some time ago and can't recall all
> the details...but this isn't a straight-forward patch to make. It probably
> involves changing the Decider API at some places, breaking of existing
> scripts included. There may be a way around this, but so far I found none.
> I also didn't find any time to properly document or publish my findings up
> to now...not to speak of improving the signature system documentation along
> the way (which is also on my TODO list).
>
> I plan to pick up my work on this, but before that I have several other
> topics to fight down...just like every other (core) developer.
>
> Finally, this bug is connected to the "MD5-timestamp" decider only...so a
> valid workaround exists in using the "MD5" decider, which should make things
> work fine again. I understand that this gives you headaches for the MongoDB
> project, regarding build performance. But we currently have some larger
> topics (slots, subprocess wrapper, new toolchains, Python 3) to address,
> that seem strategically important for the project. I hope you can understand
> that we, as core team, see it as our duty to attack these things first.
>
> Best regards,
>
> Dirk
>
> On 25.08.2015 16:33, Bill Deegan wrote:
>>
>> Andrew,
>>
>> Generally SCons releases are a compilation of bug fixes submitted by our
>> user community.
>> Nobody works on SCons full time.
>>
>> So scheduling a bug for a given release implies that there's a guarantee
>> that there will be a fix developed and/or means that a
>> given release cannot be released without that fix.
>>
>> As far as I know, no one is actively working on this particular bug, so
>> holding up a release for this bug to be fixed would mean
>> that other fixes wouldn't make there way to our user community.
>>
>> So while I may have phrased what William Blevins said differently, there's
>> some truth in his message.
>> If you want a bug fixed you have three options: 1) fix it your self, 2)
>> encourage another community member to fix it for you, 3) pay
>> someone to fix it for you.
>> This is true with all open source (with the possible exception of projects
>> where there is a commercial entity behind it).
>>
>> Also it's fair to add that this particular bug may potential require a
>> non-trivial amount of code change and understanding of SCons
>> internals. Then we'll have to make sure we have test coverage for the bug
>> as well.
>>
>> If you'll note currently we're working on getting the slots changes
>> stabilized tested, and hopefully sometime soon, released.  This
>> change provides significant runtime and memory improvements.
>>
>> If it a choice between getting that out and fixing a bug where the failure
>> is that some objects are rebuilt when they don't need to
>> be, it seems a pretty clear choice as to which we should focus on.
>>
>> Now if the bug were inverted, and objects were not getting built when they
>> should, then this would be a much higher priority bug and
>> likely would get more immediate attention.
>>
>> I don't think anyone has said they will not work on this bug, but they
>> have said they have higher priority issues they are working
>> on. Of course the bug which is bugging you now will always seem the
>> highest priority..
>>
>> Hope that helps!
>> -Bill
>>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users