[Scons-users] Bug causing corruption of .sconsign.dblite

[Bill Deegan]

Andrew,


On Tue, Aug 25, 2015 at 12:54 PM, Andrew C. Morrow <
andrew.c.morrow at gmail.com> wrote:

> Dirk, Bill, William -
>
> Thanks for providing a great deal more context. I certainly understand
> the need to prioritize features over bugs that have workarounds, and
> it appears that I misconstrued the intent of William's earlier email,
> so, apologies for that.
>
> Part of my frustration was that I actually agree with your assessment
> that fixing this issue is perhaps very challenging. It did not seem
> like something that would be reasonably tackled by anyone outside of
> the core group of SCons developers, and I'm not one of those.
>
> As to the impact of the bug, using the MD5-Timestamp provider and
> CacheDir each has substantial cost savings for us in terms of
> incremental build times, so we do rely on them for developer
> productivity. Being unable to use them together is really unfortunate.
> While I appreciate Bill's point that spurious rebuilds of files is
> 'harmless' in a technical sense, a single spuriously recompiled
> translation unit early in the graph can cause a substantial amount of
> follow-on work, in particular, very expensive re-links.
>

Perhaps better to describe this bug as not harmless, but not producing an
incorrect build.
Understood that the downstream results of the bug can be non-trivial amount
of wasted compute and CPU.


>
> Finally, I'd actually be personally happy to help the SCons project
> more with development, but my work commitments don't realistically
> allow it. As you note, MongoDB is heavily dependent on SCons, and we
> have a number of changes that we would like to see. As I'm sure is
> unsurprising though, we have our development priorities too. We
> actually have had some internal discussion recently about whether
> MongoDB might sponsor some SCons development work on features or fixes
> that are important to us. Would there be any interest from the core
> developers?
>

Indeed yes.
I do have a consulting business and bugfixing for open source projects is a
non-trivial part thereof.
Feel free to contact me off list.  Note that I'm currently bouncing back
and forth between NYC and SF.
Indeed mongo's NYC office is 3 blocks from our NYC apartment.

-Bill


>
> Thanks,
> Andrew
>
>
> On Tue, Aug 25, 2015 at 3:04 PM, Dirk Bächle <tshortik at gmx.de> wrote:
> > Hi Andrew,
> >
> > I'd like to second Bill's thoughts. The signature subsystem in SCons
> isn't
> > easy to comprehend and fix. It's scattered all over the place and pretty
> > much undocumented (at least compared to the rest of our documentation,
> it's
> > far behind).
> > So chances are slim to none that some newbie will jump on this issue and
> > come up with a valid fix (not breaking any existing behaviour) in a week
> or
> > so.
> >
> > In fact, I did have a look at the problem some time ago and can't recall
> all
> > the details...but this isn't a straight-forward patch to make. It
> probably
> > involves changing the Decider API at some places, breaking of existing
> > scripts included. There may be a way around this, but so far I found
> none.
> > I also didn't find any time to properly document or publish my findings
> up
> > to now...not to speak of improving the signature system documentation
> along
> > the way (which is also on my TODO list).
> >
> > I plan to pick up my work on this, but before that I have several other
> > topics to fight down...just like every other (core) developer.
> >
> > Finally, this bug is connected to the "MD5-timestamp" decider only...so a
> > valid workaround exists in using the "MD5" decider, which should make
> things
> > work fine again. I understand that this gives you headaches for the
> MongoDB
> > project, regarding build performance. But we currently have some larger
> > topics (slots, subprocess wrapper, new toolchains, Python 3) to address,
> > that seem strategically important for the project. I hope you can
> understand
> > that we, as core team, see it as our duty to attack these things first.
> >
> > Best regards,
> >
> > Dirk
> >
> > On 25.08.2015 16:33, Bill Deegan wrote:
> >>
> >> Andrew,
> >>
> >> Generally SCons releases are a compilation of bug fixes submitted by our
> >> user community.
> >> Nobody works on SCons full time.
> >>
> >> So scheduling a bug for a given release implies that there's a guarantee
> >> that there will be a fix developed and/or means that a
> >> given release cannot be released without that fix.
> >>
> >> As far as I know, no one is actively working on this particular bug, so
> >> holding up a release for this bug to be fixed would mean
> >> that other fixes wouldn't make there way to our user community.
> >>
> >> So while I may have phrased what William Blevins said differently,
> there's
> >> some truth in his message.
> >> If you want a bug fixed you have three options: 1) fix it your self, 2)
> >> encourage another community member to fix it for you, 3) pay
> >> someone to fix it for you.
> >> This is true with all open source (with the possible exception of
> projects
> >> where there is a commercial entity behind it).
> >>
> >> Also it's fair to add that this particular bug may potential require a
> >> non-trivial amount of code change and understanding of SCons
> >> internals. Then we'll have to make sure we have test coverage for the
> bug
> >> as well.
> >>
> >> If you'll note currently we're working on getting the slots changes
> >> stabilized tested, and hopefully sometime soon, released.  This
> >> change provides significant runtime and memory improvements.
> >>
> >> If it a choice between getting that out and fixing a bug where the
> failure
> >> is that some objects are rebuilt when they don't need to
> >> be, it seems a pretty clear choice as to which we should focus on.
> >>
> >> Now if the bug were inverted, and objects were not getting built when
> they
> >> should, then this would be a much higher priority bug and
> >> likely would get more immediate attention.
> >>
> >> I don't think anyone has said they will not work on this bug, but they
> >> have said they have higher priority issues they are working
> >> on. Of course the bug which is bugging you now will always seem the
> >> highest priority..
> >>
> >> Hope that helps!
> >> -Bill
> >>
> >
> > _______________________________________________
> > Scons-users mailing list
> > Scons-users at scons.org
> > https://pairlist4.pair.net/mailman/listinfo/scons-users
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20150825/95f031db/attachment-0001.html>