[Scons-users] SCons 3.0, sconsign files and Py2 vs Py3.. how to handle sconsign imcompatabilities

[Bill Deegan]

The pickles are compatible if you use the same pickle protocol.
I have pushed changes as suggested above with encoding='bytes' (for py3
only) as well.

There are still many rebuilds when switching python versions, but no longer
errors.

Anyone want to help dig into why things are rebuilding when switching?
I'm fairly certain most if not all are attributable to function actions
whose signature includes the bytecode which is different between py2 and
py3 ( and py3.5 and py3.6 in at least some if not all cases).

-Bill



On Mon, Sep 11, 2017 at 8:12 AM, Neal Becker <ndbecker2 at gmail.com> wrote:

> Tim Jenness wrote:
>
> >
> >
> >> On Sep 9, 2017, at 19:13, Bill Deegan <bill at baddogconsulting.com>
> wrote:
> >>
> >> Greetings,
> >>
> >> This is (I hope) the last issue gating 3.0 release.
> >
> > Fantastic. I guess the question is how many people are going to be
> > switching their scons python versions regularly and whether the .sconsign
> > file is part of a public interface outside of scons itself.
> >
> >>
> >> If we set the default pickle version to 2. (Currently set to
> >> HIGHEST_AVAILABLE, which means 4 for py3.5+ and 2 for py2.7)
> >>
> >
> > This is the safest option for portability.
> >
> >> If you run scons with py2.7 and then run again with py3.x, you'll get
> the
> >> following: ...
> >> scons: *** [$DISTDIR/scons-local-3.0.0.alpha.20170821.tar.gz]
> >> UnicodeDecodeError : 'ascii' codec can't decode byte 0x80 in position 0:
> >> ordinal not in range(128)
> >>
> >
> > is this error from SCons or from pickle? If it’s from SCons, why is py2.7
> > writing non-ascii and why is py3 using ascii to read it?
> >
> >> If you run scons with py3 and then rerun with py2.7, you'll get the
> >> following:
> >>  * It will rebuild everything you just built with py3
> >>
> >
> > Not a complete disaster but a bit weird, it’s almost like it doesn’t
> > really understand what it read from the file.
> >
> >> If we set the default pickle version  to HIGHEST_AVAILABLE, which means
> 4
> >> for py3.5+ and 2 for py2.7
> >>
> >
> > what happens?
> >
> >>
> >> Here's the options I see to deal with this:
> >> 1. Always append .py3 to PY3 scons runs' sconsign files
> >
> > This is the safe option unless .sconsign is assumed to be a public
> > interface.
> >
> > If people really are going to switch py2/py3 regularly then this sounds
> > like it might get really confusing.
> >
> >> 2. Always append .py# to sconsign files based on what version of python
> >> was used, ignore pre-existing .sconsigns from older versions
> >
> > This will lead to the same state confusion as for #1 if people are
> > switching all the time.
> >
> >
> >> 3. Do something complicated:
> >>    a. Leave pickle version as HIGHEST_AVAILABLE, detect version of
> pickle
> >>    by reading first two bytes. If incompatible version detected, change
> >>    file name to .py# and try reading that one.  Always write to
> >>    .sconsign? Or write to file read. b. ?
> >>
> >
> > The best option is to work out why pickle version 2 is not compatible.
> >
> > Can you get text dumps of the contents from py2 and py3 to see what
> > differs? It must be some unicode weirdness.
> >
> > —
> > Tim Jenness
>
> python2 and python3 pickles are not compatible AFAIK.  The only good option
> is for scons to append .py3.  This is what I did for my port of scons to
> py3.
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20170911/351d7409/attachment.html>