[Scons-users] SCons 3.0, sconsign files and Py2 vs Py3.. how to handle sconsign imcompatabilities

[Neal Becker]

Tim Jenness wrote:

> 
> 
>> On Sep 9, 2017, at 19:13, Bill Deegan <bill at baddogconsulting.com> wrote:
>> 
>> Greetings,
>> 
>> This is (I hope) the last issue gating 3.0 release.
> 
> Fantastic. I guess the question is how many people are going to be
> switching their scons python versions regularly and whether the .sconsign
> file is part of a public interface outside of scons itself.
> 
>> 
>> If we set the default pickle version to 2. (Currently set to
>> HIGHEST_AVAILABLE, which means 4 for py3.5+ and 2 for py2.7)
>> 
> 
> This is the safest option for portability.
> 
>> If you run scons with py2.7 and then run again with py3.x, you'll get the
>> following: ...
>> scons: *** [$DISTDIR/scons-local-3.0.0.alpha.20170821.tar.gz]
>> UnicodeDecodeError : 'ascii' codec can't decode byte 0x80 in position 0:
>> ordinal not in range(128)
>> 
> 
> is this error from SCons or from pickle? If it’s from SCons, why is py2.7
> writing non-ascii and why is py3 using ascii to read it?
> 
>> If you run scons with py3 and then rerun with py2.7, you'll get the
>> following:
>>  * It will rebuild everything you just built with py3
>> 
> 
> Not a complete disaster but a bit weird, it’s almost like it doesn’t
> really understand what it read from the file.
> 
>> If we set the default pickle version  to HIGHEST_AVAILABLE, which means 4
>> for py3.5+ and 2 for py2.7
>> 
> 
> what happens?
> 
>> 
>> Here's the options I see to deal with this:
>> 1. Always append .py3 to PY3 scons runs' sconsign files
> 
> This is the safe option unless .sconsign is assumed to be a public
> interface.
> 
> If people really are going to switch py2/py3 regularly then this sounds
> like it might get really confusing.
> 
>> 2. Always append .py# to sconsign files based on what version of python
>> was used, ignore pre-existing .sconsigns from older versions
> 
> This will lead to the same state confusion as for #1 if people are
> switching all the time.
> 
> 
>> 3. Do something complicated:
>>    a. Leave pickle version as HIGHEST_AVAILABLE, detect version of pickle
>>    by reading first two bytes. If incompatible version detected, change
>>    file name to .py# and try reading that one.  Always write to
>>    .sconsign? Or write to file read. b. ?
>> 
> 
> The best option is to work out why pickle version 2 is not compatible.
> 
> Can you get text dumps of the contents from py2 and py3 to see what
> differs? It must be some unicode weirdness.
> 
> —
> Tim Jenness

python2 and python3 pickles are not compatible AFAIK.  The only good option 
is for scons to append .py3.  This is what I did for my port of scons to 
py3.