[Scons-users] Reading Python 3.8.0 .dblite file from Python 2.7

[Don Baldwin]

Hi Bill,

Thanks for your response.  Just to clarify though, I’m not trying to build the same build with both Python 2 and 3.  I have external tools that call sconsign to read the .dblite file to do analyses on it, and those tools are executed under Python 2.  Therefore, I don’t think I’ll see any rebuild issues.  My only issue is that the tools we use to read the dblite file are written for Python 2, and can’t support the newer pickle protocols.

We haven’t seen any issues trying to read .dblite files from SCons 3.8.0 builds that were run under Python 2.7, so we must not be looking at any of the objects that changed.  We’re mostly just looking at dependencies.

-Don


From: Bill Deegan <bill at baddogconsulting.com>
Sent: Tuesday, December 3, 2019 9:35 PM
To: SCons users mailing list <scons-users at scons.org>
Cc: Don Baldwin <donb at qti.qualcomm.com>
Subject: [EXT] Re: [Scons-users] Reading Python 3.8.0 .dblite file from Python 2.7

Don,

Even if you use the same pickle versions you will likely still see some rebuilds as python 2 vs python 3 pickle some  objects differently. Also the signatures of python function actions will be different.
And this is explicitly specified in CHANGES.txt and the release notes for 3.0.0
You can specify the SConsign file name and add .2 and .3 on the end so at least they won't conflict with each other.

-Bill

On Tue, Dec 3, 2019 at 4:48 PM Mats Wichmann <mats at wichmann.us<mailto:mats at wichmann.us>> wrote:
On 12/3/19 5:35 PM, Don Baldwin wrote:
> Hi,
>

>     ignoring invalid `dblite' file
>
>  From what I've read, this seems to be due to the incompatibility
> between the pickle library between Python 2 and 3.

evolving protocols, yes. actually it's not just py 2/3, 3.8 added a new
pickle protocol that earlier Py3's can't read.

> I see the following code in lib/SCons/compat/__init__.py:
>
>     # Was pickle.HIGHEST_PROTOCOL
>     # Changed to 2 so py3.5+'s pickle will be compatible with py2.7.
>     PICKLE_PROTOCOL = pickle.HIGHEST_PROTOCOL
>
>
> When I modify this to set PICKLE_PROTOCOL = 2, my tools are able to
> successfully read the .dblite files, but I'd prefer not to have to
> modify the native SCons code.
>
> Is there a way to set this value from my SConscript file, or some other
> preferred way to handle this situation?
No.  SCons picks the highest available pickle version which is arguably
a dubious choice for your situation (and mine, it turns out), but that's
what it does.  There's no official API-ish way to request a different
one.  that would be a feature request, I guess.

_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20191204/aad8a1c3/attachment-0001.html>