[Scons-users] Possible bug with cache and Value()

[Tim Jenness]

The get_text_contents() vs get_contents() code seems to be an interesting part of the port to python3. I’m not clear why the get_text_contents() implementation in the lines above this is not using kid.get_text_contents(). It seems strange to me that it getting the bytes and then decoding them to strings (without really knowing how to decode) and not asking for the strings directly.

The fix below in get_contents seems wrong to me. get_contents is defined to return bytes and get_text_contents is defined to return characters, so having the code ever return the contents of get_text_contents() is always going to be wrong so that except clause must be a bug on python3. We had a similar problem in the alpha release of Node/FS.py where get_text_contents was returning bytes in the except clause.

As for the reported bug, isn’t the problem below caused by the join being ‘ ,’.join() and not b’ ,’.join()? If sigs are meant to be bytes?


> On Nov 22, 2017, at 12:34 , Bill Deegan <bill at baddogconsulting.com> wrote:
> 
> Try changing the following:
> SCons/Node/Python.py around line 139:
> 
>     def get_contents(self):
>         text_contents = self.get_text_contents()
>         try:
>             return text_contents
>         except UnicodeDecodeError:
>             # Already encoded as python2 str are bytes
>             return text_contents
> 
> 
> 
> The return text_contents is orignally test_contents.encode(), which changes it to bytes.
> Then later the code tries to join a str space with a byte string and crashes.
> 
> I'm going to say this is not actually correct for python 3, but it will work and shouldn't have any issues.
> The reason I'm saying it's not correct is because I think all signitures should be bytes and not strings.
> For py2.7 it doesn't matter.
> For Py 3.5+ I shouldn't matter, but I'm trying to keep all signatures as bytes and be calculated that way.  In the case that the Value() node had a byte string value which wasn't encodable as a unicode string I would expect it to break (read stack trace).
> 
> 
> Thoughts?
> 
> -Bill
> 
> 
> On Wed, Nov 22, 2017 at 7:18 AM, Bill Deegan <bill at baddogconsulting.com <mailto:bill at baddogconsulting.com>> wrote:
> >
> > Rob,
> >
> > At first glance looks like a bug on a code path which must not be covered by our tests.
> > string vs bytes is a dead giveaway.
> >
> > You must be running scons 3.0.1 with python 3.5+?
> > Can you try running with python 2.7.x? (It should work)
> >
> > -Bill
> >
> > On Wed, Nov 22, 2017 at 3:36 AM, Rob Spanton <rob at robspanton.com <mailto:rob at robspanton.com>> wrote:
> >>
> >> Hi,
> >>
> >> I recently upgraded to scons 3.0.1, and I'm encountering a problem with using
> >> CacheDir() in combination with Value().  I can replicate the issue with this
> >> minimal SConstruct:
> >>
> >>     CacheDir("cache")
> >>     Command("somefile", Value("test"), "echo hi > $TARGET")
> >>
> >>     I've included the traceback that this produces at the end of this email.  I
> >>     think what I'm doing with CacheDir, Command and Value is legitimate but am
> >>     entirely happy to be persuaded that it is not!
> >>
> >>     I can work with this for the moment by disabling the cache, but it would be
> >>     useful if someone with some more in-depth scons knowledge than me could shed
> >>     some light on what's going on.
> >>
> >>     Thanks,
> >>
> >>     Rob Spanton
> >>
> >>     ---
> >>
> >>     scons: Reading SConscript files ...
> >>     scons: done reading SConscript files.
> >>     scons: Building targets ...
> >>     scons: *** [somefile] TypeError : sequence item 0: expected str instance, bytes found
> >>     Traceback (most recent call last):
> >>       File "/usr/lib/scons/SCons/Action.py", line 689, in __call__
> >>         cmd = self.strfunction(target, source, env, executor)
> >>     TypeError: CacheRetrieveString() takes 3 positional arguments but 4 were given
> >>
> >>     During handling of the above exception, another exception occurred:
> >>
> >>     Traceback (most recent call last):
> >>       File "/usr/lib/scons/SCons/Taskmaster.py", line 241, in execute
> >>         if not t.retrieve_from_cache():
> >>       File "/usr/lib/scons/SCons/Node/FS.py", line 2925, in retrieve_from_cache
> >>         return self.get_build_env().get_CacheDir().retrieve(self)
> >>       File "/usr/lib/scons/SCons/CacheDir.py", line 263, in retrieve
> >>         if CacheRetrieve(node, [], env, execute=1) == 0:
> >>       File "/usr/lib/scons/SCons/Action.py", line 691, in __call__
> >>         cmd = self.strfunction(target, source, env)
> >>       File "/usr/lib/scons/SCons/CacheDir.py", line 66, in CacheRetrieveString
> >>         cachedir, cachefile = cd.cachepath(t)
> >>       File "/usr/lib/scons/SCons/CacheDir.py", line 224, in cachepath
> >>         sig = node.get_cachedir_bsig()
> >>       File "/usr/lib/scons/SCons/Node/FS.py", line 3391, in get_cachedir_bsig
> >>         result = self.cachesig = SCons.Util.MD5collect(sigs)
> >>       File "/usr/lib/scons/SCons/Util.py", line 1550, in MD5collect
> >>         return MD5signature(', '.join(signatures))
> >>     TypeError: sequence item 0: expected str instance, bytes found
> >>     scons: building terminated because of errors.
> >> _______________________________________________
> >> Scons-users mailing list
> >> Scons-users at scons.org <mailto:Scons-users at scons.org>
> >> https://pairlist4.pair.net/mailman/listinfo/scons-users <https://pairlist4.pair.net/mailman/listinfo/scons-users>
> >
> >
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20171122/98c995c4/attachment.html>