[Scons-users] Scons swallowing exception stack trace for jinja2.exceptions.TemplateNotFound
Samir Mahaboob Khan Kagadkar
kagadkar at qti.qualcomm.com
Wed May 15 12:32:17 EDT 2024
Thanks Mats and Bill for your valuable feedback here.
Yes, I’m catching the exception with a try-except and re-raising as a different exception to make forward progress.
Would you like me to file an scons bug?
Thanks
Samir
From: Scons-users <scons-users-bounces at scons.org> on behalf of scons-users-request at scons.org <scons-users-request at scons.org>
Date: Tuesday, May 14, 2024 at 10:36 PM
To: scons-users at scons.org <scons-users at scons.org>
Subject: Scons-users Digest, Vol 153, Issue 2
WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.
Send Scons-users mailing list submissions to
scons-users at scons.org
To subscribe or unsubscribe via the World Wide Web, visit
https://pairlist4.pair.net/mailman/listinfo/scons-users
or, via email, send a message with subject or body 'help' to
scons-users-request at scons.org
You can reach the person managing the list at
scons-users-owner at scons.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Scons-users digest..."
Today's Topics:
1. Re: Scons swallowing exception stack trace for
jinja2.exceptions.TemplateNotFound (Mats Wichmann)
2. Re: Scons swallowing exception stack trace for
jinja2.exceptions.TemplateNotFound (Bill Deegan)
----------------------------------------------------------------------
Message: 1
Date: Tue, 14 May 2024 08:04:23 -0600
From: Mats Wichmann <mats at wichmann.us>
To: scons-users at scons.org
Subject: Re: [Scons-users] Scons swallowing exception stack trace for
jinja2.exceptions.TemplateNotFound
Message-ID: <337e236d-ded6-42e1-b29e-7efe4519c922 at wichmann.us>
Content-Type: text/plain; charset=UTF-8; format=flowed
On 5/13/24 22:16, Samir Mahaboob Khan Kagadkar via Scons-users wrote:
> Hi SCons Team,
>
> I use the Jinja2 templating language
> (https://jinja.palletsprojects.com/en/3.1.x/
> <https://jinja.palletsprojects.com/en/3.1.x/>) to generate scripts from
> within SCons.
>
> Jinja2 raises the TemplateNotFound (jinja2.exceptions.TemplateNotFound)
> exception when, say, a file included in the template cannot be found.
>
> Unfortunately scons does not show the exception stack trace when this is
> raised as part of a builder.
>
> Here is a simple SConstruct that demonstrates this:
>
> import jinja2
>
> ?def small_builder(target, source, env):
>
> ???? raise jinja2.exceptions.TemplateNotFound("You should see this
> message")
>
> ?env=Environment()
>
> env.Command(['dummy/TEST.COMPLETE'], [], small_builder, chdir=0)
>
> Here is the output not showing any stack trace:
>
> $ scons -f <path_to_sconstruct_dir>/SConstruct dummy/TEST.COMPLETE
>
> scons: Reading SConscript files ...
>
> scons: done reading SConscript files.
>
> scons: Building targets ...
>
> small_builder(["dummy/TEST.COMPLETE"], [])
>
> scons: done building targets.
>
> The TemplateNotFound can be raised in a simple standalone python3 script
> invocation:
>
> import jinja2
>
> raise jinja2.exceptions.TemplateNotFound("You should see this message")
>
> with the below output
>
> $ python3 test1.py
>
> Traceback (most recent call last):
>
> ? File "test1.py", line 2, in <module>
>
> ??? raise jinja2.exceptions.TemplateNotFound("You should see this message")
>
> jinja2.exceptions.TemplateNotFound: You should see this message
>
> Other exceptions work in SCons builders, of course. For example:
>
> import jinja2
>
> ?def small_builder(target, source, env):
>
> ???? #raise jinja2.exceptions.TemplateNotFound("You should see this
> message")
>
> ???? raise ValueError("You should see this message")
>
> ?env=Environment()
>
> env.Command(['dummy/TEST.COMPLETE'], [], small_builder, chdir=0)
>
> with the following output:
>
> scons: Reading SConscript files ...
>
> scons: done reading SConscript files.
>
> scons: Building targets ...
>
> small_builder(["dummy/TEST.COMPLETE"], [])
>
> scons: *** [dummy/TEST.COMPLETE] ValueError : You should see this message
>
> Traceback (most recent call last):
>
> ? File ?<?.>/SCons/Action.py", line 1441, in execute
>
> ??? result = self.execfunction(target=target, source=rsources, env=env)
>
> ? File "<?.> /SConstruct", line 5, in small_builder
>
> ??? raise ValueError("You should see this message")
>
> ValueError: You should see this message
>
> scons: building terminated because of errors.
>
> Could you please take a look?
SCons has a routine which tries to synthesize a BuildError exception
upon detecting other exceptions that may have been raised during the
call to the action function. If doesn't do that right for the jinja
template exception - which I see is itself synthesized. I don't really
see how to cause it to do better at the moment.
convert_to_BuildError recognizes the template error as an instance of
IOError. In that case it tries to do this:
filename = getattr(status, 'filename', None)
strerror = getattr(status, 'strerror', str(status))
errno = getattr(status, 'errno', 2)
buildError = BuildError(
errstr=strerror,
status=errno,
exitstatus=2,
filename=filename,
exc_info=exc_info)
the TemplateNotFound has all three attributes that are checked for - set
to None, so the getattr defaults aren't used. The BuildError thus ends
up with an error message and status of None, which not surprisingly,
doesn't yield any interesting message.
I suppose the attribute check could be a little more pedantic, like this:
strerror = getattr(status, 'strerror', None)
if strerror is None:
strerror = str(status)
@bdbaddog - any thoughts?
------------------------------
Message: 2
Date: Tue, 14 May 2024 09:08:56 -0700
From: Bill Deegan <bill at baddogconsulting.com>
To: SCons users mailing list <scons-users at scons.org>
Subject: Re: [Scons-users] Scons swallowing exception stack trace for
jinja2.exceptions.TemplateNotFound
Message-ID:
<CAEyG4CEEgxgHC9fz1H-Fw1e562JfHwS-caZqr9eZgOacndh4+A at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
I'm guessing you've created a python action to process the jinja2 template?
If so you should catch whatever exceptions jinja2 may through and re-raise
them as the relevant SCons exception.
That's likely the best fit.
Doing a lot of processing, especially file operations in python actions,
can hold the GIL and limit parallelism, and generally not encouraged.
So another way to resolve that is to move your jinja2 processing to a
standalone script, then when that script fails, scons would see the error.
-Bill
On Tue, May 14, 2024 at 7:04?AM Mats Wichmann <mats at wichmann.us> wrote:
> On 5/13/24 22:16, Samir Mahaboob Khan Kagadkar via Scons-users wrote:
> > Hi SCons Team,
> >
> > I use the Jinja2 templating language
> > (https://jinja.palletsprojects.com/en/3.1.x/
> > <https://jinja.palletsprojects.com/en/3.1.x/>) to generate scripts from
> > within SCons.
> >
> > Jinja2 raises the TemplateNotFound (jinja2.exceptions.TemplateNotFound)
> > exception when, say, a file included in the template cannot be found.
> >
> > Unfortunately scons does not show the exception stack trace when this is
> > raised as part of a builder.
> >
> > Here is a simple SConstruct that demonstrates this:
> >
> > import jinja2
> >
> > def small_builder(target, source, env):
> >
> > raise jinja2.exceptions.TemplateNotFound("You should see this
> > message")
> >
> > env=Environment()
> >
> > env.Command(['dummy/TEST.COMPLETE'], [], small_builder, chdir=0)
> >
> > Here is the output not showing any stack trace:
> >
> > $ scons -f <path_to_sconstruct_dir>/SConstruct dummy/TEST.COMPLETE
> >
> > scons: Reading SConscript files ...
> >
> > scons: done reading SConscript files.
> >
> > scons: Building targets ...
> >
> > small_builder(["dummy/TEST.COMPLETE"], [])
> >
> > scons: done building targets.
> >
> > The TemplateNotFound can be raised in a simple standalone python3 script
> > invocation:
> >
> > import jinja2
> >
> > raise jinja2.exceptions.TemplateNotFound("You should see this message")
> >
> > with the below output
> >
> > $ python3 test1.py
> >
> > Traceback (most recent call last):
> >
> > File "test1.py", line 2, in <module>
> >
> > raise jinja2.exceptions.TemplateNotFound("You should see this
> message")
> >
> > jinja2.exceptions.TemplateNotFound: You should see this message
> >
> > Other exceptions work in SCons builders, of course. For example:
> >
> > import jinja2
> >
> > def small_builder(target, source, env):
> >
> > #raise jinja2.exceptions.TemplateNotFound("You should see this
> > message")
> >
> > raise ValueError("You should see this message")
> >
> > env=Environment()
> >
> > env.Command(['dummy/TEST.COMPLETE'], [], small_builder, chdir=0)
> >
> > with the following output:
> >
> > scons: Reading SConscript files ...
> >
> > scons: done reading SConscript files.
> >
> > scons: Building targets ...
> >
> > small_builder(["dummy/TEST.COMPLETE"], [])
> >
> > scons: *** [dummy/TEST.COMPLETE] ValueError : You should see this message
> >
> > Traceback (most recent call last):
> >
> > File ?<?.>/SCons/Action.py", line 1441, in execute
> >
> > result = self.execfunction(target=target, source=rsources, env=env)
> >
> > File "<?.> /SConstruct", line 5, in small_builder
> >
> > raise ValueError("You should see this message")
> >
> > ValueError: You should see this message
> >
> > scons: building terminated because of errors.
>
> >
> > Could you please take a look?
>
> SCons has a routine which tries to synthesize a BuildError exception
> upon detecting other exceptions that may have been raised during the
> call to the action function. If doesn't do that right for the jinja
> template exception - which I see is itself synthesized. I don't really
> see how to cause it to do better at the moment.
>
> convert_to_BuildError recognizes the template error as an instance of
> IOError. In that case it tries to do this:
>
> filename = getattr(status, 'filename', None)
>
> strerror = getattr(status, 'strerror', str(status))
>
> errno = getattr(status, 'errno', 2)
>
>
>
> buildError = BuildError(
>
> errstr=strerror,
>
> status=errno,
>
> exitstatus=2,
>
> filename=filename,
>
> exc_info=exc_info)
>
> the TemplateNotFound has all three attributes that are checked for - set
> to None, so the getattr defaults aren't used. The BuildError thus ends
> up with an error message and status of None, which not surprisingly,
> doesn't yield any interesting message.
>
> I suppose the attribute check could be a little more pedantic, like this:
>
> strerror = getattr(status, 'strerror', None)
>
> if strerror is None:
>
> strerror = str(status)
>
> @bdbaddog - any thoughts?
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20240514/32ae8863/attachment-0001.htm>
------------------------------
Subject: Digest Footer
_______________________________________________
Scons-users mailing list
Scons-users at scons.org
https://pairlist4.pair.net/mailman/listinfo/scons-users
------------------------------
End of Scons-users Digest, Vol 153, Issue 2
*******************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20240515/8cbabab9/attachment-0001.htm>
More information about the Scons-users
mailing list