[Scons-users] Problems with a pseudo-builder

[Marc Joliet]

OK, here's the follow up I promised.

Am Fri, 7 Jun 2013 21:24:09 +0200
schrieb Marc Joliet <marcec at gmx.de>:

[...]

> And in pseudo_builders.py I have:

> 

>     def doc_builder(env, target, source, *args, **kwargs):

>         """

>         A pseudo-builder for generating documentation via faust -mdoc.

> 

>         In addition to what the doc builder does, this pseudo-builder marks the

>         mdoc directory for cleaning.

>         """

> 

>         source = [env.File(s) for s in source]

>         r = builders.doc(env, target, source, *args, **kwargs)

> 

>         for t, s in zip(r, source):

> 

>             # get a bunch of paths

>             svg_dir     = env.Dir(os.sep.join([t.abspath, "svg", "svg-01"]))

>             process_svg = svg_dir.File("process.svg")

>             process_pdf = svg_dir.File("process.pdf")

>             s_basename  = os.path.splitext(os.path.basename(s.path))[0]

>             tex_path    = env.File(os.sep.join([t.abspath, "tex", s_basename+".tex"]))

> 	      pdf_path    = env.File(os.sep.join([t.abspath, "pdf", s_basename+".pdf"]))

> 

>             print(pdf_path)

> 

>             # compile the LaTeX sources to PDF

>             process_pdf = builders.cairosvg(env, process_pdf.abspath, process_svg.abspath)

>             cur_pdf  = env.PDF(pdf_path, tex_path)

> 

>             # set the dependencies straight

>             env.Depends(process_pdf, t)

>             env.Depends(cur_pdf, process_pdf)

> 

>             # the svg and pdf subdirectories change because of process_pdf and

>             # cur_pdf, so make sure the *-mdoc directory ignores them

>             env.Ignore(t, t.Dir("svg"))

>             env.Ignore(t, t.Dir("pdf"))

> 

>             # make sure to clean up the documentation directory

>             env.Clean(t, t.abspath)

> 

>             r.extend([cur_pdf, process_pdf])

> 

>         return r

[...]

So I made two changes and the pseudo-builder now looks like this:

    def doc_builder(env, target, source, *args, **kwargs):
        """
        A pseudo-builder for generating documentation via faust -mdoc.

        In addition to what the doc builder does, this pseudo-builder marks the mdoc
        directory for cleaning.
        """

        source = [env.File(s) for s in source]
        r = builders.doc(env, target, source, *args, **kwargs)

        for t, s in zip(r, source):

            # get a bunch of paths
            svg_dir     = env.Dir(os.sep.join([t.abspath, "svg", "svg-01"]))
            process_svg = svg_dir.File("process.svg")
            process_pdf = svg_dir.File("process.pdf")
            s_basename  = os.path.splitext(os.path.basename(s.path))[0]
            tex_path    = env.File(os.sep.join([t.abspath, "tex", s_basename+".tex"]))
            pdf_path    = env.File(os.sep.join([t.abspath, "tex", s_basename+".pdf"]))

            print(pdf_path)

            # compile the LaTeX sources to PDF
            process_pdf = builders.cairosvg(env, process_pdf.abspath, process_svg.abspath)
            cur_pdf  = env.PDF(pdf_path, tex_path)

            # the svg and pdf subdirectories change because of process_pdf and
            # cur_pdf, so make sure the *-mdoc directory ignores them
            env.Ignore(t, t.Dir("svg"))
            env.Ignore(t, t.Dir("tex"))

            # set the dependencies straight
            env.Depends(process_svg, t)
            env.Depends(cur_pdf, process_pdf)

            # make sure to clean up the documentation directory
            env.Clean(t, t.abspath)

            r.extend([cur_pdf, process_pdf])

        return r

The two changes I made are:

- the first env.Depends() now has process_svg (instead of process_pdf) depend on t
- pdf_path now resides in the same directory as the LaTeX source

The first change consistently moves the error when using "-j2" to the cur_pdf
object, i.e., process_pdf always builds and the error is now with cur_pdf.  I
don't know *why* that should be the case, but that's what happens.  The scons
output is thus always similar to:

    $ scons -j2 rmfbs_sum-mdoc
    scons: Reading SConscript files ...

    scons: warning: QT4DIR variable is not defined, using moc executable as a hint (QT4DIR=/usr)
    File "/home/marcec/.scons/site_scons/site_tools/qt4/__init__.py", line 384, in _detect
    /home/marcec/programming/faust_rm_filter_bank/mdoc/mbstereophonyd_sum-mdoc/tex/mbstereophonyd_sum.pdf
    /home/marcec/programming/faust_rm_filter_bank/mdoc/mbstereophonyd_syn-mdoc/tex/mbstereophonyd_syn.pdf
    /home/marcec/programming/faust_rm_filter_bank/mdoc/mbstereophonys_sum-mdoc/tex/mbstereophonys_sum.pdf
    /home/marcec/programming/faust_rm_filter_bank/mdoc/mbstereophonys_syn-mdoc/tex/mbstereophonys_syn.pdf
    /home/marcec/programming/faust_rm_filter_bank/mdoc/rmfbd_sum-mdoc/tex/rmfbd_sum.pdf
    /home/marcec/programming/faust_rm_filter_bank/mdoc/rmfbd_syn-mdoc/tex/rmfbd_syn.pdf
    /home/marcec/programming/faust_rm_filter_bank/mdoc/rmfbs_sum-mdoc/tex/rmfbs_sum.pdf
    /home/marcec/programming/faust_rm_filter_bank/mdoc/rmfbs_syn-mdoc/tex/rmfbs_syn.pdf
    scons: done reading SConscript files.
    scons: Building targets ...
    /usr/bin/faust -t 4800 -vec -o /dev/null -mdoc src/rmfbs_sum.dsp
    Move("mdoc/rmfbs_sum-mdoc", "src/rmfbs_sum-mdoc")
    cairosvg --format=pdf -o mdoc/rmfbs_sum-mdoc/svg/svg-01/process.pdf mdoc/rmfbs_sum-mdoc/svg/svg-01/process.svg
    scons: *** [mdoc/rmfbs_sum-mdoc/tex/rmfbs_sum.pdf] Source `mdoc/rmfbs_sum-mdoc/tex/rmfbs_sum.tex' not found, needed by target `mdoc/rmfbs_sum-mdoc/tex/rmfbs_sum.pdf'.
    scons: building terminated because of errors.

The second change makes "-j1" work consistently (except again for the case with
an empty target argument; I also tried passing the source as a keyword argument
"source=dsp", in case there was an issue with it being wrapped in a
BuilderWrapper, but that changed nothing).

So as to why I had to make the second change above: that probably has to do
with the fact that $PDFTEXCOM et.al. use ${SOURCE.file} instead of $SOURCE. I
presume there is/was a reason for this (I couldn't find one when searching
through CHANGES.txt)?

And as to why PDF() still fails on the first run with "-j2"... I have no
idea :( .

At least I'm a little bit further than I was before.

There are some other issues I found with the tool that I will work on (to do
with the dsp source scanner and variant dirs with duplicate=False), but I don't
think they have anything to do with this.

Greetings,
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
Url : http://four.pairlist.net/pipermail/scons-users/attachments/20130608/222dff17/attachment.pgp