[Scons-users] Doing a build time Glob()

[Marc Branchaud]

On 13-11-28 04:04 AM, Pico Geyer wrote:

> Hi all.

> 

> I'm trying to build an external project (Python) by calling configure/make

> from scons and then Install() a subset of that projects files into my own

> directory.

> 

> I'm not having trouble with build part of the project, however the Install

> part goes wrong.

> I naively started with something like this:

> env.Install('mydir/python/lib/python2.7',

> Glob('python_install_dir/lib/python2.7/*.py'))

> 

> This of course doesn't work because the Glob() gets run at the SConscript

> reading and at that stage the desired files don't exist yet (My dependencies

> are setup correctly to build python before trying to install the files)

> If I run my install target twice however, the files will exist and then be

> installed correctly.

> It's be very clumsy to tell the users of the build to run the install target

> twice, so I need a better solution.

> 

> I don't suppose there's any built in way to run the Glob at build time, when

> the files will exist?


This is one of my top frustrations when using SCons to run external
(non-SCons) builds.  AFAIK there is no built-in way to do what you ask.

SCons desperately needs to know about every file it deals with:  Either the
file is an original source (that exists in the "clean" project) or SCons
knows how to build the file.

I too have wrangled SCons into building and installing Python.  My case is a
bit different from yours, in that I only install a subset of Python's
modules.  My SConscript file literally lists *each*and*every* file I care
about that Python's "make install DESTDIR=..." creates (including the Python
binaries and other files).  This list forms the targets of the Builder that
runs the make command.  I then Install() those as I like.

I would dearly like for SCons to allow me to say something like "Just copy
whatever files you happen to find in Directory A to Directory B".  But I
think that fundamentally this would require SCons to support dynamic (i.e.
build-time) modifications of its dependency tree.

In your case, if you don't want to list all the files, I see a couple of options:

1. Use AddPostAction() to do a "cp * foo/" after you run the Python build.

2. Write a custom MyInstall() builder that runs "cp * foo/".

Both of those options can be problematic if you want to have something else
depend on what you install.  Even implicit dependencies will get tripped up:
Suppose after building Python you want to build some C code that uses some of
the Python include files.  If you do this, the first time you build Python
and your C code it'll work fine.  But if you then do an incremental rebuild
of your C code SCons will delete the Python .h files your C code uses!  This
is because you haven't told SCons how those include files came to be, and
when SCons sees that your C sources depend on those files it'll delete them
as it's scanning dependencies.

(When building an external library with SCons, my main Builder that runs
"make install" in some way has as its targets the .a and/or .so files, and
I'll use a SideEffect() of that builder to tell SCons about the generated .h
files.)

So if you need something to depend on the Python stuff you install, you have
no choice but to list that stuff explicitly.

(With Python's library of .py files, the installed ones are basically copied
from the Lib/ directory in Python's source tree, so you could use a Glob() of
the *.py files there as the basis of a list of the installed *.py files.
However, the Python build also creates a bunch of .so files it installs under
lib/python2.7/lib-dynload/, and you need some of those to have your
Install()'d Python work at all.  I think these .so are kinda-sorta derived
from the .c files in the Modules/ directory, but I dunno for sure.)

After struggling with this sort of problem for a long time, I've given up
trying to be clever and I always just list everything an external build
command creates.  It's a hassle to maintain the SConscripts as the external
code changes, but with SCons it's the best solution I've found.

		M.