[Scons-users] intermediate builder step -- wish for no-build when the original files are unch

[Jay Damask]

Hello again. 

My two posts with subjects:
sconsign db access -- different results from Dir() and File() apis
sconsign db access -- different results from python cmdline and scons

both related to my intent to better control intermediate builds with 
different tools. The tool for this topic is protoc, but lex / yacc are good 
alternative examples. 

$ uname -a
Linux wdac-nyk-lx01 3.5.0-25-generic #39-Ubuntu SMP Mon Feb 25 18:26:58 UTC 
2013 x86_64 x86_64 x86_64 GNU/Linux
$ scons --version
SCons by Steven Knight et al.:
	script: v2.1.0.r5357[MODIFIED], 2011/09/09 21:31:03, by bdeegan on 
ubuntu
	engine: v2.1.0.r5357[MODIFIED], 2011/09/09 21:31:03, by bdeegan on 
ubuntu

Highlights of my repo --

SConstruct
site_scons/protoc_builder.py
src/SConscript
src/p/
      a.proto
      b.proto
      a.pb.h   <-- after protoc invocation
      a.pb.cc  <-- after protoc invocation
      b.pb.h   <-- after protoc invocation
      b.pb.cc  <-- after protoc invocation


Highlights of my python code:

protoc_builder.py:  (ref: https://github.com/petriborg/Scons-Protoc)

...
def _detect(env):

   detect and return protoc

def _protoc_emitter(target, source, env):

   manipulate target(s) and source(s)

   return target, source

_protoc_builder = SCons.Script.Builder(
        action = SCons.Script.Action('$PROTOC_COM', '$PROTOC_COMSTR'),
        suffix = '$PROTO_CCSUFFIX',
        src_suffix = '$PROTO_SUFFIX',
        emitter = _protoc_emitter,
        )

def generate(env):
    """Add Builders and construction variables."""

    env['PROTOC'] = _detect(env)
    env.SetDefault(
      ...
      PROTOC_COM       = "$PROTOC $PROTOC_FLAGS $SOURCES",
      ...
    )

    env['BUILDERS']['Protoc'] = _protoc_builder

def exists(env):
    return _detect(env)


SConscript:

...
proto_targets_cc = env.Protoc(
    [
        'p/a.proto',
        'p/b.proto'
    ],
    PROTO_PATH  = '#p',
    PROTO_CCOUT = '#p'
    )

commLib = env.SharedLibrary('comm',
    [
        # proto target cc files
        proto_targets_cc,

	# other sources...
    ])

Return('commLib')


Clean first run:
$ scons commLib
invokes:
protoc --proto_path=<foo> --cpp_out=<bar> #/p/a.proto #/p/b.proto
which makes #/p/
a.pb.h, a.pb.cc, b.pb.h, b.pb.cc
invokes:
g++ -o #/b/a.pb.os #/p/a.pb.cc 
g++ -o #/b/b.pb.os #/p/b.pb.cc
g++ <make comm_lib>


Dirty second run:
$ scons commLib
invokes:
protoc --proto_path=<foo> --cpp_out=<bar> a.proto b.proto
which makes #/p/
a.pb.h, a.pb.cc, b.pb.h, b.pb.cc
build completes


The problem is that each dirty scons run after the first invokes protoc, 
which builds *.pb.h and *.pb.cc files. This occurs even when the *.proto 
files are unchanged. Since I have these files in my editor, my editor keeps 
asking me to update the *.pb.* files -- its not only annoying but the scons 
build is inefficient. 

I have read about Decider objects, they appear to be global and set the 
signature (md5, ts, etc). 

I have read about the Depends fcxn, but its n/a b/c I want the 
proto_targets_cc files compiled into the comm_lib, not just that comm_lib 
depends on the proto_targets_cc. 

I have read about the Requires fcxn, but imo its n/a, too. 

Since we are in python its pretty easy to get the md5sum of the source file 
and the persisted md5sum from the sconsign db. If there's a diff then invoke 
protoc, otherwise skip the protoc step. 

In fact, with the code layout in protoc_builder.py I don't see an 'obvious' 
way to skip the protoc invocation. The way I intended to kludge it is to 
replace the $PROTOC_COM string the env dict with ''. Its a hack, if there's 
better form I'd like to know. 

What I'm addressing is a broader pattern, and I appreciate its touched on in 
the scons dox. For instance there's CFile and CXXFile builders... Lex and 
Yacc are excellent similar examples where I don't want the .cc files 
generated when the .l and .y files are unchanged. 

Any help with how to layout this code, and in particular cease the protoc 
invocation when all *.proto files are unch, is much appreciated.

Thank you and regards,
-Jay