[Scons-users] Having issue with scanner in Command()
Bill Deegan
bill at baddogconsulting.com
Mon May 14 13:05:02 EDT 2018
Jason,
I believe I have a patch for the rebuilds.
I found a simliar issue working on another problem.
If sconsign crashes reading the .sconsign then you likely have the same
issue.
Give it a try and let me know.
I'm away from computer for a bit, but I'll send you the patch to try.
Basically the sconsign ends up with mismatched md5's for file names but
only when ignore is being used because the names aren't filtered, but the
md5's are.
-Bill
On Mon, May 14, 2018 at 12:44 PM, Jason Kenny <dragon512 at live.com> wrote:
> Hi Bill
> I was out traveling last week, however, I made some progress on this.
> If I disable the scanner ( changed the code to always return [] ) The
> build worked fine. So the issue is the scanner and how it adds depends to a
> target. I am unclear on why Ignore() is not working as expected. Scons
> wants to keep rebuilding because buildinf.h changed. If I don't have the
> Ignore() I get an error from the taskmaster on the second run with a
> circular depends. I would expect the only Ignore call would prevent rebuilds
> As a workaround, I change the code from using the Parts Pattern object.
> which is basically a recursive Glob() at this point, to just copy the
> openssl source directory to the variant directory. This work as the c
> scanner is global on any C file being defined. By making this a directory
> the copy logic does not define any C file nodes. Since the C files are not
> defined as nodes they are not seen by the scanner to be scanned and added
> as a depends to the target of the Command() that generated the Makefile.
> Is there an official way to turn off the scanner for a given command?? Or
> is this more of a bug with the Ignore() logic as Scons does not seem to
> "fully" ignore the depends added by the scanner.
> either way, I have a workaround. However, it seems this is could be an
> unexpected catch 22 for people that have to call a "third party" build
> system to get a full build to work.
> -Jason
>
> ------------------------------
> *From:* Jason Kenny
> *Sent:* Friday, May 4, 2018 7:29 PM
> *To:* SCons users mailing list
> *Subject:* RE: [Scons-users] Having issue with scanner in Command()
>
>
> In Parts the VCS_DIR is where the code is checked out too. The BUILD_DIR
> is basically the SCons Variant directory for the given “Part”/component. I
> was copying data to this place as OpenSSL does not have an out of source
> build option, and as such would “pollute” the source area.
>
>
>
> I was looking at it more in details. If I build
>
> “Scons _build/build_debug_posix-x86_64/openssl/Makefile”
>
>
>
> Everything is fine SCons says everything is up to date the next round. If
> I build “_makeinstall/openssl/bin/c_rehash” which is defined in the
> second Command() the next pass has this problem. It seems for some reason
> once the generated building.h file is generated something is adding it as a
> depends. I am not sure at the moment if this is seen as implicit or
> explicated by the system. I am assuming it is implicate and being added by
> a scanner. However I have not found what added it as of yet. I don’t think
> it is anything in Parts. I did notice that if I remove the copyed->[] after
> everything is built remove the scons db file everything is fine. I think it
> has to do with my coping object as the source and they are getting scanned
> by the c-scanner but I am unclear how this would be adding it as a depends
> to the target. It has been a while since I have been in this code. I think
> SCons does not make a tree like:
>
>
>
> Foo.exe
>
> + foo.obj
>
> + foo.c
>
> +foo.h
>
>
>
> But makes a
>
>
>
> Foo.exe
>
> + foo.obj
>
> + foo.c
>
> + foo.h
>
>
>
> My theory is the since SCons does the last case. Scons might be adding
> “generated” file of the openssl make build via the scanner at the wrong
> level. Since Scons would add the .h file not as a depends of the .c but to
> the target that depends on the .c file.
>
>
>
> I think I need to do a test run with the C-scanner disabled to verify this
>
>
>
> Jason
>
>
>
> *From:* Scons-users <scons-users-bounces at scons.org> *On Behalf Of *Bill
> Deegan
> *Sent:* Friday, May 4, 2018 3:23 PM
> *To:* SCons users mailing list <scons-users at scons.org>
> *Subject:* Re: [Scons-users] Having issue with scanner in Command()
>
>
>
> What's VCS_DIR and BUILD_DIR ?
>
>
>
> On Fri, May 4, 2018 at 1:18 PM, Jason Kenny <dragon512 at live.com> wrote:
>
> to simplify the log file the tree looks like this:
>
> +-build::alias::openssl
>
> +-build::alias::openssl::SDKBIN
>
> | +-_sdks/debug_posix-x86_64_default/openssl_1.1.0_
> 51bed083228027f76730230d116bace9/bin/c_rehash
>
> | | +-_build/build_debug_posix-x86_64/openssl/_set_RPATH_/c_rehash
>
> | | +-_makeinstall/openssl/bin/c_rehash
>
> | | +-_build/build_debug_posix-x86_64/openssl/Makefile
>
> | | +-_build/build_debug_posix-x86_64/openssl/crypto/ripemd/build.info
> <https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fbuild.info&data=02%7C01%7C%7Cc105254391724c91923b08d5b1fcdf80%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636610621989426130&sdata=1LliS9gWIMdwRhA93U2fKW7HYbJLqp19YyohEjAcZGk%3D&reserved=0>
>
> | | | +-_vcs/openssl/crypto/ripemd/build.info
> <https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fbuild.info&data=02%7C01%7C%7Cc105254391724c91923b08d5b1fcdf80%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636610621989426130&sdata=1LliS9gWIMdwRhA93U2fKW7HYbJLqp19YyohEjAcZGk%3D&reserved=0>
>
> ....
>
> | | +-[_build/build_debug_posix-x86_64/openssl/crypto/LPdir_win.c]
>
> | | +-[_build/build_debug_posix-x86_64/openssl/crypto/arm_arch.h]
>
> | | +-_build/build_debug_posix-x86_64/openssl/crypto/buildinf.h
>
> | | | +-[_build/build_debug_posix-x86_64/openssl/Makefile]
>
> | | +-[_build/build_debug_posix-x86_64/openssl/crypto/LPdir_nyi.c]
>
>
>
> I am not sure why _makeinstall/openssl/bin/c_rehash ->
> _build/build_debug_posix-x86_64/openssl/Makefile ->
> _build/build_debug_posix-x86_64/openssl/crypto/buildinf.h ->
> _build/build_debug_posix-x86_64/openssl/Makefile
>
> I would think c_rehash should depend on the Makefile. I would not think
> Scons would "know" anything about buildinf.h
>
> - Jason
> ------------------------------
>
> *From:* Jason Kenny <dragon512 at live.com>
> *Sent:* Friday, May 4, 2018 11:26 AM
>
>
> *To:* SCons users mailing list
> *Subject:* Re: [Scons-users] Having issue with scanner in Command()
>
>
>
> *copyed* was me coping data to the variant directory to allow cross build
> to work better
>
> log file is here https://bitbucket.org/snippets/dragon512/KeGax5
> <https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbitbucket.org%2Fsnippets%2Fdragon512%2FKeGax5&data=02%7C01%7C%7Cc105254391724c91923b08d5b1fcdf80%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636610621989426130&sdata=%2Fn83xUdN3J5rmYczw5ss%2F9zcjkQnka5BC1nqEw8eLtk%3D&reserved=0>
>
>
> Below is the Parts file I have been hacking on ( I hope to clean it up
> more...) It is based on work I have yet to publish in Parts in my branch.
>
>
> -----
> Import('*')
> PartVersion("1.1.0")
> files = Pattern(src_dir="${VCS_DIR}", excludes=[".git/*"])
>
> copyed = env.CCopy(
> source=files,
> target="$BUILD_DIR",
> copy_logic=CCopy.copy # could hard links be an issue????
> )
> out = env.Command(
> ["Makefile"],
> copyed,
> "cd ${TARGET.dir} ; ./Configure -DSSL_ALLOW_ADH
> enable-ec_nistp_64_gcc_128\
> --prefix=${ABSPATH('#_makeinstall')}/${PART_NAME}\
> --openssldir=${ABSPATH('#_makeinstall')}/${PART_NAME} linux-x86_64
> shared\
> "
> )
>
> # env.Ignore("_build/build_debug_posix-x86_64/openssl/
> crypto/buildinf.h","${VCS_DIR}/crypto/buildinf.h")
> env.Ignore(out, ["crypto/buildinf.h"])
>
>
>
> out = env.Command(
> [
> env.File("crypto/buildinf.h"),
> "#_makeinstall/${PART_NAME}/bin/c_rehash",
> "#_makeinstall/${PART_NAME}/bin/openssl",
> "#_makeinstall/${PART_NAME}/include/openssl/aes.h",
> "#_makeinstall/${PART_NAME}/include/openssl/camellia.h",
> "#_makeinstall/${PART_NAME}/include/openssl/des.h",
> "#_makeinstall/${PART_NAME}/include/openssl/e_os2.h",
> "#_makeinstall/${PART_NAME}/include/openssl/md5.h",
> "#_makeinstall/${PART_NAME}/include/openssl/pem2.h",
> "#_makeinstall/${PART_NAME}/include/openssl/rsa.h",
> "#_makeinstall/${PART_NAME}/include/openssl/stack.h",
> "#_makeinstall/${PART_NAME}/include/openssl/x509_vfy.h",
> "#_makeinstall/${PART_NAME}/include/openssl/asn1.h",
> "#_makeinstall/${PART_NAME}/include/openssl/cast.h",
> "#_makeinstall/${PART_NAME}/include/openssl/dh.h",
> "#_makeinstall/${PART_NAME}/include/openssl/err.h",
> "#_makeinstall/${PART_NAME}/include/openssl/mdc2.h",
> "#_makeinstall/${PART_NAME}/include/openssl/pem.h",
> "#_makeinstall/${PART_NAME}/include/openssl/safestack.h",
> "#_makeinstall/${PART_NAME}/include/openssl/symhacks.h",
> "#_makeinstall/${PART_NAME}/include/openssl/asn1_mac.h",
> "#_makeinstall/${PART_NAME}/include/openssl/cmac.h",
> "#_makeinstall/${PART_NAME}/include/openssl/dsa.h",
> "#_makeinstall/${PART_NAME}/include/openssl/evp.h",
> "#_makeinstall/${PART_NAME}/include/openssl/modes.h",
> "#_makeinstall/${PART_NAME}/include/openssl/pkcs12.h",
> "#_makeinstall/${PART_NAME}/include/openssl/seed.h",
> "#_makeinstall/${PART_NAME}/include/openssl/tls1.h",
> "#_makeinstall/${PART_NAME}/include/openssl/asn1t.h",
> "#_makeinstall/${PART_NAME}/include/openssl/cms.h",
> "#_makeinstall/${PART_NAME}/include/openssl/dtls1.h",
> "#_makeinstall/${PART_NAME}/include/openssl/hmac.h",
> "#_makeinstall/${PART_NAME}/include/openssl/objects.h",
> "#_makeinstall/${PART_NAME}/include/openssl/pkcs7.h",
> "#_makeinstall/${PART_NAME}/include/openssl/sha.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ts.h",
> "#_makeinstall/${PART_NAME}/include/openssl/async.h",
> "#_makeinstall/${PART_NAME}/include/openssl/comp.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ebcdic.h",
> "#_makeinstall/${PART_NAME}/include/openssl/idea.h",
> "#_makeinstall/${PART_NAME}/include/openssl/obj_mac.h",
> "#_makeinstall/${PART_NAME}/include/openssl/rand.h",
> "#_makeinstall/${PART_NAME}/include/openssl/srp.h",
> "#_makeinstall/${PART_NAME}/include/openssl/txt_db.h",
> "#_makeinstall/${PART_NAME}/include/openssl/bio.h",
> "#_makeinstall/${PART_NAME}/include/openssl/conf_api.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ecdh.h",
> "#_makeinstall/${PART_NAME}/include/openssl/kdf.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ocsp.h",
> "#_makeinstall/${PART_NAME}/include/openssl/rc2.h",
> "#_makeinstall/${PART_NAME}/include/openssl/srtp.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ui.h",
> "#_makeinstall/${PART_NAME}/include/openssl/blowfish.h",
> "#_makeinstall/${PART_NAME}/include/openssl/conf.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ecdsa.h",
> "#_makeinstall/${PART_NAME}/include/openssl/lhash.h",
> "#_makeinstall/${PART_NAME}/include/openssl/opensslconf.h",
> "#_makeinstall/${PART_NAME}/include/openssl/rc4.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ssl2.h",
> "#_makeinstall/${PART_NAME}/include/openssl/whrlpool.h",
> "#_makeinstall/${PART_NAME}/include/openssl/bn.h",
> "#_makeinstall/${PART_NAME}/include/openssl/crypto.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ec.h",
> "#_makeinstall/${PART_NAME}/include/openssl/md2.h",
> "#_makeinstall/${PART_NAME}/include/openssl/opensslv.h",
> "#_makeinstall/${PART_NAME}/include/openssl/rc5.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ssl3.h",
> "#_makeinstall/${PART_NAME}/include/openssl/x509.h",
> "#_makeinstall/${PART_NAME}/include/openssl/buffer.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ct.h",
> "#_makeinstall/${PART_NAME}/include/openssl/engine.h",
> "#_makeinstall/${PART_NAME}/include/openssl/md4.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ossl_typ.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ripemd.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ssl.h",
> "#_makeinstall/${PART_NAME}/include/openssl/x509v3.h",
> "#_makeinstall/${PART_NAME}/lib/engines-1.1/afalg.so",
> "#_makeinstall/${PART_NAME}/lib/libssl.a",
> "#_makeinstall/${PART_NAME}/lib/engines-1.1/capi.so",
> "#_makeinstall/${PART_NAME}/lib/engines-1.1/padlock.so",
> "#_makeinstall/${PART_NAME}/lib/libssl.so",
> "#_makeinstall/${PART_NAME}/lib/libssl.so.1.1",
> "#_makeinstall/${PART_NAME}/lib/libcrypto.a",
> "#_makeinstall/${PART_NAME}/lib/pkgconfig/libcrypto.pc",
> "#_makeinstall/${PART_NAME}/lib/pkgconfig/libssl.pc",
> "#_makeinstall/${PART_NAME}/lib/libcrypto.so",
> "#_makeinstall/${PART_NAME}/lib/libcrypto.so.1.1",
> "#_makeinstall/${PART_NAME}/lib/pkgconfig/openssl.pc",
> ],
> ["Makefile"],
> [
> "cd ${{SOURCE.dir}} ; make all CC=${{CC}} CXX=${{CXX}}
> $(-j{jobs}$)".format(jobs=env.GetOption('num_jobs')),
> "cd ${SOURCE.dir} ; make install"
> ]
> )
>
> env.InstallBin(
> env.Glob("#_makeinstall/${PART_NAME}/bin/*")
> )
> env.InstallLib(env.Glob("#_makeinstall/${PART_NAME}/lib/*.a"),
> auto_add_libs=False)
> env.InstallLib(
> env.Glob("#_makeinstall/${PART_NAME}/lib/*.so*")
>
> )
> env.InstallLib(env.Glob("#_makeinstall/${PART_NAME}/libengines-1.1/*.so"),
> auto_add_libs=False, sub_dir="libengines-1.1")
> env.InstallInclude( # bug we need to fix -JK
> env.SdkInclude(
> env.Glob("#_makeinstall/${PART_NAME}/include/openssl/*"),
> sub_dir="openssl",
> add_to_path=False
> ),
> sub_dir="openssl"
> )
> env.InstallLib(env.Glob("#_makeinstall/${PART_NAME}/lib/pkgconfig/*.pc"),
> auto_add_libs=False, sub_dir="pkgconfig", add_to_path=False)
>
>
> ------------------------------
>
> *From:* Scons-users <scons-users-bounces at scons.org> on behalf of Bill
> Deegan <bill at baddogconsulting.com>
> *Sent:* Friday, May 4, 2018 10:40 AM
> *To:* SCons users mailing list
> *Subject:* Re: [Scons-users] Having issue with scanner in Command()
>
>
>
> Jason,
>
>
>
> I"m not aware of a Makefile scanner.
>
>
>
> What's in variable 'copyed'?
>
>
>
> Can you paste output of --tree=prune?
>
> -Bill
>
>
>
> On Fri, May 4, 2018 at 10:31 AM, Jason Kenny <dragon512 at live.com> wrote:
>
> Hi,
>
> I am setting up a build with SCons that has to build openssl. I am using
> the Command() logic to call the native build system. I am having an issue
> with this code wanting to rebuild because of a file "crypto/buildinf.h" is
> rebuilt as part of the process with a new date value/ This causes SCons to
> say stuff needs to be rebuilt. The issue here is an odd circular reference
> that is forming up and I am unclear on what is adding it at this time. I am
> hoping before I dig to deep into this that some one might know, or have
> some idea on what is happening. Details
>
> I have a two commands:
>
>
> out = env.Command(
>
> ["Makefile"],
>
> copyed,
>
> "cd ${TARGET.dir} ; ./Configure -DSSL_ALLOW_ADH
> enable-ec_nistp_64_gcc_128\
>
> --prefix=${ABSPATH('#_makeinstall')}/${PART_NAME}\
>
> --openssldir=${ABSPATH('#_makeinstall')}/${PART_NAME} linux-x86_64
> shared\
>
> "
>
> )
>
>
> Then I have a command
>
> env.Ignore(out, ["crypto/buildinf.h"])
>
>
>
> env.Command(
> [
>
> env.File("crypto/buildinf.h"),
>
> "#_makeinstall/${PART_NAME}/bin/c_rehash",
>
> ...
>
> ],
>
> ["Makefile"],
>
> [
>
> "cd ${{SOURCE.dir}} ; make all CC=${{CC}} CXX=${{CXX}}
> $(-j{jobs}$)".format(jobs=env.GetOption('num_jobs')),
>
> "cd ${SOURCE.dir} ; make install"
>
> ] )
>
>
>
> What is happening is that after the first run Scons will tell me it needs
> to rebuild because:
>
> scons: rebuilding `_build/build_debug_posix-x86_64/openssl/Makefile'
> because `_build/build_debug_posix-x86_64/openssl/crypto/buildinf.h'
> changed
>
> I have an Ignore line that should remove this... What is odd for me is
> that the next build command is run because:
>
> scons: rebuilding `_build/build_debug_posix-x86_64/openssl/crypto/buildinf.h'
> because `_build/build_debug_posix-x86_64/openssl/Makefile' changed
>
>
> I seem to have a cycle of some kind. I am unclear on how to break it.
>
>
> I am unclear on what scanner is adding this depends in Command. Do we have
> a generic Makefile scanner??
> Any thoughts??
>
> Jason
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
> <https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7Ccae88553915f47d8c91a08d5b1d5712f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636610452634906164&sdata=LJ6F2%2BiZCnp1WZOIAfNDmAs4%2F3h3JJNnFtz0GnVCX4g%3D&reserved=0>
>
>
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
> <https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7Cc105254391724c91923b08d5b1fcdf80%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636610621989426130&sdata=6C3axeb9ZEUCd0ra2hdKQ8%2BP5Nr1WGdNl0wPQPgBAwI%3D&reserved=0>
>
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20180514/7cd63b86/attachment-0001.html>
More information about the Scons-users
mailing list