[Scons-users] Having issue with scanner in Command()
Bill Deegan
bill at baddogconsulting.com
Mon May 14 16:49:51 EDT 2018
Did you try running sconsign on your .sconsign file?
On Mon, May 14, 2018 at 2:05 PM, Jason Kenny <dragon512 at live.com> wrote:
> will be happy to try it out
> -Jason
> ------------------------------
> *From:* Scons-users <scons-users-bounces at scons.org> on behalf of Bill
> Deegan <bill at baddogconsulting.com>
> *Sent:* Monday, May 14, 2018 12:05 PM
> *To:* SCons users mailing list
> *Subject:* Re: [Scons-users] Having issue with scanner in Command()
>
> Jason,
>
> I believe I have a patch for the rebuilds.
> I found a simliar issue working on another problem.
>
> If sconsign crashes reading the .sconsign then you likely have the same
> issue.
> Give it a try and let me know.
> I'm away from computer for a bit, but I'll send you the patch to try.
> Basically the sconsign ends up with mismatched md5's for file names but
> only when ignore is being used because the names aren't filtered, but the
> md5's are.
>
> -Bill
>
>
> On Mon, May 14, 2018 at 12:44 PM, Jason Kenny <dragon512 at live.com> wrote:
>
> Hi Bill
> I was out traveling last week, however, I made some progress on this.
> If I disable the scanner ( changed the code to always return [] ) The
> build worked fine. So the issue is the scanner and how it adds depends to a
> target. I am unclear on why Ignore() is not working as expected. Scons
> wants to keep rebuilding because buildinf.h changed. If I don't have the
> Ignore() I get an error from the taskmaster on the second run with a
> circular depends. I would expect the only Ignore call would prevent rebuilds
> As a workaround, I change the code from using the Parts Pattern object.
> which is basically a recursive Glob() at this point, to just copy the
> openssl source directory to the variant directory. This work as the c
> scanner is global on any C file being defined. By making this a directory
> the copy logic does not define any C file nodes. Since the C files are not
> defined as nodes they are not seen by the scanner to be scanned and added
> as a depends to the target of the Command() that generated the Makefile.
> Is there an official way to turn off the scanner for a given command?? Or
> is this more of a bug with the Ignore() logic as Scons does not seem to
> "fully" ignore the depends added by the scanner.
> either way, I have a workaround. However, it seems this is could be an
> unexpected catch 22 for people that have to call a "third party" build
> system to get a full build to work.
> -Jason
>
> ------------------------------
> *From:* Jason Kenny
> *Sent:* Friday, May 4, 2018 7:29 PM
> *To:* SCons users mailing list
> *Subject:* RE: [Scons-users] Having issue with scanner in Command()
>
>
> In Parts the VCS_DIR is where the code is checked out too. The BUILD_DIR
> is basically the SCons Variant directory for the given “Part”/component. I
> was copying data to this place as OpenSSL does not have an out of source
> build option, and as such would “pollute” the source area.
>
>
>
> I was looking at it more in details. If I build
>
> “Scons _build/build_debug_posix-x86_64/openssl/Makefile”
>
>
>
> Everything is fine SCons says everything is up to date the next round. If
> I build “_makeinstall/openssl/bin/c_rehash” which is defined in the
> second Command() the next pass has this problem. It seems for some reason
> once the generated building.h file is generated something is adding it as a
> depends. I am not sure at the moment if this is seen as implicit or
> explicated by the system. I am assuming it is implicate and being added by
> a scanner. However I have not found what added it as of yet. I don’t think
> it is anything in Parts. I did notice that if I remove the copyed->[] after
> everything is built remove the scons db file everything is fine. I think it
> has to do with my coping object as the source and they are getting scanned
> by the c-scanner but I am unclear how this would be adding it as a depends
> to the target. It has been a while since I have been in this code. I think
> SCons does not make a tree like:
>
>
>
> Foo.exe
>
> + foo.obj
>
> + foo.c
>
> +foo.h
>
>
>
> But makes a
>
>
>
> Foo.exe
>
> + foo.obj
>
> + foo.c
>
> + foo.h
>
>
>
> My theory is the since SCons does the last case. Scons might be adding
> “generated” file of the openssl make build via the scanner at the wrong
> level. Since Scons would add the .h file not as a depends of the .c but to
> the target that depends on the .c file.
>
>
>
> I think I need to do a test run with the C-scanner disabled to verify this
>
>
>
> Jason
>
>
>
> *From:* Scons-users <scons-users-bounces at scons.org> *On Behalf Of *Bill
> Deegan
> *Sent:* Friday, May 4, 2018 3:23 PM
> *To:* SCons users mailing list <scons-users at scons.org>
> *Subject:* Re: [Scons-users] Having issue with scanner in Command()
>
>
>
> What's VCS_DIR and BUILD_DIR ?
>
>
>
> On Fri, May 4, 2018 at 1:18 PM, Jason Kenny <dragon512 at live.com> wrote:
>
> to simplify the log file the tree looks like this:
>
> +-build::alias::openssl
>
> +-build::alias::openssl::SDKBIN
>
> | +-_sdks/debug_posix-x86_64_default/openssl_1.1.0_51bed083228
> 027f76730230d116bace9/bin/c_rehash
>
> | | +-_build/build_debug_posix-x86_64/openssl/_set_RPATH_/c_rehash
>
> | | +-_makeinstall/openssl/bin/c_rehash
>
> | | +-_build/build_debug_posix-x86_64/openssl/Makefile
>
> | | +-_build/build_debug_posix-x86_64/openssl/crypto/ripemd/build.info
> <https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fbuild.info&data=02%7C01%7C%7Cc105254391724c91923b08d5b1fcdf80%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636610621989426130&sdata=1LliS9gWIMdwRhA93U2fKW7HYbJLqp19YyohEjAcZGk%3D&reserved=0>
>
> | | | +-_vcs/openssl/crypto/ripemd/build.info
> <https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fbuild.info&data=02%7C01%7C%7Cc105254391724c91923b08d5b1fcdf80%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636610621989426130&sdata=1LliS9gWIMdwRhA93U2fKW7HYbJLqp19YyohEjAcZGk%3D&reserved=0>
>
> ....
>
> | | +-[_build/build_debug_posix-x86_64/openssl/crypto/LPdir_win.c]
>
> | | +-[_build/build_debug_posix-x86_64/openssl/crypto/arm_arch.h]
>
> | | +-_build/build_debug_posix-x86_64/openssl/crypto/buildinf.h
>
> | | | +-[_build/build_debug_posix-x86_64/openssl/Makefile]
>
> | | +-[_build/build_debug_posix-x86_64/openssl/crypto/LPdir_nyi.c]
>
>
>
> I am not sure why _makeinstall/openssl/bin/c_rehash ->
> _build/build_debug_posix-x86_64/openssl/Makefile ->
> _build/build_debug_posix-x86_64/openssl/crypto/buildinf.h ->
> _build/build_debug_posix-x86_64/openssl/Makefile
>
> I would think c_rehash should depend on the Makefile. I would not think
> Scons would "know" anything about buildinf.h
>
> - Jason
> ------------------------------
>
> *From:* Jason Kenny <dragon512 at live.com>
> *Sent:* Friday, May 4, 2018 11:26 AM
>
>
> *To:* SCons users mailing list
> *Subject:* Re: [Scons-users] Having issue with scanner in Command()
>
>
>
> *copyed* was me coping data to the variant directory to allow cross build
> to work better
>
> log file is here https://bitbucket.org/snippets/dragon512/KeGax5
> <https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbitbucket.org%2Fsnippets%2Fdragon512%2FKeGax5&data=02%7C01%7C%7Cc105254391724c91923b08d5b1fcdf80%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636610621989426130&sdata=%2Fn83xUdN3J5rmYczw5ss%2F9zcjkQnka5BC1nqEw8eLtk%3D&reserved=0>
>
>
> Below is the Parts file I have been hacking on ( I hope to clean it up
> more...) It is based on work I have yet to publish in Parts in my branch.
>
>
> -----
> Import('*')
> PartVersion("1.1.0")
> files = Pattern(src_dir="${VCS_DIR}", excludes=[".git/*"])
>
> copyed = env.CCopy(
> source=files,
> target="$BUILD_DIR",
> copy_logic=CCopy.copy # could hard links be an issue????
> )
> out = env.Command(
> ["Makefile"],
> copyed,
> "cd ${TARGET.dir} ; ./Configure -DSSL_ALLOW_ADH
> enable-ec_nistp_64_gcc_128\
> --prefix=${ABSPATH('#_makeinstall')}/${PART_NAME}\
> --openssldir=${ABSPATH('#_makeinstall')}/${PART_NAME} linux-x86_64
> shared\
> "
> )
>
> # env.Ignore("_build/build_debug_posix-x86_64/openssl/crypto/
> buildinf.h","${VCS_DIR}/crypto/buildinf.h")
> env.Ignore(out, ["crypto/buildinf.h"])
>
>
>
> out = env.Command(
> [
> env.File("crypto/buildinf.h"),
> "#_makeinstall/${PART_NAME}/bin/c_rehash",
> "#_makeinstall/${PART_NAME}/bin/openssl",
> "#_makeinstall/${PART_NAME}/include/openssl/aes.h",
> "#_makeinstall/${PART_NAME}/include/openssl/camellia.h",
> "#_makeinstall/${PART_NAME}/include/openssl/des.h",
> "#_makeinstall/${PART_NAME}/include/openssl/e_os2.h",
> "#_makeinstall/${PART_NAME}/include/openssl/md5.h",
> "#_makeinstall/${PART_NAME}/include/openssl/pem2.h",
> "#_makeinstall/${PART_NAME}/include/openssl/rsa.h",
> "#_makeinstall/${PART_NAME}/include/openssl/stack.h",
> "#_makeinstall/${PART_NAME}/include/openssl/x509_vfy.h",
> "#_makeinstall/${PART_NAME}/include/openssl/asn1.h",
> "#_makeinstall/${PART_NAME}/include/openssl/cast.h",
> "#_makeinstall/${PART_NAME}/include/openssl/dh.h",
> "#_makeinstall/${PART_NAME}/include/openssl/err.h",
> "#_makeinstall/${PART_NAME}/include/openssl/mdc2.h",
> "#_makeinstall/${PART_NAME}/include/openssl/pem.h",
> "#_makeinstall/${PART_NAME}/include/openssl/safestack.h",
> "#_makeinstall/${PART_NAME}/include/openssl/symhacks.h",
> "#_makeinstall/${PART_NAME}/include/openssl/asn1_mac.h",
> "#_makeinstall/${PART_NAME}/include/openssl/cmac.h",
> "#_makeinstall/${PART_NAME}/include/openssl/dsa.h",
> "#_makeinstall/${PART_NAME}/include/openssl/evp.h",
> "#_makeinstall/${PART_NAME}/include/openssl/modes.h",
> "#_makeinstall/${PART_NAME}/include/openssl/pkcs12.h",
> "#_makeinstall/${PART_NAME}/include/openssl/seed.h",
> "#_makeinstall/${PART_NAME}/include/openssl/tls1.h",
> "#_makeinstall/${PART_NAME}/include/openssl/asn1t.h",
> "#_makeinstall/${PART_NAME}/include/openssl/cms.h",
> "#_makeinstall/${PART_NAME}/include/openssl/dtls1.h",
> "#_makeinstall/${PART_NAME}/include/openssl/hmac.h",
> "#_makeinstall/${PART_NAME}/include/openssl/objects.h",
> "#_makeinstall/${PART_NAME}/include/openssl/pkcs7.h",
> "#_makeinstall/${PART_NAME}/include/openssl/sha.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ts.h",
> "#_makeinstall/${PART_NAME}/include/openssl/async.h",
> "#_makeinstall/${PART_NAME}/include/openssl/comp.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ebcdic.h",
> "#_makeinstall/${PART_NAME}/include/openssl/idea.h",
> "#_makeinstall/${PART_NAME}/include/openssl/obj_mac.h",
> "#_makeinstall/${PART_NAME}/include/openssl/rand.h",
> "#_makeinstall/${PART_NAME}/include/openssl/srp.h",
> "#_makeinstall/${PART_NAME}/include/openssl/txt_db.h",
> "#_makeinstall/${PART_NAME}/include/openssl/bio.h",
> "#_makeinstall/${PART_NAME}/include/openssl/conf_api.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ecdh.h",
> "#_makeinstall/${PART_NAME}/include/openssl/kdf.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ocsp.h",
> "#_makeinstall/${PART_NAME}/include/openssl/rc2.h",
> "#_makeinstall/${PART_NAME}/include/openssl/srtp.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ui.h",
> "#_makeinstall/${PART_NAME}/include/openssl/blowfish.h",
> "#_makeinstall/${PART_NAME}/include/openssl/conf.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ecdsa.h",
> "#_makeinstall/${PART_NAME}/include/openssl/lhash.h",
> "#_makeinstall/${PART_NAME}/include/openssl/opensslconf.h",
> "#_makeinstall/${PART_NAME}/include/openssl/rc4.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ssl2.h",
> "#_makeinstall/${PART_NAME}/include/openssl/whrlpool.h",
> "#_makeinstall/${PART_NAME}/include/openssl/bn.h",
> "#_makeinstall/${PART_NAME}/include/openssl/crypto.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ec.h",
> "#_makeinstall/${PART_NAME}/include/openssl/md2.h",
> "#_makeinstall/${PART_NAME}/include/openssl/opensslv.h",
> "#_makeinstall/${PART_NAME}/include/openssl/rc5.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ssl3.h",
> "#_makeinstall/${PART_NAME}/include/openssl/x509.h",
> "#_makeinstall/${PART_NAME}/include/openssl/buffer.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ct.h",
> "#_makeinstall/${PART_NAME}/include/openssl/engine.h",
> "#_makeinstall/${PART_NAME}/include/openssl/md4.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ossl_typ.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ripemd.h",
> "#_makeinstall/${PART_NAME}/include/openssl/ssl.h",
> "#_makeinstall/${PART_NAME}/include/openssl/x509v3.h",
> "#_makeinstall/${PART_NAME}/lib/engines-1.1/afalg.so",
> "#_makeinstall/${PART_NAME}/lib/libssl.a",
> "#_makeinstall/${PART_NAME}/lib/engines-1.1/capi.so",
> "#_makeinstall/${PART_NAME}/lib/engines-1.1/padlock.so",
> "#_makeinstall/${PART_NAME}/lib/libssl.so",
> "#_makeinstall/${PART_NAME}/lib/libssl.so.1.1",
> "#_makeinstall/${PART_NAME}/lib/libcrypto.a",
> "#_makeinstall/${PART_NAME}/lib/pkgconfig/libcrypto.pc",
> "#_makeinstall/${PART_NAME}/lib/pkgconfig/libssl.pc",
> "#_makeinstall/${PART_NAME}/lib/libcrypto.so",
> "#_makeinstall/${PART_NAME}/lib/libcrypto.so.1.1",
> "#_makeinstall/${PART_NAME}/lib/pkgconfig/openssl.pc",
> ],
> ["Makefile"],
> [
> "cd ${{SOURCE.dir}} ; make all CC=${{CC}} CXX=${{CXX}}
> $(-j{jobs}$)".format(jobs=env.GetOption('num_jobs')),
> "cd ${SOURCE.dir} ; make install"
> ]
> )
>
> env.InstallBin(
> env.Glob("#_makeinstall/${PART_NAME}/bin/*")
> )
> env.InstallLib(env.Glob("#_makeinstall/${PART_NAME}/lib/*.a"),
> auto_add_libs=False)
> env.InstallLib(
> env.Glob("#_makeinstall/${PART_NAME}/lib/*.so*")
>
> )
> env.InstallLib(env.Glob("#_makeinstall/${PART_NAME}/libengines-1.1/*.so"),
> auto_add_libs=False, sub_dir="libengines-1.1")
> env.InstallInclude( # bug we need to fix -JK
> env.SdkInclude(
> env.Glob("#_makeinstall/${PART_NAME}/include/openssl/*"),
> sub_dir="openssl",
> add_to_path=False
> ),
> sub_dir="openssl"
> )
> env.InstallLib(env.Glob("#_makeinstall/${PART_NAME}/lib/pkgconfig/*.pc"),
> auto_add_libs=False, sub_dir="pkgconfig", add_to_path=False)
>
>
> ------------------------------
>
> *From:* Scons-users <scons-users-bounces at scons.org> on behalf of Bill
> Deegan <bill at baddogconsulting.com>
> *Sent:* Friday, May 4, 2018 10:40 AM
> *To:* SCons users mailing list
> *Subject:* Re: [Scons-users] Having issue with scanner in Command()
>
>
>
> Jason,
>
>
>
> I"m not aware of a Makefile scanner.
>
>
>
> What's in variable 'copyed'?
>
>
>
> Can you paste output of --tree=prune?
>
> -Bill
>
>
>
> On Fri, May 4, 2018 at 10:31 AM, Jason Kenny <dragon512 at live.com> wrote:
>
> Hi,
>
> I am setting up a build with SCons that has to build openssl. I am using
> the Command() logic to call the native build system. I am having an issue
> with this code wanting to rebuild because of a file "crypto/buildinf.h" is
> rebuilt as part of the process with a new date value/ This causes SCons to
> say stuff needs to be rebuilt. The issue here is an odd circular reference
> that is forming up and I am unclear on what is adding it at this time. I am
> hoping before I dig to deep into this that some one might know, or have
> some idea on what is happening. Details
>
> I have a two commands:
>
>
> out = env.Command(
>
> ["Makefile"],
>
> copyed,
>
> "cd ${TARGET.dir} ; ./Configure -DSSL_ALLOW_ADH
> enable-ec_nistp_64_gcc_128\
>
> --prefix=${ABSPATH('#_makeinstall')}/${PART_NAME}\
>
> --openssldir=${ABSPATH('#_makeinstall')}/${PART_NAME} linux-x86_64
> shared\
>
> "
>
> )
>
>
> Then I have a command
>
> env.Ignore(out, ["crypto/buildinf.h"])
>
>
>
> env.Command(
> [
>
> env.File("crypto/buildinf.h"),
>
> "#_makeinstall/${PART_NAME}/bin/c_rehash",
>
> ...
>
> ],
>
> ["Makefile"],
>
> [
>
> "cd ${{SOURCE.dir}} ; make all CC=${{CC}} CXX=${{CXX}}
> $(-j{jobs}$)".format(jobs=env.GetOption('num_jobs')),
>
> "cd ${SOURCE.dir} ; make install"
>
> ] )
>
>
>
> What is happening is that after the first run Scons will tell me it needs
> to rebuild because:
>
> scons: rebuilding `_build/build_debug_posix-x86_64/openssl/Makefile'
> because `_build/build_debug_posix-x86_64/openssl/crypto/buildinf.h'
> changed
>
> I have an Ignore line that should remove this... What is odd for me is
> that the next build command is run because:
>
> scons: rebuilding `_build/build_debug_posix-x86_64/openssl/crypto/buildinf.h'
> because `_build/build_debug_posix-x86_64/openssl/Makefile' changed
>
>
> I seem to have a cycle of some kind. I am unclear on how to break it.
>
>
> I am unclear on what scanner is adding this depends in Command. Do we have
> a generic Makefile scanner??
> Any thoughts??
>
> Jason
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
> <https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7Ccae88553915f47d8c91a08d5b1d5712f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636610452634906164&sdata=LJ6F2%2BiZCnp1WZOIAfNDmAs4%2F3h3JJNnFtz0GnVCX4g%3D&reserved=0>
>
>
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
> <https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7Cc105254391724c91923b08d5b1fcdf80%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636610621989426130&sdata=6C3axeb9ZEUCd0ra2hdKQ8%2BP5Nr1WGdNl0wPQPgBAwI%3D&reserved=0>
>
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
> <https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7C3d6fa86b3b704e0d3b0408d5b9bcdf81%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636619143209334764&sdata=9XQji%2F6OPpBOB%2Fvk5%2F0KXcpUbuBudRz2trdZxTUM9Ac%3D&reserved=0>
>
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20180514/532fc1d5/attachment-0001.html>
More information about the Scons-users
mailing list