[Scons-users] Having issue with scanner in Command()
Jason Kenny
dragon512 at live.com
Fri May 4 20:29:32 EDT 2018
In Parts the VCS_DIR is where the code is checked out too. The BUILD_DIR is basically the SCons Variant directory for the given “Part”/component. I was copying data to this place as OpenSSL does not have an out of source build option, and as such would “pollute” the source area.
I was looking at it more in details. If I build
“Scons _build/build_debug_posix-x86_64/openssl/Makefile”
Everything is fine SCons says everything is up to date the next round. If I build “_makeinstall/openssl/bin/c_rehash” which is defined in the second Command() the next pass has this problem. It seems for some reason once the generated building.h file is generated something is adding it as a depends. I am not sure at the moment if this is seen as implicit or explicated by the system. I am assuming it is implicate and being added by a scanner. However I have not found what added it as of yet. I don’t think it is anything in Parts. I did notice that if I remove the copyed->[] after everything is built remove the scons db file everything is fine. I think it has to do with my coping object as the source and they are getting scanned by the c-scanner but I am unclear how this would be adding it as a depends to the target. It has been a while since I have been in this code. I think SCons does not make a tree like:
Foo.exe
+ foo.obj
+ foo.c
+foo.h
But makes a
Foo.exe
+ foo.obj
+ foo.c
+ foo.h
My theory is the since SCons does the last case. Scons might be adding “generated” file of the openssl make build via the scanner at the wrong level. Since Scons would add the .h file not as a depends of the .c but to the target that depends on the .c file.
I think I need to do a test run with the C-scanner disabled to verify this
Jason
From: Scons-users <scons-users-bounces at scons.org> On Behalf Of Bill Deegan
Sent: Friday, May 4, 2018 3:23 PM
To: SCons users mailing list <scons-users at scons.org>
Subject: Re: [Scons-users] Having issue with scanner in Command()
What's VCS_DIR and BUILD_DIR ?
On Fri, May 4, 2018 at 1:18 PM, Jason Kenny <dragon512 at live.com<mailto:dragon512 at live.com>> wrote:
to simplify the log file the tree looks like this:
+-build::alias::openssl
+-build::alias::openssl::SDKBIN
| +-_sdks/debug_posix-x86_64_default/openssl_1.1.0_51bed083228027f76730230d116bace9/bin/c_rehash
| | +-_build/build_debug_posix-x86_64/openssl/_set_RPATH_/c_rehash
| | +-_makeinstall/openssl/bin/c_rehash
| | +-_build/build_debug_posix-x86_64/openssl/Makefile
| | +-_build/build_debug_posix-x86_64/openssl/crypto/ripemd/build.info<https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fbuild.info&data=02%7C01%7C%7Cc105254391724c91923b08d5b1fcdf80%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636610621989426130&sdata=1LliS9gWIMdwRhA93U2fKW7HYbJLqp19YyohEjAcZGk%3D&reserved=0>
| | | +-_vcs/openssl/crypto/ripemd/build.info<https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fbuild.info&data=02%7C01%7C%7Cc105254391724c91923b08d5b1fcdf80%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636610621989426130&sdata=1LliS9gWIMdwRhA93U2fKW7HYbJLqp19YyohEjAcZGk%3D&reserved=0>
....
| | +-[_build/build_debug_posix-x86_64/openssl/crypto/LPdir_win.c]
| | +-[_build/build_debug_posix-x86_64/openssl/crypto/arm_arch.h]
| | +-_build/build_debug_posix-x86_64/openssl/crypto/buildinf.h
| | | +-[_build/build_debug_posix-x86_64/openssl/Makefile]
| | +-[_build/build_debug_posix-x86_64/openssl/crypto/LPdir_nyi.c]
I am not sure why _makeinstall/openssl/bin/c_rehash -> _build/build_debug_posix-x86_64/openssl/Makefile ->_build/build_debug_posix-x86_64/openssl/crypto/buildinf.h -> _build/build_debug_posix-x86_64/openssl/Makefile
I would think c_rehash should depend on the Makefile. I would not think Scons would "know" anything about buildinf.h
- Jason
________________________________
From: Jason Kenny <dragon512 at live.com<mailto:dragon512 at live.com>>
Sent: Friday, May 4, 2018 11:26 AM
To: SCons users mailing list
Subject: Re: [Scons-users] Having issue with scanner in Command()
copyed was me coping data to the variant directory to allow cross build to work better
log file is here https://bitbucket.org/snippets/dragon512/KeGax5<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbitbucket.org%2Fsnippets%2Fdragon512%2FKeGax5&data=02%7C01%7C%7Cc105254391724c91923b08d5b1fcdf80%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636610621989426130&sdata=%2Fn83xUdN3J5rmYczw5ss%2F9zcjkQnka5BC1nqEw8eLtk%3D&reserved=0>
Below is the Parts file I have been hacking on ( I hope to clean it up more...) It is based on work I have yet to publish in Parts in my branch.
-----
Import('*')
PartVersion("1.1.0")
files = Pattern(src_dir="${VCS_DIR}", excludes=[".git/*"])
copyed = env.CCopy(
source=files,
target="$BUILD_DIR",
copy_logic=CCopy.copy # could hard links be an issue????
)
out = env.Command(
["Makefile"],
copyed,
"cd ${TARGET.dir} ; ./Configure -DSSL_ALLOW_ADH enable-ec_nistp_64_gcc_128\
--prefix=${ABSPATH('#_makeinstall')}/${PART_NAME}\
--openssldir=${ABSPATH('#_makeinstall')}/${PART_NAME} linux-x86_64 shared\
"
)
# env.Ignore("_build/build_debug_posix-x86_64/openssl/crypto/buildinf.h","${VCS_DIR}/crypto/buildinf.h")
env.Ignore(out, ["crypto/buildinf.h"])
out = env.Command(
[
env.File("crypto/buildinf.h"),
"#_makeinstall/${PART_NAME}/bin/c_rehash",
"#_makeinstall/${PART_NAME}/bin/openssl",
"#_makeinstall/${PART_NAME}/include/openssl/aes.h",
"#_makeinstall/${PART_NAME}/include/openssl/camellia.h",
"#_makeinstall/${PART_NAME}/include/openssl/des.h",
"#_makeinstall/${PART_NAME}/include/openssl/e_os2.h",
"#_makeinstall/${PART_NAME}/include/openssl/md5.h",
"#_makeinstall/${PART_NAME}/include/openssl/pem2.h",
"#_makeinstall/${PART_NAME}/include/openssl/rsa.h",
"#_makeinstall/${PART_NAME}/include/openssl/stack.h",
"#_makeinstall/${PART_NAME}/include/openssl/x509_vfy.h",
"#_makeinstall/${PART_NAME}/include/openssl/asn1.h",
"#_makeinstall/${PART_NAME}/include/openssl/cast.h",
"#_makeinstall/${PART_NAME}/include/openssl/dh.h",
"#_makeinstall/${PART_NAME}/include/openssl/err.h",
"#_makeinstall/${PART_NAME}/include/openssl/mdc2.h",
"#_makeinstall/${PART_NAME}/include/openssl/pem.h",
"#_makeinstall/${PART_NAME}/include/openssl/safestack.h",
"#_makeinstall/${PART_NAME}/include/openssl/symhacks.h",
"#_makeinstall/${PART_NAME}/include/openssl/asn1_mac.h",
"#_makeinstall/${PART_NAME}/include/openssl/cmac.h",
"#_makeinstall/${PART_NAME}/include/openssl/dsa.h",
"#_makeinstall/${PART_NAME}/include/openssl/evp.h",
"#_makeinstall/${PART_NAME}/include/openssl/modes.h",
"#_makeinstall/${PART_NAME}/include/openssl/pkcs12.h",
"#_makeinstall/${PART_NAME}/include/openssl/seed.h",
"#_makeinstall/${PART_NAME}/include/openssl/tls1.h",
"#_makeinstall/${PART_NAME}/include/openssl/asn1t.h",
"#_makeinstall/${PART_NAME}/include/openssl/cms.h",
"#_makeinstall/${PART_NAME}/include/openssl/dtls1.h",
"#_makeinstall/${PART_NAME}/include/openssl/hmac.h",
"#_makeinstall/${PART_NAME}/include/openssl/objects.h",
"#_makeinstall/${PART_NAME}/include/openssl/pkcs7.h",
"#_makeinstall/${PART_NAME}/include/openssl/sha.h",
"#_makeinstall/${PART_NAME}/include/openssl/ts.h",
"#_makeinstall/${PART_NAME}/include/openssl/async.h",
"#_makeinstall/${PART_NAME}/include/openssl/comp.h",
"#_makeinstall/${PART_NAME}/include/openssl/ebcdic.h",
"#_makeinstall/${PART_NAME}/include/openssl/idea.h",
"#_makeinstall/${PART_NAME}/include/openssl/obj_mac.h",
"#_makeinstall/${PART_NAME}/include/openssl/rand.h",
"#_makeinstall/${PART_NAME}/include/openssl/srp.h",
"#_makeinstall/${PART_NAME}/include/openssl/txt_db.h",
"#_makeinstall/${PART_NAME}/include/openssl/bio.h",
"#_makeinstall/${PART_NAME}/include/openssl/conf_api.h",
"#_makeinstall/${PART_NAME}/include/openssl/ecdh.h",
"#_makeinstall/${PART_NAME}/include/openssl/kdf.h",
"#_makeinstall/${PART_NAME}/include/openssl/ocsp.h",
"#_makeinstall/${PART_NAME}/include/openssl/rc2.h",
"#_makeinstall/${PART_NAME}/include/openssl/srtp.h",
"#_makeinstall/${PART_NAME}/include/openssl/ui.h",
"#_makeinstall/${PART_NAME}/include/openssl/blowfish.h",
"#_makeinstall/${PART_NAME}/include/openssl/conf.h",
"#_makeinstall/${PART_NAME}/include/openssl/ecdsa.h",
"#_makeinstall/${PART_NAME}/include/openssl/lhash.h",
"#_makeinstall/${PART_NAME}/include/openssl/opensslconf.h",
"#_makeinstall/${PART_NAME}/include/openssl/rc4.h",
"#_makeinstall/${PART_NAME}/include/openssl/ssl2.h",
"#_makeinstall/${PART_NAME}/include/openssl/whrlpool.h",
"#_makeinstall/${PART_NAME}/include/openssl/bn.h",
"#_makeinstall/${PART_NAME}/include/openssl/crypto.h",
"#_makeinstall/${PART_NAME}/include/openssl/ec.h",
"#_makeinstall/${PART_NAME}/include/openssl/md2.h",
"#_makeinstall/${PART_NAME}/include/openssl/opensslv.h",
"#_makeinstall/${PART_NAME}/include/openssl/rc5.h",
"#_makeinstall/${PART_NAME}/include/openssl/ssl3.h",
"#_makeinstall/${PART_NAME}/include/openssl/x509.h",
"#_makeinstall/${PART_NAME}/include/openssl/buffer.h",
"#_makeinstall/${PART_NAME}/include/openssl/ct.h",
"#_makeinstall/${PART_NAME}/include/openssl/engine.h",
"#_makeinstall/${PART_NAME}/include/openssl/md4.h",
"#_makeinstall/${PART_NAME}/include/openssl/ossl_typ.h",
"#_makeinstall/${PART_NAME}/include/openssl/ripemd.h",
"#_makeinstall/${PART_NAME}/include/openssl/ssl.h",
"#_makeinstall/${PART_NAME}/include/openssl/x509v3.h",
"#_makeinstall/${PART_NAME}/lib/engines-1.1/afalg.so",
"#_makeinstall/${PART_NAME}/lib/libssl.a",
"#_makeinstall/${PART_NAME}/lib/engines-1.1/capi.so",
"#_makeinstall/${PART_NAME}/lib/engines-1.1/padlock.so",
"#_makeinstall/${PART_NAME}/lib/libssl.so",
"#_makeinstall/${PART_NAME}/lib/libssl.so.1.1",
"#_makeinstall/${PART_NAME}/lib/libcrypto.a",
"#_makeinstall/${PART_NAME}/lib/pkgconfig/libcrypto.pc",
"#_makeinstall/${PART_NAME}/lib/pkgconfig/libssl.pc",
"#_makeinstall/${PART_NAME}/lib/libcrypto.so",
"#_makeinstall/${PART_NAME}/lib/libcrypto.so.1.1",
"#_makeinstall/${PART_NAME}/lib/pkgconfig/openssl.pc",
],
["Makefile"],
[
"cd ${{SOURCE.dir}} ; make all CC=${{CC}} CXX=${{CXX}} $(-j{jobs}$)".format(jobs=env.GetOption('num_jobs')),
"cd ${SOURCE.dir} ; make install"
]
)
env.InstallBin(
env.Glob("#_makeinstall/${PART_NAME}/bin/*")
)
env.InstallLib(env.Glob("#_makeinstall/${PART_NAME}/lib/*.a"), auto_add_libs=False)
env.InstallLib(
env.Glob("#_makeinstall/${PART_NAME}/lib/*.so*")
)
env.InstallLib(env.Glob("#_makeinstall/${PART_NAME}/libengines-1.1/*.so"), auto_add_libs=False, sub_dir="libengines-1.1")
env.InstallInclude( # bug we need to fix -JK
env.SdkInclude(
env.Glob("#_makeinstall/${PART_NAME}/include/openssl/*"),
sub_dir="openssl",
add_to_path=False
),
sub_dir="openssl"
)
env.InstallLib(env.Glob("#_makeinstall/${PART_NAME}/lib/pkgconfig/*.pc"), auto_add_libs=False, sub_dir="pkgconfig", add_to_path=False)
________________________________
From: Scons-users <scons-users-bounces at scons.org<mailto:scons-users-bounces at scons.org>> on behalf of Bill Deegan <bill at baddogconsulting.com<mailto:bill at baddogconsulting.com>>
Sent: Friday, May 4, 2018 10:40 AM
To: SCons users mailing list
Subject: Re: [Scons-users] Having issue with scanner in Command()
Jason,
I"m not aware of a Makefile scanner.
What's in variable 'copyed'?
Can you paste output of --tree=prune?
-Bill
On Fri, May 4, 2018 at 10:31 AM, Jason Kenny <dragon512 at live.com<mailto:dragon512 at live.com>> wrote:
Hi,
I am setting up a build with SCons that has to build openssl. I am using the Command() logic to call the native build system. I am having an issue with this code wanting to rebuild because of a file "crypto/buildinf.h" is rebuilt as part of the process with a new date value/ This causes SCons to say stuff needs to be rebuilt. The issue here is an odd circular reference that is forming up and I am unclear on what is adding it at this time. I am hoping before I dig to deep into this that some one might know, or have some idea on what is happening. Details
I have a two commands:
out = env.Command(
["Makefile"],
copyed,
"cd ${TARGET.dir} ; ./Configure -DSSL_ALLOW_ADH enable-ec_nistp_64_gcc_128\
--prefix=${ABSPATH('#_makeinstall')}/${PART_NAME}\
--openssldir=${ABSPATH('#_makeinstall')}/${PART_NAME} linux-x86_64 shared\
"
)
Then I have a command
env.Ignore(out, ["crypto/buildinf.h"])
env.Command(
[
env.File("crypto/buildinf.h"),
"#_makeinstall/${PART_NAME}/bin/c_rehash",
...
],
["Makefile"],
[
"cd ${{SOURCE.dir}} ; make all CC=${{CC}} CXX=${{CXX}} $(-j{jobs}$)".format(jobs=env.GetOption('num_jobs')),
"cd ${SOURCE.dir} ; make install"
] )
What is happening is that after the first run Scons will tell me it needs to rebuild because:
scons: rebuilding `_build/build_debug_posix-x86_64/openssl/Makefile' because `_build/build_debug_posix-x86_64/openssl/crypto/buildinf.h' changed
I have an Ignore line that should remove this... What is odd for me is that the next build command is run because:
scons: rebuilding `_build/build_debug_posix-x86_64/openssl/crypto/buildinf.h' because `_build/build_debug_posix-x86_64/openssl/Makefile' changed
I seem to have a cycle of some kind. I am unclear on how to break it.
I am unclear on what scanner is adding this depends in Command. Do we have a generic Makefile scanner??
Any thoughts??
Jason
_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7Ccae88553915f47d8c91a08d5b1d5712f%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636610452634906164&sdata=LJ6F2%2BiZCnp1WZOIAfNDmAs4%2F3h3JJNnFtz0GnVCX4g%3D&reserved=0>
_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpairlist4.pair.net%2Fmailman%2Flistinfo%2Fscons-users&data=02%7C01%7C%7Cc105254391724c91923b08d5b1fcdf80%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636610621989426130&sdata=6C3axeb9ZEUCd0ra2hdKQ8%2BP5Nr1WGdNl0wPQPgBAwI%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20180505/9e288bc6/attachment-0001.html>
More information about the Scons-users
mailing list