[Scons-users] InstallVersionedLibrary sometimes fails when symlinking
Andrew C. Morrow
andrew.c.morrow at gmail.com
Tue Jun 25 15:34:32 EDT 2013
Thanks for taking a look.
The package issues are separate, and it may be that I'm just using it
wrong. I'm still learning how it works, but if there is anyone else on
the list who does have experience with the Package tools, it would be
much appreciated if they could take a look at the repo I posted and
comment on what I'm trying to do.
Thanks,
Andrew
On Mon, Jun 24, 2013 at 2:24 PM, Managan, Rob <managan1 at llnl.gov> wrote:
> Andrew,
>
> I will try to take a look at this week. I should disclose that I know
> almost nothing about the Package builder so I will be relying heavily on
> your notesŠ.
>
> Thanks for the feedback!
>
> *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-
> Rob Managan email managan at llnl.gov
> LLNL phone: 925-423-0903
> P.O. Box 808, L-095 FAX: 925-422-3389
> Livermore, CA 94551-0808
>
>
>
>
>
> On 6/24/13 11:13 AM, "Andrew C. Morrow" <andrew.c.morrow at gmail.com> wrote:
>
>>Hi Rob -
>>
>>I found some time to experiment with this again. I've posted my
>>example project to:
>>https://github.com/acmorrow/scons-versioned-lib-demo
>>
>>There are a few issues:
>>
>>1) I don't see that the issue with rebuilding the symlink in the
>>install or build directory if it is manually removed is fixed.
>>2) The change to _INSTALL_TARGETS.extend needs, I think, to place its
>>arguments in brackets (see README and or NOTE 2)
>>3) Because InstallVersionedLib doesn't return all of the artifacts (I
>>think) there is no way to apply different tags to each artifact (e.g.
>>the dev link goes in a -dev package, and the hard file and soname
>>symlink go in the main package). See (README or NOTEs 1 and 3).
>>
>>Also, there appears to be an issue with how Package works with targz,
>>since it always encodes the variant directory and packaging directory
>>into the tarfiles, instead of capturing only the $PREFIX path. FWIW, I
>>also think it would be useful to instruct Package to not include any
>>of the $PREFIX, for cases where the project is relocatable in the
>>filesystem due to all targets having proper $ORIGIN settings in RPATH,
>>etc. Just a thought. Perhaps there is already a way to do this.
>>
>>Thanks,
>>Andrew
>>
>>
>>
>>On Tue, Jun 18, 2013 at 11:51 AM, Managan, Rob <managan1 at llnl.gov> wrote:
>>> I just got a test added and made a pull request. Your email had me look
>>>at
>>> this again and I just updated the fork to include a potential fix.
>>>
>>> Please try it out and is there a way to make a simple test case?
>>>
>>> *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-
>>> Rob Managan email managan at llnl.gov
>>> LLNL phone: 925-423-0903
>>> P.O. Box 808, L-095 FAX: 925-422-3389
>>> Livermore, CA 94551-0808
>>>
>>>
>>>
>>>
>>>
>>> On 6/18/13 5:34 AM, "Andrew C. Morrow" <andrew.c.morrow at gmail.com>
>>>wrote:
>>>
>>>>Hi Rob -
>>>>
>>>>Any update on the dependency tracking issues with
>>>>InstallVersionedLibrary?
>>>>
>>>>Thanks,
>>>>Andrew
>>>>
>>>>
>>>>On Mon, Jun 3, 2013 at 11:41 AM, Managan, Rob <managan1 at llnl.gov> wrote:
>>>>> Hi Andrew,
>>>>>
>>>>> Good catch. They probably are not cleaned as well. Let me see what is
>>>>> involved to fix that.
>>>>>
>>>>> Rob
>>>>>
>>>>> *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-
>>>>> Rob Managan email managan at llnl.gov
>>>>> LLNL phone: 925-423-0903
>>>>> P.O. Box 808, L-095 FAX: 925-422-3389
>>>>> Livermore, CA 94551-0808
>>>>>
>>>>>
>>>>> On 6/2/13 10:13 AM, "Andrew C. Morrow" <andrew.c.morrow at gmail.com>
>>>>>wrote:
>>>>>
>>>>>
>>>>> This is most likely related to the above, but it also appears that the
>>>>> installed symlinks are not captured by FindInstalledFiles, which means
>>>>>that
>>>>> Package (if I'm reading it correctly) won't package the symlinks.
>>>>>
>>>>> Thanks,
>>>>> Andrew
>>>>>
>>>>>
>>>>>
>>>>> On Sat, Jun 1, 2013 at 2:09 PM, Andrew C. Morrow
>>>>><andrew.c.morrow at gmail.com>
>>>>> wrote:
>>>>>>
>>>>>>
>>>>>> Hi Rob -
>>>>>>
>>>>>> Sorry it took me a couple days to get back to this. I just tried out
>>>>>>your
>>>>>> branch. The good news is that the issue with the build failing due to
>>>>>>errors
>>>>>> while symlinking is fixed, so that is a big help.
>>>>>>
>>>>>> While experimenting with the fix though, I've noticed another issue.
>>>>>>Using
>>>>>> the same test case as before:
>>>>>>
>>>>>> $ python $MYSCONS/script/scons.py --prefix=$(pwd)/install install
>>>>>> SCons import failed. Trying to run from source directory
>>>>>> scons: Reading SConscript files ...
>>>>>> scons: done reading SConscript files.
>>>>>> scons: Building targets ...
>>>>>> Install file: "build/foo.hpp" as "install/include/foo.hpp"
>>>>>> g++ -o build/foo.os -c -fPIC build/foo.cpp
>>>>>> g++ -o build/libfoo.0.1.2.dylib -dynamiclib -current_version 0.1.2
>>>>>> -compatibility_version 0.1.2 -undefined dynamic_lookup build/foo.os
>>>>>> Install file: "build/libfoo.0.1.2.dylib" as
>>>>>> "install/lib/libfoo.0.1.2.dylib"
>>>>>> scons: done building targets.
>>>>>>
>>>>>> Everything builds OK. Lets remove the symlink in the install
>>>>>>directory:
>>>>>>
>>>>>> $ rm install/lib/libfoo.dylib
>>>>>>
>>>>>> Rerunning the build claims install is up to date:
>>>>>>
>>>>>> $ python $MYSCONS/script/scons.py --prefix=$(pwd)/install install
>>>>>> SCons import failed. Trying to run from source directory
>>>>>> scons: Reading SConscript files ...
>>>>>> scons: done reading SConscript files.
>>>>>> scons: Building targets ...
>>>>>> scons: `install' is up to date.
>>>>>> scons: done building targets.
>>>>>>
>>>>>> And the file was not re-created:
>>>>>>
>>>>>> $ ls install/lib/libfoo.dylib
>>>>>> ls: install/lib/libfoo.dylib: No such file or directory
>>>>>>
>>>>>> The same effect is observed removing libfoo.dylib from the build
>>>>>> directory. So it appears that there are some dependency edges that
>>>>>>are
>>>>>>not
>>>>>> being captured correctly.
>>>>>>
>>>>>> Thanks,
>>>>>> Andrew
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, May 30, 2013 at 2:08 PM, Managan, Rob <managan1 at llnl.gov>
>>>>>>wrote:
>>>>>>>
>>>>>>> Andrew,
>>>>>>>
>>>>>>> I have a proposed fix at
>>>>>>>https://bitbucket.org/managan/scons_versionedlib
>>>>>>> .
>>>>>>> Let me know if it works for you.
>>>>>>> I added code to delete any existing symlink before creating them
>>>>>>>again.
>>>>>>> That was how it failed for me even if the error message seemed to
>>>>>>>say
>>>>>>> something else. I also made sure the symlinks in the install
>>>>>>>directory were
>>>>>>> set up the same as in the build directory. Not sure how that ever
>>>>>>>was
>>>>>>> different.
>>>>>>>
>>>>>>> I need to add a test case before generating a pull request but this
>>>>>>> should be good to go.
>>>>>>>
>>>>>>> Rob
>>>>>>>
>>>>>>> *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-
>>>>>>> Rob Managan email managan at llnl.gov
>>>>>>> LLNL phone: 925-423-0903
>>>>>>> P.O. Box 808, L-095 FAX: 925-422-3389
>>>>>>> Livermore, CA 94551-0808
>>>>>>>
>>>>>>>
>>>>>>> On 5/27/13 10:46 PM, "William Deegan" <bill at baddogconsulting.com>
>>>>>>>wrote:
>>>>>>>
>>>>>>> Andrew,
>>>>>>>
>>>>>>> On May 27, 2013, at 10:04 PM, Andrew C. Morrow
>>>>>>> <andrew.c.morrow at gmail.com> wrote:
>>>>>>>
>>>>>>>
>>>>>>> On Mon, May 27, 2013 at 11:03 PM, William Deegan
>>>>>>> <bill at baddogconsulting.com> wrote:
>>>>>>>>
>>>>>>>> Andrew,
>>>>>>>>
>>>>>>>> Were you able to repro this? If so, should I file a ticket?
>>>>>>>>
>>>>>>>>
>>>>>>>> Unfortunately, very busy IRL at the moment so haven't had a chance
>>>>>>>>to
>>>>>>>> try it out.
>>>>>>>> Lots of work leading up to a conference, once that's over (6/6)
>>>>>>>>I'll
>>>>>>>> have some time to take a look at this.
>>>>>>>
>>>>>>>
>>>>>>> Sounds good. I just wanted to make sure it didn't get missed. Built
>>>>>>>in
>>>>>>> support for versioned shared libraries is a really useful feature
>>>>>>>and
>>>>>>>I'd
>>>>>>> very much like to use it. I'm sure I'm not the only one.
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Also, is there some documentation on the SCons bugfix and release
>>>>>>>>cycle?
>>>>>>>> I've noticed that there are only infrequently 'z +1' releases of
>>>>>>>>SCons
>>>>>>>> x.y.z. If the problem with InstallVersionedLibrary is legitimate,
>>>>>>>>should I
>>>>>>>> expect to see a fix in a to-be-released-somewhat-soonish SCons
>>>>>>>>2.3.1
>>>>>>>>bugfix
>>>>>>>> release, or longer term in some future SCons 2.4 release?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> SCons is a volunteer run open source project.
>>>>>>>> If you want to ensure that a fix is made, you'll either have to
>>>>>>>>submit a
>>>>>>>> fix, hope that someone in the community picks up the issue and
>>>>>>>>fixes
>>>>>>>>it, or
>>>>>>>> finance a consultant to resolve the issue for you.
>>>>>>>
>>>>>>>
>>>>>>> I'm aware of the nature of the project. Still, this is a feature
>>>>>>> documented in the SCons manual pages. Someone loved this feature
>>>>>>>enough to
>>>>>>> shepherd it through development, code review, testing,
>>>>>>>documentation,
>>>>>>>and
>>>>>>> release.
>>>>>>>
>>>>>>>
>>>>>>> Though all of this is true, that doesn't guarantee that the
>>>>>>>developer
>>>>>>>of
>>>>>>> any feature is immediately available to address a bug.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> In general we will release a new version either when something
>>>>>>>>notable
>>>>>>>> has been fixed or if there's been a sufficient number of small
>>>>>>>>fixes.
>>>>>>>> If a fix is submitted for your issue (assuming it's not user error
>>>>>>>> (which from a cursory glance it doesn't look like it is)), then it
>>>>>>>>would be
>>>>>>>> in the next release.
>>>>>>>
>>>>>>>
>>>>>>> Right, but "next release" at which revision level, and on (roughly)
>>>>>>>what
>>>>>>> time scale? Without some insight into the release process, intended
>>>>>>> schedule, defect triage process, backport policy, etc., it is
>>>>>>>difficult to
>>>>>>> reason about whether I should abandon my attempts to use this new
>>>>>>>feature or
>>>>>>> not. Even if I were to debug this particular issue and submit a
>>>>>>>patch
>>>>>>>as you
>>>>>>> have suggested, the adoption of that fix into a published release
>>>>>>>may
>>>>>>>not
>>>>>>> mesh with my own release schedule. I'm not complaining - I merely
>>>>>>>want to
>>>>>>> better understand: If I or someone else submitted a patch tonight,
>>>>>>>would it
>>>>>>> likely land in a 2.3.1 release in one month, or in a 2.4.0 in 6
>>>>>>>months, or
>>>>>>> something else entirely?
>>>>>>>
>>>>>>>
>>>>>>> If this is a bug in an existing feature (and assuming the fix
>>>>>>>doesn't
>>>>>>> require breaking compatability), then the fix would show in the next
>>>>>>>minor
>>>>>>> release published after a fix is accepted.
>>>>>>> This likely means 2.3.1, however if other less trivial (assuming the
>>>>>>>fix
>>>>>>> is fairly trivial) changes have already landed in the main branch,
>>>>>>>then it
>>>>>>> might appear in the 2.4.0 branch.
>>>>>>> There is not direct correlation between a 2.3.1 release or a 2.4.0
>>>>>>> release and a timeline.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Also keep in mind all the work is done in a public mercurial repo
>>>>>>>>hosted
>>>>>>>> on bitbucket, so as soon as anyone published a fix you can try it
>>>>>>>>against
>>>>>>>> your build, and use that code if it resolves your issue.
>>>>>>>
>>>>>>>
>>>>>>> That works well for projects that I plan to build for myself.
>>>>>>>However,
>>>>>>> for open source projects which use SCons as the publicly facing
>>>>>>>build
>>>>>>>system
>>>>>>> it is not a realistic solution to require downstream consumers to
>>>>>>>use
>>>>>>>the
>>>>>>> SCons mainline or manually patch their SCons installs. Such projects
>>>>>>>are
>>>>>>> constrained to use the demonstrably working subset of the official
>>>>>>>SCons
>>>>>>> releases that make their way into distribution and language package
>>>>>>> repositories. Potentially, a project could drag in the entire slug
>>>>>>>of
>>>>>>>fixed
>>>>>>> code and redefine the InstallVersionedLibrary, but this has
>>>>>>>different
>>>>>>> problems.
>>>>>>>
>>>>>>>
>>>>>>> Fair enough.
>>>>>>> Note however that SCons can be run in place with a scons-local
>>>>>>>package,
>>>>>>> so you could unpackage a build scons-local package from the latest
>>>>>>>code and
>>>>>>> check that into your project until such time as the needed fix is
>>>>>>>released.
>>>>>>>
>>>>>>> Hope that helps.
>>>>>>>
>>>>>>> In the meantime, it looks like Rob is hot on the trail of this
>>>>>>>issue.. so
>>>>>>> stay tuned.
>>>>>>>
>>>>>>> -Bill
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Scons-users mailing list
>>>>>>> Scons-users at scons.org
>>>>>>> http://four.pairlist.net/mailman/listinfo/scons-users
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Scons-users mailing list
>>>>> Scons-users at scons.org
>>>>> http://four.pairlist.net/mailman/listinfo/scons-users
>>>>>
>>>>_______________________________________________
>>>>Scons-users mailing list
>>>>Scons-users at scons.org
>>>>http://four.pairlist.net/mailman/listinfo/scons-users
>>>
>>> _______________________________________________
>>> Scons-users mailing list
>>> Scons-users at scons.org
>>> http://four.pairlist.net/mailman/listinfo/scons-users
>>_______________________________________________
>>Scons-users mailing list
>>Scons-users at scons.org
>>http://four.pairlist.net/mailman/listinfo/scons-users
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> http://four.pairlist.net/mailman/listinfo/scons-users
More information about the Scons-users
mailing list