[Scons-users] InstallVersionedLibrary sometimes fails when symlinking

Managan, Rob managan1 at llnl.gov
Mon Jun 24 14:24:02 EDT 2013

Andrew,

I will try to take a look at this week. I should disclose that I know
almost nothing about the Package builder so I will be relying heavily on
your notesŠ.

Thanks for the feedback!

*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-
Rob Managan email managan at llnl.gov
LLNL phone: 925-423-0903
P.O. Box 808, L-095 FAX: 925-422-3389
Livermore, CA 94551-0808





On 6/24/13 11:13 AM, "Andrew C. Morrow" <andrew.c.morrow at gmail.com> wrote:


>Hi Rob -

>

>I found some time to experiment with this again. I've posted my

>example project to:

>https://github.com/acmorrow/scons-versioned-lib-demo

>

>There are a few issues:

>

>1) I don't see that the issue with rebuilding the symlink in the

>install or build directory if it is manually removed is fixed.

>2) The change to _INSTALL_TARGETS.extend needs, I think, to place its

>arguments in brackets (see README and or NOTE 2)

>3) Because InstallVersionedLib doesn't return all of the artifacts (I

>think) there is no way to apply different tags to each artifact (e.g.

>the dev link goes in a -dev package, and the hard file and soname

>symlink go in the main package). See (README or NOTEs 1 and 3).

>

>Also, there appears to be an issue with how Package works with targz,

>since it always encodes the variant directory and packaging directory

>into the tarfiles, instead of capturing only the $PREFIX path. FWIW, I

>also think it would be useful to instruct Package to not include any

>of the $PREFIX, for cases where the project is relocatable in the

>filesystem due to all targets having proper $ORIGIN settings in RPATH,

>etc. Just a thought. Perhaps there is already a way to do this.

>

>Thanks,

>Andrew

>

>

>

>On Tue, Jun 18, 2013 at 11:51 AM, Managan, Rob <managan1 at llnl.gov> wrote:

>> I just got a test added and made a pull request. Your email had me look

>>at

>> this again and I just updated the fork to include a potential fix.

>>

>> Please try it out and is there a way to make a simple test case?

>>

>> *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

>> Rob Managan email managan at llnl.gov

>> LLNL phone: 925-423-0903

>> P.O. Box 808, L-095 FAX: 925-422-3389

>> Livermore, CA 94551-0808

>>

>>

>>

>>

>>

>> On 6/18/13 5:34 AM, "Andrew C. Morrow" <andrew.c.morrow at gmail.com>

>>wrote:

>>

>>>Hi Rob -

>>>

>>>Any update on the dependency tracking issues with

>>>InstallVersionedLibrary?

>>>

>>>Thanks,

>>>Andrew

>>>

>>>

>>>On Mon, Jun 3, 2013 at 11:41 AM, Managan, Rob <managan1 at llnl.gov> wrote:

>>>> Hi Andrew,

>>>>

>>>> Good catch. They probably are not cleaned as well. Let me see what is

>>>> involved to fix that.

>>>>

>>>> Rob

>>>>

>>>> *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

>>>> Rob Managan email managan at llnl.gov

>>>> LLNL phone: 925-423-0903

>>>> P.O. Box 808, L-095 FAX: 925-422-3389

>>>> Livermore, CA 94551-0808

>>>>

>>>>

>>>> On 6/2/13 10:13 AM, "Andrew C. Morrow" <andrew.c.morrow at gmail.com>

>>>>wrote:

>>>>

>>>>

>>>> This is most likely related to the above, but it also appears that the

>>>> installed symlinks are not captured by FindInstalledFiles, which means

>>>>that

>>>> Package (if I'm reading it correctly) won't package the symlinks.

>>>>

>>>> Thanks,

>>>> Andrew

>>>>

>>>>

>>>>

>>>> On Sat, Jun 1, 2013 at 2:09 PM, Andrew C. Morrow

>>>><andrew.c.morrow at gmail.com>

>>>> wrote:

>>>>>

>>>>>

>>>>> Hi Rob -

>>>>>

>>>>> Sorry it took me a couple days to get back to this. I just tried out

>>>>>your

>>>>> branch. The good news is that the issue with the build failing due to

>>>>>errors

>>>>> while symlinking is fixed, so that is a big help.

>>>>>

>>>>> While experimenting with the fix though, I've noticed another issue.

>>>>>Using

>>>>> the same test case as before:

>>>>>

>>>>> $ python $MYSCONS/script/scons.py --prefix=$(pwd)/install install

>>>>> SCons import failed. Trying to run from source directory

>>>>> scons: Reading SConscript files ...

>>>>> scons: done reading SConscript files.

>>>>> scons: Building targets ...

>>>>> Install file: "build/foo.hpp" as "install/include/foo.hpp"

>>>>> g++ -o build/foo.os -c -fPIC build/foo.cpp

>>>>> g++ -o build/libfoo.0.1.2.dylib -dynamiclib -current_version 0.1.2

>>>>> -compatibility_version 0.1.2 -undefined dynamic_lookup build/foo.os

>>>>> Install file: "build/libfoo.0.1.2.dylib" as

>>>>> "install/lib/libfoo.0.1.2.dylib"

>>>>> scons: done building targets.

>>>>>

>>>>> Everything builds OK. Lets remove the symlink in the install

>>>>>directory:

>>>>>

>>>>> $ rm install/lib/libfoo.dylib

>>>>>

>>>>> Rerunning the build claims install is up to date:

>>>>>

>>>>> $ python $MYSCONS/script/scons.py --prefix=$(pwd)/install install

>>>>> SCons import failed. Trying to run from source directory

>>>>> scons: Reading SConscript files ...

>>>>> scons: done reading SConscript files.

>>>>> scons: Building targets ...

>>>>> scons: `install' is up to date.

>>>>> scons: done building targets.

>>>>>

>>>>> And the file was not re-created:

>>>>>

>>>>> $ ls install/lib/libfoo.dylib

>>>>> ls: install/lib/libfoo.dylib: No such file or directory

>>>>>

>>>>> The same effect is observed removing libfoo.dylib from the build

>>>>> directory. So it appears that there are some dependency edges that

>>>>>are

>>>>>not

>>>>> being captured correctly.

>>>>>

>>>>> Thanks,

>>>>> Andrew

>>>>>

>>>>>

>>>>>

>>>>> On Thu, May 30, 2013 at 2:08 PM, Managan, Rob <managan1 at llnl.gov>

>>>>>wrote:

>>>>>>

>>>>>> Andrew,

>>>>>>

>>>>>> I have a proposed fix at

>>>>>>https://bitbucket.org/managan/scons_versionedlib

>>>>>> .

>>>>>> Let me know if it works for you.

>>>>>> I added code to delete any existing symlink before creating them

>>>>>>again.

>>>>>> That was how it failed for me even if the error message seemed to

>>>>>>say

>>>>>> something else. I also made sure the symlinks in the install

>>>>>>directory were

>>>>>> set up the same as in the build directory. Not sure how that ever

>>>>>>was

>>>>>> different.

>>>>>>

>>>>>> I need to add a test case before generating a pull request but this

>>>>>> should be good to go.

>>>>>>

>>>>>> Rob

>>>>>>

>>>>>> *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

>>>>>> Rob Managan email managan at llnl.gov

>>>>>> LLNL phone: 925-423-0903

>>>>>> P.O. Box 808, L-095 FAX: 925-422-3389

>>>>>> Livermore, CA 94551-0808

>>>>>>

>>>>>>

>>>>>> On 5/27/13 10:46 PM, "William Deegan" <bill at baddogconsulting.com>

>>>>>>wrote:

>>>>>>

>>>>>> Andrew,

>>>>>>

>>>>>> On May 27, 2013, at 10:04 PM, Andrew C. Morrow

>>>>>> <andrew.c.morrow at gmail.com> wrote:

>>>>>>

>>>>>>

>>>>>> On Mon, May 27, 2013 at 11:03 PM, William Deegan

>>>>>> <bill at baddogconsulting.com> wrote:

>>>>>>>

>>>>>>> Andrew,

>>>>>>>

>>>>>>> Were you able to repro this? If so, should I file a ticket?

>>>>>>>

>>>>>>>

>>>>>>> Unfortunately, very busy IRL at the moment so haven't had a chance

>>>>>>>to

>>>>>>> try it out.

>>>>>>> Lots of work leading up to a conference, once that's over (6/6)

>>>>>>>I'll

>>>>>>> have some time to take a look at this.

>>>>>>

>>>>>>

>>>>>> Sounds good. I just wanted to make sure it didn't get missed. Built

>>>>>>in

>>>>>> support for versioned shared libraries is a really useful feature

>>>>>>and

>>>>>>I'd

>>>>>> very much like to use it. I'm sure I'm not the only one.

>>>>>>

>>>>>>>

>>>>>>>

>>>>>>>

>>>>>>> Also, is there some documentation on the SCons bugfix and release

>>>>>>>cycle?

>>>>>>> I've noticed that there are only infrequently 'z +1' releases of

>>>>>>>SCons

>>>>>>> x.y.z. If the problem with InstallVersionedLibrary is legitimate,

>>>>>>>should I

>>>>>>> expect to see a fix in a to-be-released-somewhat-soonish SCons

>>>>>>>2.3.1

>>>>>>>bugfix

>>>>>>> release, or longer term in some future SCons 2.4 release?

>>>>>>>

>>>>>>>

>>>>>>>

>>>>>>> SCons is a volunteer run open source project.

>>>>>>> If you want to ensure that a fix is made, you'll either have to

>>>>>>>submit a

>>>>>>> fix, hope that someone in the community picks up the issue and

>>>>>>>fixes

>>>>>>>it, or

>>>>>>> finance a consultant to resolve the issue for you.

>>>>>>

>>>>>>

>>>>>> I'm aware of the nature of the project. Still, this is a feature

>>>>>> documented in the SCons manual pages. Someone loved this feature

>>>>>>enough to

>>>>>> shepherd it through development, code review, testing,

>>>>>>documentation,

>>>>>>and

>>>>>> release.

>>>>>>

>>>>>>

>>>>>> Though all of this is true, that doesn't guarantee that the

>>>>>>developer

>>>>>>of

>>>>>> any feature is immediately available to address a bug.

>>>>>>

>>>>>>

>>>>>>>

>>>>>>> In general we will release a new version either when something

>>>>>>>notable

>>>>>>> has been fixed or if there's been a sufficient number of small

>>>>>>>fixes.

>>>>>>> If a fix is submitted for your issue (assuming it's not user error

>>>>>>> (which from a cursory glance it doesn't look like it is)), then it

>>>>>>>would be

>>>>>>> in the next release.

>>>>>>

>>>>>>

>>>>>> Right, but "next release" at which revision level, and on (roughly)

>>>>>>what

>>>>>> time scale? Without some insight into the release process, intended

>>>>>> schedule, defect triage process, backport policy, etc., it is

>>>>>>difficult to

>>>>>> reason about whether I should abandon my attempts to use this new

>>>>>>feature or

>>>>>> not. Even if I were to debug this particular issue and submit a

>>>>>>patch

>>>>>>as you

>>>>>> have suggested, the adoption of that fix into a published release

>>>>>>may

>>>>>>not

>>>>>> mesh with my own release schedule. I'm not complaining - I merely

>>>>>>want to

>>>>>> better understand: If I or someone else submitted a patch tonight,

>>>>>>would it

>>>>>> likely land in a 2.3.1 release in one month, or in a 2.4.0 in 6

>>>>>>months, or

>>>>>> something else entirely?

>>>>>>

>>>>>>

>>>>>> If this is a bug in an existing feature (and assuming the fix

>>>>>>doesn't

>>>>>> require breaking compatability), then the fix would show in the next

>>>>>>minor

>>>>>> release published after a fix is accepted.

>>>>>> This likely means 2.3.1, however if other less trivial (assuming the

>>>>>>fix

>>>>>> is fairly trivial) changes have already landed in the main branch,

>>>>>>then it

>>>>>> might appear in the 2.4.0 branch.

>>>>>> There is not direct correlation between a 2.3.1 release or a 2.4.0

>>>>>> release and a timeline.

>>>>>>

>>>>>>

>>>>>>

>>>>>>>

>>>>>>> Also keep in mind all the work is done in a public mercurial repo

>>>>>>>hosted

>>>>>>> on bitbucket, so as soon as anyone published a fix you can try it

>>>>>>>against

>>>>>>> your build, and use that code if it resolves your issue.

>>>>>>

>>>>>>

>>>>>> That works well for projects that I plan to build for myself.

>>>>>>However,

>>>>>> for open source projects which use SCons as the publicly facing

>>>>>>build

>>>>>>system

>>>>>> it is not a realistic solution to require downstream consumers to

>>>>>>use

>>>>>>the

>>>>>> SCons mainline or manually patch their SCons installs. Such projects

>>>>>>are

>>>>>> constrained to use the demonstrably working subset of the official

>>>>>>SCons

>>>>>> releases that make their way into distribution and language package

>>>>>> repositories. Potentially, a project could drag in the entire slug

>>>>>>of

>>>>>>fixed

>>>>>> code and redefine the InstallVersionedLibrary, but this has

>>>>>>different

>>>>>> problems.

>>>>>>

>>>>>>

>>>>>> Fair enough.

>>>>>> Note however that SCons can be run in place with a scons-local

>>>>>>package,

>>>>>> so you could unpackage a build scons-local package from the latest

>>>>>>code and

>>>>>> check that into your project until such time as the needed fix is

>>>>>>released.

>>>>>>

>>>>>> Hope that helps.

>>>>>>

>>>>>> In the meantime, it looks like Rob is hot on the trail of this

>>>>>>issue.. so

>>>>>> stay tuned.

>>>>>>

>>>>>> -Bill

>>>>>>

>>>>>> _______________________________________________

>>>>>> Scons-users mailing list

>>>>>> Scons-users at scons.org

>>>>>> http://four.pairlist.net/mailman/listinfo/scons-users

>>>>>>

>>>>>

>>>>

>>>>

>>>> _______________________________________________

>>>> Scons-users mailing list

>>>> Scons-users at scons.org

>>>> http://four.pairlist.net/mailman/listinfo/scons-users

>>>>

>>>_______________________________________________

>>>Scons-users mailing list

>>>Scons-users at scons.org

>>>http://four.pairlist.net/mailman/listinfo/scons-users

>>

>> _______________________________________________

>> Scons-users mailing list

>> Scons-users at scons.org

>> http://four.pairlist.net/mailman/listinfo/scons-users

>_______________________________________________

>Scons-users mailing list

>Scons-users at scons.org

>http://four.pairlist.net/mailman/listinfo/scons-users


More information about the Scons-users mailing list