[Scons-users] CudaTool

Wed Jun 17 13:00:52 EDT 2015

David,

So rereading your email.  After the env fix, the linkinfo isn't giving you
any issues or are you still having to workaround it?

V/R,
William

On Wed, Jun 17, 2015 at 11:41 AM, William Blevins <wblevins001 at gmail.com>
wrote:

> David and Bill,
>
> After Bill and I discussed the linkinfo files for a while and I did some
> googling, I came to the conclusion that the linkinfo files shouldn't be
> returned from the emitters because I don't think they are even used:
> https://devtalk.nvidia.com/default/topic/405518/linkinfo-files
>
>
>> def CUDANVCCStaticObjectEmitter(target, source, env):
>>         tgt, src = SCons.Defaults.StaticObjectEmitter(target, source, env)
>>         for file in src:
>>                 lifile = os.path.splitext(src[0].rstr())[0] + '.linkinfo'
>>                 tgt.append(lifile)
>>         return tgt, src
>> def CUDANVCCSharedObjectEmitter(target, source, env):
>>         tgt, src = SCons.Defaults.SharedObjectEmitter(target, source, env)
>>         for file in src:
>>                 lifile = os.path.splitext(src[0].rstr())[0] + '.linkinfo'
>>                 tgt.append(lifile)
>>         return tgt, src
>
>
> For these reasons, I believe we can tweak the Emitters to handle cleaning
> and dependencies correctly without returning the linkinfo files in the
> emitters meaning you won't need to muck with the return values from the
> builders.  This will help keep your SConscripts simple which is always a
> good thing.
>
> David,
>
> Can you try the following?  I don't have a setup for CUDA, but I believe
> this will be a good improvement over the current wiki implementation.
> Update the emitters like so:
>
>
>> def CUDANVCCStaticObjectEmitter(target, source, env):
>>         tgt, src = SCons.Defaults.StaticObjectEmitter(target, source, env)
>>         for file in tgt:
>>                 lifile = os.path.splitext(file.rstr())[0] + '.linkinfo'
>>                 env.SideEffect( lifile, file )
>>                 env.Clean( file, lifile )
>
>         return tgt, src
>>
>
>
> def CUDANVCCSharedObjectEmitter(target, source, env):
>>         tgt, src = SCons.Defaults.SharedObjectEmitter(target, source, env)
>>         for file in tgt:
>>                 lifile = os.path.splitext(file.rstr())[0] + '.linkinfo'
>>                 env.SideEffect( lifile, file )
>>                 env.Clean( file, lifile )
>>         return tgt, src
>
>
> Bill,
>
> Can you double check my patch to make sure I'm not doing anything silly
> like have the Clean/SideEffect parameters in the wrong order?  Also, I am
> pretty sure the previous loop logic didn't behave correctly for multiple
> sources and/or if the targets weren't put into the same directory...
>
> Just some guy wanting to make technology fun,
> William
>
> On Wed, Jun 17, 2015 at 2:41 AM, David Wade <DAWAD at statoil.com> wrote:
>
>>  … and from the land of CET, good morning!
>>
>>
>>
>> Thanks to both of you for looking into this for me. I would indeed have
>> gone to the developer of the SCons CUDA tool first, but unfortunately I
>> couldn’t find who that is! I think if you’re able to figure out who wrote
>> it and ask them to investigate when/if the .linkinfo file is needed in the
>> toolchain that would be great.
>>
>>
>>
>> However, using your suggestions for a better workaround (you’re quite
>> right I made a typo with env1 twice) I’ve been able to revert to the
>> original cuda.py (although for reference I used str(x) rather than x.str()
>> ).
>>
>>
>>
>> And… as a bonus I now have a better idea what SCons is doing in there!
>>
>>
>>
>> Thanks again!
>>
>> David
>>
>>
>>
>> *From:* Scons-users [mailto:scons-users-bounces at scons.org] *On Behalf Of
>> *William Blevins
>> *Sent:* 17 June 2015 01:18
>> *To:* SCons users mailing list
>> *Subject:* Re: [Scons-users] CudaTool
>>
>>
>>
>> Bill,
>>
>>
>>
>> The SideEffect documentation states that sideeffects are not cleaned, and
>> Clean must be called explicitly, so maybe we need to do both.
>>
>>
>>
>> V/R,
>>
>> William
>>
>>
>>
>> On Tue, Jun 16, 2015 at 5:32 PM, William Blevins <wblevins001 at gmail.com>
>> wrote:
>>
>> Bill,
>>
>>
>>
>> Hmm.  Unless I am missing something, the entirety of the Tool exists on
>> the Wiki, so if it doesn't add it there then it isn't used.  I haven't
>> heard of that file extension from any other compiler.
>>
>>
>>
>> David,
>>
>>
>>
>> Can you try making the change Bill and I are discussing.  See if that
>> works for you?   If so, I imagine this will simplify your build scripts a
>> lot.  We can make an update to the SCons wiki also.  Let me know if you
>> need a delta explicitly.
>>
>>
>>
>> V/R,
>>
>> William
>>
>>
>>
>> On Tue, Jun 16, 2015 at 5:03 PM, Bill Deegan <bill at baddogconsulting.com>
>> wrote:
>>
>> Likely the right way for the Cuda tool to handle this is to mark those
>> files as SideEffect(), unless they are somehow an input to other tools..
>>
>> Similar issues happen with env.SharedLibrary() on win32.. You'll get a
>> .dll, .lib and/or .def file so if you want to use the returned nodes you
>> may have to filter them for the one you want.
>>
>> -Bill
>>
>>
>>
>> On Tue, Jun 16, 2015 at 1:57 PM, William Blevins <wblevins001 at gmail.com>
>> wrote:
>>
>> Bill,
>>
>>
>>
>> Or is the SConsWiki the original source, so we are the developers?
>>
>>
>>
>> V/R,
>>
>> William
>>
>>
>>
>> On Tue, Jun 16, 2015 at 4:55 PM, William Blevins <wblevins001 at gmail.com>
>> wrote:
>>
>> David,
>>
>>
>>
>> All in all, it seems that may have been a question more appropriate for
>> the SCons CUDA tool developer, but looking at
>> https://bitbucket.org/scons/scons/wiki/CudaTool notes section, it
>> appears that those files are only added to the Emitter so that scons -c
>> works for those files.
>>
>>
>>
>> Bill,
>>
>>
>>
>> Can the CUDA emitter simply call Clean( 'item.linkinfo' ) in the emitter
>> to explicitly without needing to return the item itself?  I assume this
>> would work, and it would simply that Builder if no one uses the
>> "*.linkinfo" file.  If you think this will, I will contact the developer.
>>
>>
>>
>> V/R,
>>
>> William
>>
>>
>>
>>
>>
>> On Tue, Jun 16, 2015 at 11:00 AM, Bill Deegan <bill at baddogconsulting.com>
>> wrote:
>>
>> William,
>>
>> Thanks for the assist, you are correct in your corrections.
>>
>> Good question about the purpose of the .linkinfo. (They are added in the
>> cuda tool..)
>>
>> I did take a course on cuda programming a while back and I"m thinking
>> these are used to link the native code to code which will run on the GPU
>> hardware.
>>
>> So while they (I believe) aren't needed in the .a they are used by other
>> tools in the tool chain.
>>
>> -Bill
>>
>>
>>
>> On Tue, Jun 16, 2015 at 7:56 AM, William Blevins <wblevins001 at gmail.com>
>> wrote:
>>
>>
>>
>> On Tue, Jun 16, 2015 at 10:29 AM, Bill Deegan <bill at baddogconsulting.com>
>> wrote:
>>
>> David,
>>
>> How about:
>>
>> foo = env.StaticObject(“foo.cu”,<foo-options>)
>> bar = env.StaticObject(“bar.cu”,<bar-options>)
>>
>> foo_objects = [ x for x in foo if x.str().endswith('.o') ]
>> bar_objects = [ x for x in bar if x.str().endswith('.o') ]
>>
>>
>> env.StaticLibrary(libname, [foo, bar]
>>
>>
>>
>> I think the about line needs to be env.StaticLibrary(libname,
>> [foo_objects, bar_objects] though right?
>>
>>
>>
>>  [Also note your example you added bar-options to env1 instead of env2]
>> I didn't try the code above but that's the general way I'd go about it.
>> Remember that all builders return list of nodes.
>>
>> -Bill
>>
>>
>>
>> Thanks Bill.
>>
>>
>>
>> I wasn't 100% sure I understood his question, but even though we have
>> given him a workaround I think we have sidestepped a more important
>> question.
>>
>>
>>
>> What are *.linkinfo used for, and if they aren't being used why are they
>> being created?  Are linkinfo specific to certain toolchains?
>>
>>
>>
>> V/R,
>>
>> William
>>
>>
>>
>>
>>
>> On Tue, Jun 16, 2015 at 5:31 AM, David Wade <DAWAD at statoil.com> wrote:
>>
>>   Hi,
>>
>>
>>
>> I have been using this nifty bit of python for some time:
>> https://bitbucket.org/scons/scons/wiki/CudaTool
>>
>> Just ran into an issue of finding scons was trying to run “ar rc foo.o
>> foo.linkinfo bar.o bar.linkinfo” when I tried this:
>>
>>
>>
>> env1 = env.Clone()
>>
>> env2 = env.Clone()
>>
>>
>>
>> env1.Append(<foo-options>)
>>
>> foo = env1.StaticObject(“foo.cu”)
>>
>> env1.Append(<bar-options>)
>>
>> bar = env2.StaticObject(“bar.cu”)
>>
>>
>>
>> env.StaticLibrary(libname, [foo, bar])
>>
>>
>>
>> My workaround has been to comment out references to the “Emitter”
>> functions in cuda.py … but is this the best thing to do?
>>
>>
>>
>> Many thanks!
>>
>>
>> *David Wade*
>> Senior Analyst, Seismic Imaging Development
>> *ITV SUB RH / HPC Development Team, *Statoil ASA
>>
>> *Phone*: +47 97572157
>> *Email*: *dawad at statoil.com <dawad at statoil.com>*
>> *Visitor address*: Vassbotnen 23, Forus, Norway
>>
>>
>> Incorporation number: NO 923 609 016 MVA
>> *www.statoil.com <http://www.statoil.com/>*
>> Please consider the environment before printing this e-mail.
>>
>>
>>
>>
>>
>> -------------------------------------------------------------------
>> The information contained in this message may be CONFIDENTIAL and is
>> intended for the addressee only. Any unauthorised use, dissemination of
>> the
>> information or copying of this message is prohibited. If you are not the
>> addressee, please notify the sender immediately by return e-mail and
>> delete
>> this message.
>> Thank you
>>
>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>>
>>
>>
>>
>> -------------------------------------------------------------------
>> The information contained in this message may be CONFIDENTIAL and is
>> intended for the addressee only. Any unauthorised use, dissemination of
>> the
>> information or copying of this message is prohibited. If you are not the
>> addressee, please notify the sender immediately by return e-mail and
>> delete
>> this message.
>> Thank you
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20150617/f1b30603/attachment-0001.html>