[Scons-users] CudaTool
William Blevins
wblevins001 at gmail.com
Wed Jun 17 11:41:45 EDT 2015
David and Bill,
After Bill and I discussed the linkinfo files for a while and I did some
googling, I came to the conclusion that the linkinfo files shouldn't be
returned from the emitters because I don't think they are even used:
https://devtalk.nvidia.com/default/topic/405518/linkinfo-files
> def CUDANVCCStaticObjectEmitter(target, source, env):
> tgt, src = SCons.Defaults.StaticObjectEmitter(target, source, env)
> for file in src:
> lifile = os.path.splitext(src[0].rstr())[0] + '.linkinfo'
> tgt.append(lifile)
> return tgt, src
> def CUDANVCCSharedObjectEmitter(target, source, env):
> tgt, src = SCons.Defaults.SharedObjectEmitter(target, source, env)
> for file in src:
> lifile = os.path.splitext(src[0].rstr())[0] + '.linkinfo'
> tgt.append(lifile)
> return tgt, src
For these reasons, I believe we can tweak the Emitters to handle cleaning
and dependencies correctly without returning the linkinfo files in the
emitters meaning you won't need to muck with the return values from the
builders. This will help keep your SConscripts simple which is always a
good thing.
David,
Can you try the following? I don't have a setup for CUDA, but I believe
this will be a good improvement over the current wiki implementation.
Update the emitters like so:
> def CUDANVCCStaticObjectEmitter(target, source, env):
> tgt, src = SCons.Defaults.StaticObjectEmitter(target, source, env)
> for file in tgt:
> lifile = os.path.splitext(file.rstr())[0] + '.linkinfo'
> env.SideEffect( lifile, file )
> env.Clean( file, lifile )
return tgt, src
>
def CUDANVCCSharedObjectEmitter(target, source, env):
> tgt, src = SCons.Defaults.SharedObjectEmitter(target, source, env)
> for file in tgt:
> lifile = os.path.splitext(file.rstr())[0] + '.linkinfo'
> env.SideEffect( lifile, file )
> env.Clean( file, lifile )
> return tgt, src
Bill,
Can you double check my patch to make sure I'm not doing anything silly
like have the Clean/SideEffect parameters in the wrong order? Also, I am
pretty sure the previous loop logic didn't behave correctly for multiple
sources and/or if the targets weren't put into the same directory...
Just some guy wanting to make technology fun,
William
On Wed, Jun 17, 2015 at 2:41 AM, David Wade <DAWAD at statoil.com> wrote:
> … and from the land of CET, good morning!
>
>
>
> Thanks to both of you for looking into this for me. I would indeed have
> gone to the developer of the SCons CUDA tool first, but unfortunately I
> couldn’t find who that is! I think if you’re able to figure out who wrote
> it and ask them to investigate when/if the .linkinfo file is needed in the
> toolchain that would be great.
>
>
>
> However, using your suggestions for a better workaround (you’re quite
> right I made a typo with env1 twice) I’ve been able to revert to the
> original cuda.py (although for reference I used str(x) rather than x.str()
> ).
>
>
>
> And… as a bonus I now have a better idea what SCons is doing in there!
>
>
>
> Thanks again!
>
> David
>
>
>
> *From:* Scons-users [mailto:scons-users-bounces at scons.org] *On Behalf Of *William
> Blevins
> *Sent:* 17 June 2015 01:18
> *To:* SCons users mailing list
> *Subject:* Re: [Scons-users] CudaTool
>
>
>
> Bill,
>
>
>
> The SideEffect documentation states that sideeffects are not cleaned, and
> Clean must be called explicitly, so maybe we need to do both.
>
>
>
> V/R,
>
> William
>
>
>
> On Tue, Jun 16, 2015 at 5:32 PM, William Blevins <wblevins001 at gmail.com>
> wrote:
>
> Bill,
>
>
>
> Hmm. Unless I am missing something, the entirety of the Tool exists on
> the Wiki, so if it doesn't add it there then it isn't used. I haven't
> heard of that file extension from any other compiler.
>
>
>
> David,
>
>
>
> Can you try making the change Bill and I are discussing. See if that
> works for you? If so, I imagine this will simplify your build scripts a
> lot. We can make an update to the SCons wiki also. Let me know if you
> need a delta explicitly.
>
>
>
> V/R,
>
> William
>
>
>
> On Tue, Jun 16, 2015 at 5:03 PM, Bill Deegan <bill at baddogconsulting.com>
> wrote:
>
> Likely the right way for the Cuda tool to handle this is to mark those
> files as SideEffect(), unless they are somehow an input to other tools..
>
> Similar issues happen with env.SharedLibrary() on win32.. You'll get a
> .dll, .lib and/or .def file so if you want to use the returned nodes you
> may have to filter them for the one you want.
>
> -Bill
>
>
>
> On Tue, Jun 16, 2015 at 1:57 PM, William Blevins <wblevins001 at gmail.com>
> wrote:
>
> Bill,
>
>
>
> Or is the SConsWiki the original source, so we are the developers?
>
>
>
> V/R,
>
> William
>
>
>
> On Tue, Jun 16, 2015 at 4:55 PM, William Blevins <wblevins001 at gmail.com>
> wrote:
>
> David,
>
>
>
> All in all, it seems that may have been a question more appropriate for
> the SCons CUDA tool developer, but looking at
> https://bitbucket.org/scons/scons/wiki/CudaTool notes section, it appears
> that those files are only added to the Emitter so that scons -c works for
> those files.
>
>
>
> Bill,
>
>
>
> Can the CUDA emitter simply call Clean( 'item.linkinfo' ) in the emitter
> to explicitly without needing to return the item itself? I assume this
> would work, and it would simply that Builder if no one uses the
> "*.linkinfo" file. If you think this will, I will contact the developer.
>
>
>
> V/R,
>
> William
>
>
>
>
>
> On Tue, Jun 16, 2015 at 11:00 AM, Bill Deegan <bill at baddogconsulting.com>
> wrote:
>
> William,
>
> Thanks for the assist, you are correct in your corrections.
>
> Good question about the purpose of the .linkinfo. (They are added in the
> cuda tool..)
>
> I did take a course on cuda programming a while back and I"m thinking
> these are used to link the native code to code which will run on the GPU
> hardware.
>
> So while they (I believe) aren't needed in the .a they are used by other
> tools in the tool chain.
>
> -Bill
>
>
>
> On Tue, Jun 16, 2015 at 7:56 AM, William Blevins <wblevins001 at gmail.com>
> wrote:
>
>
>
> On Tue, Jun 16, 2015 at 10:29 AM, Bill Deegan <bill at baddogconsulting.com>
> wrote:
>
> David,
>
> How about:
>
> foo = env.StaticObject(“foo.cu”,<foo-options>)
> bar = env.StaticObject(“bar.cu”,<bar-options>)
>
> foo_objects = [ x for x in foo if x.str().endswith('.o') ]
> bar_objects = [ x for x in bar if x.str().endswith('.o') ]
>
>
> env.StaticLibrary(libname, [foo, bar]
>
>
>
> I think the about line needs to be env.StaticLibrary(libname,
> [foo_objects, bar_objects] though right?
>
>
>
> [Also note your example you added bar-options to env1 instead of env2]
> I didn't try the code above but that's the general way I'd go about it.
> Remember that all builders return list of nodes.
>
> -Bill
>
>
>
> Thanks Bill.
>
>
>
> I wasn't 100% sure I understood his question, but even though we have
> given him a workaround I think we have sidestepped a more important
> question.
>
>
>
> What are *.linkinfo used for, and if they aren't being used why are they
> being created? Are linkinfo specific to certain toolchains?
>
>
>
> V/R,
>
> William
>
>
>
>
>
> On Tue, Jun 16, 2015 at 5:31 AM, David Wade <DAWAD at statoil.com> wrote:
>
> Hi,
>
>
>
> I have been using this nifty bit of python for some time:
> https://bitbucket.org/scons/scons/wiki/CudaTool
>
> Just ran into an issue of finding scons was trying to run “ar rc foo.o
> foo.linkinfo bar.o bar.linkinfo” when I tried this:
>
>
>
> env1 = env.Clone()
>
> env2 = env.Clone()
>
>
>
> env1.Append(<foo-options>)
>
> foo = env1.StaticObject(“foo.cu”)
>
> env1.Append(<bar-options>)
>
> bar = env2.StaticObject(“bar.cu”)
>
>
>
> env.StaticLibrary(libname, [foo, bar])
>
>
>
> My workaround has been to comment out references to the “Emitter”
> functions in cuda.py … but is this the best thing to do?
>
>
>
> Many thanks!
>
>
> *David Wade*
> Senior Analyst, Seismic Imaging Development
> *ITV SUB RH / HPC Development Team, *Statoil ASA
>
> *Phone*: +47 97572157
> *Email*: *dawad at statoil.com <dawad at statoil.com>*
> *Visitor address*: Vassbotnen 23, Forus, Norway
>
>
> Incorporation number: NO 923 609 016 MVA
> *www.statoil.com <http://www.statoil.com/>*
> Please consider the environment before printing this e-mail.
>
>
>
>
>
> -------------------------------------------------------------------
> The information contained in this message may be CONFIDENTIAL and is
> intended for the addressee only. Any unauthorised use, dissemination of the
> information or copying of this message is prohibited. If you are not the
> addressee, please notify the sender immediately by return e-mail and delete
> this message.
> Thank you
>
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
>
>
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
>
>
>
>
> -------------------------------------------------------------------
> The information contained in this message may be CONFIDENTIAL and is
> intended for the addressee only. Any unauthorised use, dissemination of the
> information or copying of this message is prohibited. If you are not the
> addressee, please notify the sender immediately by return e-mail and delete
> this message.
> Thank you
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20150617/93389ab4/attachment-0001.html>
More information about the Scons-users
mailing list