[Scons-users] CudaTool

Thu Jun 18 11:47:27 EDT 2015

Hi William,

There were some other priorities earlier today while your original workaround was holding things together okay…
I’ve now found a moment to look at this, and it works like a charm. SConscript back to being simple and intuitive – thank you!
I would say the new cuda.py is ready to go up on you wiki for everyone else ☺

Best regards,
David

From: Scons-users [mailto:scons-users-bounces at scons.org] On Behalf Of William Blevins
Sent: 17 June 2015 19:01
To: SCons users mailing list
Subject: Re: [Scons-users] CudaTool

David,

So rereading your email.  After the env fix, the linkinfo isn't giving you any issues or are you still having to workaround it?

V/R,
William

On Wed, Jun 17, 2015 at 11:41 AM, William Blevins <wblevins001 at gmail.com<mailto:wblevins001 at gmail.com>> wrote:
David and Bill,

After Bill and I discussed the linkinfo files for a while and I did some googling, I came to the conclusion that the linkinfo files shouldn't be returned from the emitters because I don't think they are even used: https://devtalk.nvidia.com/default/topic/405518/linkinfo-files

def CUDANVCCStaticObjectEmitter(target, source, env):
        tgt, src = SCons.Defaults.StaticObjectEmitter(target, source, env)
        for file in src:
                lifile = os.path.splitext(src[0].rstr())[0] + '.linkinfo'
                tgt.append(lifile)
        return tgt, src
def CUDANVCCSharedObjectEmitter(target, source, env):
        tgt, src = SCons.Defaults.SharedObjectEmitter(target, source, env)
        for file in src:
                lifile = os.path.splitext(src[0].rstr())[0] + '.linkinfo'
                tgt.append(lifile)
        return tgt, src

For these reasons, I believe we can tweak the Emitters to handle cleaning and dependencies correctly without returning the linkinfo files in the emitters meaning you won't need to muck with the return values from the builders.  This will help keep your SConscripts simple which is always a good thing.

David,

Can you try the following?  I don't have a setup for CUDA, but I believe this will be a good improvement over the current wiki implementation.  Update the emitters like so:

def CUDANVCCStaticObjectEmitter(target, source, env):
        tgt, src = SCons.Defaults.StaticObjectEmitter(target, source, env)
        for file in tgt:
                lifile = os.path.splitext(file.rstr())[0] + '.linkinfo'
                env.SideEffect( lifile, file )
                env.Clean( file, lifile )
        return tgt, src

def CUDANVCCSharedObjectEmitter(target, source, env):
        tgt, src = SCons.Defaults.SharedObjectEmitter(target, source, env)
        for file in tgt:
                lifile = os.path.splitext(file.rstr())[0] + '.linkinfo'
                env.SideEffect( lifile, file )
                env.Clean( file, lifile )
        return tgt, src

Bill,

Can you double check my patch to make sure I'm not doing anything silly like have the Clean/SideEffect parameters in the wrong order?  Also, I am pretty sure the previous loop logic didn't behave correctly for multiple sources and/or if the targets weren't put into the same directory...

Just some guy wanting to make technology fun,
William

On Wed, Jun 17, 2015 at 2:41 AM, David Wade <DAWAD at statoil.com<mailto:DAWAD at statoil.com>> wrote:
… and from the land of CET, good morning!

Thanks to both of you for looking into this for me. I would indeed have gone to the developer of the SCons CUDA tool first, but unfortunately I couldn’t find who that is! I think if you’re able to figure out who wrote it and ask them to investigate when/if the .linkinfo file is needed in the toolchain that would be great.

However, using your suggestions for a better workaround (you’re quite right I made a typo with env1 twice) I’ve been able to revert to the original cuda.py (although for reference I used str(x) rather than x.str() ).

And… as a bonus I now have a better idea what SCons is doing in there!

Thanks again!
David

From: Scons-users [mailto:scons-users-bounces at scons.org<mailto:scons-users-bounces at scons.org>] On Behalf Of William Blevins
Sent: 17 June 2015 01:18
To: SCons users mailing list
Subject: Re: [Scons-users] CudaTool

Bill,

The SideEffect documentation states that sideeffects are not cleaned, and Clean must be called explicitly, so maybe we need to do both.

V/R,
William

On Tue, Jun 16, 2015 at 5:32 PM, William Blevins <wblevins001 at gmail.com<mailto:wblevins001 at gmail.com>> wrote:
Bill,

Hmm.  Unless I am missing something, the entirety of the Tool exists on the Wiki, so if it doesn't add it there then it isn't used.  I haven't heard of that file extension from any other compiler.

David,

Can you try making the change Bill and I are discussing.  See if that works for you?   If so, I imagine this will simplify your build scripts a lot.  We can make an update to the SCons wiki also.  Let me know if you need a delta explicitly.

V/R,
William

On Tue, Jun 16, 2015 at 5:03 PM, Bill Deegan <bill at baddogconsulting.com<mailto:bill at baddogconsulting.com>> wrote:
Likely the right way for the Cuda tool to handle this is to mark those files as SideEffect(), unless they are somehow an input to other tools..
Similar issues happen with env.SharedLibrary() on win32.. You'll get a .dll, .lib and/or .def file so if you want to use the returned nodes you may have to filter them for the one you want.
-Bill

On Tue, Jun 16, 2015 at 1:57 PM, William Blevins <wblevins001 at gmail.com<mailto:wblevins001 at gmail.com>> wrote:
Bill,

Or is the SConsWiki the original source, so we are the developers?

V/R,
William

On Tue, Jun 16, 2015 at 4:55 PM, William Blevins <wblevins001 at gmail.com<mailto:wblevins001 at gmail.com>> wrote:
David,

All in all, it seems that may have been a question more appropriate for the SCons CUDA tool developer, but looking at https://bitbucket.org/scons/scons/wiki/CudaTool notes section, it appears that those files are only added to the Emitter so that scons -c works for those files.

Bill,

Can the CUDA emitter simply call Clean( 'item.linkinfo' ) in the emitter to explicitly without needing to return the item itself?  I assume this would work, and it would simply that Builder if no one uses the "*.linkinfo" file.  If you think this will, I will contact the developer.

V/R,
William

On Tue, Jun 16, 2015 at 11:00 AM, Bill Deegan <bill at baddogconsulting.com<mailto:bill at baddogconsulting.com>> wrote:
William,
Thanks for the assist, you are correct in your corrections.
Good question about the purpose of the .linkinfo. (They are added in the cuda tool..)
I did take a course on cuda programming a while back and I"m thinking these are used to link the native code to code which will run on the GPU hardware.
So while they (I believe) aren't needed in the .a they are used by other tools in the tool chain.
-Bill

On Tue, Jun 16, 2015 at 7:56 AM, William Blevins <wblevins001 at gmail.com<mailto:wblevins001 at gmail.com>> wrote:

On Tue, Jun 16, 2015 at 10:29 AM, Bill Deegan <bill at baddogconsulting.com<mailto:bill at baddogconsulting.com>> wrote:
David,

How about:

foo = env.StaticObject(“foo.cu<http://foo.cu>”,<foo-options>)
bar = env.StaticObject(“bar.cu<http://bar.cu>”,<bar-options>)

foo_objects = [ x for x in foo if x.str().endswith('.o') ]
bar_objects = [ x for x in bar if x.str().endswith('.o') ]

env.StaticLibrary(libname, [foo, bar]

I think the about line needs to be env.StaticLibrary(libname, [foo_objects, bar_objects] though right?

[Also note your example you added bar-options to env1 instead of env2]
I didn't try the code above but that's the general way I'd go about it.
Remember that all builders return list of nodes.

-Bill

Thanks Bill.

I wasn't 100% sure I understood his question, but even though we have given him a workaround I think we have sidestepped a more important question.

What are *.linkinfo used for, and if they aren't being used why are they being created?  Are linkinfo specific to certain toolchains?

V/R,
William

On Tue, Jun 16, 2015 at 5:31 AM, David Wade <DAWAD at statoil.com<mailto:DAWAD at statoil.com>> wrote:
Hi,

I have been using this nifty bit of python for some time: https://bitbucket.org/scons/scons/wiki/CudaTool
Just ran into an issue of finding scons was trying to run “ar rc foo.o foo.linkinfo bar.o bar.linkinfo” when I tried this:

env1 = env.Clone()
env2 = env.Clone()

env1.Append(<foo-options>)
foo = env1.StaticObject(“foo.cu<http://foo.cu>”)
env1.Append(<bar-options>)
bar = env2.StaticObject(“bar.cu<http://bar.cu>”)

env.StaticLibrary(libname, [foo, bar])

My workaround has been to comment out references to the “Emitter” functions in cuda.py … but is this the best thing to do?

Many thanks!

David Wade
Senior Analyst, Seismic Imaging Development
ITV SUB RH / HPC Development Team, Statoil ASA

Phone: +47 97572157<tel:%2B47%C2%A097572157>
Email: dawad at statoil.com<mailto:dawad at statoil.com>
Visitor address: Vassbotnen 23, Forus, Norway

Incorporation number: NO 923 609 016 MVA
www.statoil.com<http://www.statoil.com/>
Please consider the environment before printing this e-mail.

-------------------------------------------------------------------
The information contained in this message may be CONFIDENTIAL and is
intended for the addressee only. Any unauthorised use, dissemination of the
information or copying of this message is prohibited. If you are not the
addressee, please notify the sender immediately by return e-mail and delete
this message.
Thank you

_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users

-------------------------------------------------------------------
The information contained in this message may be CONFIDENTIAL and is
intended for the addressee only. Any unauthorised use, dissemination of the
information or copying of this message is prohibited. If you are not the
addressee, please notify the sender immediately by return e-mail and delete
this message.
Thank you

_______________________________________________
Scons-users mailing list
Scons-users at scons.org<mailto:Scons-users at scons.org>
https://pairlist4.pair.net/mailman/listinfo/scons-users

-------------------------------------------------------------------
The information contained in this message may be CONFIDENTIAL and is
intended for the addressee only. Any unauthorised use, dissemination of the
information or copying of this message is prohibited. If you are not the
addressee, please notify the sender immediately by return e-mail and delete
this message.
Thank you
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20150618/efdd85fe/attachment-0001.html>