[Scons-users] SCons popularity
Ray Sheppard
rsheppar at iu.edu
Sun Jun 14 03:10:15 EDT 2015
Hello again William,
I am giving you my impressions of the product. I already said I am
not a developer. I will address some of your comments. Today's
supercomputers seldom permit serial code. All code is parallelized.
Usage of 10's of thousands of ranks is not uncommon. The general build
directory is a working scratch file system. Installations must put
their executables in a different file system. Several systems do not
even have home directory access during the execution of a job. While
computer scientists seem to love shared libraries, in a modern system,
they needlessly light up the network fabric in order to slow down
execution. Often, shared expectations (as in *.so) must be accomplished
as a a static import from a *.a library. This creates complicated
compiling systems. Using a Cray as an example, the underlying compiler
might be gcc/gfortran. However, the actual compiler has a name like
ftn. It has a section for putting ftn specific flags (first section of
the compile line) as well as native compiler flags (middle section of
the compile line). Code which is looking for a clue, like mpicc to
decide if it is an MPI code is out of luck because the compilers parse
the code and auto link in its version of message passing as well as most
common math libraries without specific links. Adding in unnecessary
links (like -lmpi) only result in an aborted compilation. All of this
requires modifications (from my experience) deep in the SCons specific
files. When I use the term "head" I am referring to the framework's
ability to divine its surroundings and make decisions. Your framework
certainly does try to do that. I have read and even hard copied printed
your user guide. Were I creating a project and trying to use SCons to
write it, your user guide would be very helpful. If I have a package
that thinks it has write permissions to /usr/local and needs to have
specific flag structures, it is not so much. I do not wish to write my
own builder or scanner. I just want to invoke a build with something like:
scons all INSTALL_ROOT=/somewhere CC=something CFLAGS= "-x -y -z"
FC=something_else FFLAGS="-a -b -c"
Reading variables like LD_LIBRARY_PATH and CFLAGS rather than making up
a new ones like LIBPATH (or at least aliasing one to another) would be
a nice touch. I seem to have missed that section of the UG. Again, this
is just the impression of someone who uses your product. You can do
with these comments as you wish.
Ray
On 6/14/2015 2:18 AM, William Blevins wrote:
> Ray,
>
> I might have been mistaken about something which is rather
> unfortunate. Apparently, commandline defines don't get placed
> directly into the DefaultEnvironment. I would say this could be done
> in 1-2 lines of code, but you will run into the issue of which
> variable are replaced vs which are appended...
>
> Maybe we could come up with a scheme for doing this with all
> internally supported variables...
>
> I'm a bit sad now,
> William
>
> On Sun, Jun 14, 2015 at 1:49 AM, William Blevins
> <wblevins001 at gmail.com <mailto:wblevins001 at gmail.com>> wrote:
>
> Ray,
>
> Variable examples are C/C++.
>
> On Sat, Jun 13, 2015 at 11:50 PM, Ray Sheppard <rsheppar at iu.edu
> <mailto:rsheppar at iu.edu>> wrote:
>
> Hello William,
> I only have experience with the SCons-based packages handed
> to me. So possibly my issues are with the way those were
> created and not what is possible. In general, they assume
> that they know what directory structures to expect and assume
> they have write privileges where they don't.
>
>
> I'm not sure how this is an issue with the tool; might be an issue
> with where the code was located when executed, build requirements
> failure, etc. I would expect general practice to have file writes
> down the SConstruct directory tree, but this is not required.
>
> In addition, they usually "see" a compiler that they should
> not use and grab it.
>
>
> SCons has to make some assumption or it wouldn't do anything.
> Compiler can be overridden via variable like CCCOM.
>
> The fact that it is just a piece of a more complicated
> compiler with multi-staged compiler flags goes right over its
> head.
>
>
> SCons doesn't have a head, but I assume you just want to add
> compiler flags: CCFLAGS?
>
> In general, executables need to be static because execution
> directories are thousands of clock cycles away from either
> home directories or system libraries.
>
>
> I'm not sure how this is an issue with the tool; might be an issue
> with where the code was located when executed, build requirements
> failure, etc. I would expect general practice to have file writes
> down the SConstruct directory tree, but this is not required.
>
> I have posed MPI questions here before that went unresolved.
>
>
> SCons doesn't support clustering across machines natively; not
> sure why else you might want messaging passing. There might be a
> 3-rd party extension, but I haven't heard of one.
>
> There is a shared caching mechanism that might be helpful:
> http://www.scons.org/doc/HTML/scons-user/ch24.html
>
> I am not a big fan of Auto-xxx/M4/Configure. I was happy
> setting my flags and compilers in a Makefile and letting it go.
>
>
> Unfortunately, how SCons behaves is up to the build script writer
> in many cases, but any environment variable can be overridden on
> the SCons command line including common ones like LIBPATH,
> CPPPATH, CCFLAGS, compilers, or any other variable. Here is a
> bigger list of entries that SCons supports:
> http://www.scons.org/doc/HTML/scons-user/apa.html
>
> There are cases where this will not work if the script writer has
> explicitly env.Replaced the variable in question thus overriding
> any cmdline inputs, but I'm not sure how that can be helped other
> than via communicating build requirements.
>
> We can promote good practices, but we cannot enforced them. I
> find that many developers learn a tool by learning just enough to
> do "anything" and brute forcing the rest. I have on many
> occasions scolded my own coworkers for not trying to lookup the
> API before they "reinvented the wheel".
>
> However, a simple set of command line flags to override SCons
> assumptions, similar to ./configure --yyy, is not widely
> publicized.
>
>
> The User Guide is very publicized.
>
> V/R,
> William
>
> Ray
>
>
> On 6/13/2015 11:28 PM, William Blevins wrote:
>>
>>
>> On Sat, Jun 13, 2015 at 5:14 PM, Ray Sheppard
>> <rsheppar at iu.edu <mailto:rsheppar at iu.edu>> wrote:
>>
>> Hi All,
>> For what it is worth, without this functionality, use
>> of SCons on supercomputers is problematic.
>> Ray
>>
>> On 6/13/2015 10:44 AM, Vasily wrote:
>>
>>
>> Yes, I was talking about those. I believe you can
>> override their default values by passing them as
>> arguments, like
>> $ scons all INSTALL_ROOT=some-path
>>
>> That would need some testing, though - I did not try
>> setting it to a directory *not* under SConstruct's
>> directory.
>>
>> Thanks,
>> Vasily
>>
>>
>>
>>
>> It's not clear what you are implying.
>>
>> SCons supports setting variables on the command line, and
>> SCons supports installing outside the SConstruct directory
>> tree, and supercomputing has nothing to do with either of
>> these items.
>>
>> V/R,
>> William
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org <mailto:Scons-users at scons.org>
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org <mailto:Scons-users at scons.org>
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org <mailto:Scons-users at scons.org>
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
>
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20150614/7e97c3d8/attachment-0001.html>
More information about the Scons-users
mailing list