[Scons-users] SCons popularity
William Blevins
wblevins001 at gmail.com
Sun Jun 14 12:49:55 EDT 2015
On Sun, Jun 14, 2015 at 12:39 PM, Ray Sheppard <rsheppar at iu.edu> wrote:
> Hi William,
> Sorry, I guess it was late. I read a little about your StaticLibrary
> builder and also know that SharedLibrary is a WinDoze only function.
>
Not a WinDoze only function :)
> I should not have picked a specific use case. All I meant was that if the
> code is expecting one set of libraries, often I need to force it to read an
> equivalent library with a different name/link than expected (a naming
> example: think Atlas/LAPACK/MKL/LibSci etc). I am not sure there is an easy
> way a user could ever change link objects. It sure would be nice, but it
> isn't just a SCons issue. I was more soapboxing to the folks that write
> the "shared" restriction into their builds. Sorry.
> About the main subject, I understand that SCons needs to be very
> adaptable to many environments. On top of that, developers might create
> their own environment. What I meant was that to know the values of all the
> variables (for example CCCOM and LIBPATH) which currently set, gives a
> shorthand snapshot of what might be needed to be overridden on the command
> line. "In the old days" this was done by a "dry run." Later, they started
> calling it a "scan." When you were talking about your append and replace
> switches it just struck me how nice to would be to do a dump of them first
> so I could have a quick snapshot of the current state of the world before I
> started monkeying with it.
>
You can indeed dump the environment:
http://www.scons.org/doc/HTML/scons-user/ch28s02.html
> Thanks for considering and responding to my comments.
> Ray
>
>
>
>
> On 6/14/2015 10:42 AM, William Blevins wrote:
>
>
> Ray,
>
> And I think you for your impressions, but it's hard for me to divine the
> desired behavior from the problem behavior so please excuse my excess
> questions.
>
> On Sun, Jun 14, 2015 at 3:10 AM, Ray Sheppard <rsheppar at iu.edu> wrote:
>
>> Hello again William,
>> I am giving you my impressions of the product. I already said I am not
>> a developer. I will address some of your comments. Today's supercomputers
>> seldom permit serial code. All code is parallelized. Usage of 10's of
>> thousands of ranks is not uncommon. The general build directory is a
>> working scratch file system. Installations must put their executables in a
>> different file system. Several systems do not even have home directory
>> access during the execution of a job. While computer scientists seem to
>> love shared libraries, in a modern system, they needlessly light up the
>> network fabric in order to slow down execution. Often, shared expectations
>> (as in *.so) must be accomplished as a a static import from a *.a library.
>>
>
> I'm not sure if I understand the desired behavior for the shared/static
> comments.
>
> 1. Do you mean SCons building shared vs static?
> In theory, we could update the SCons LibraryAPI so something like
> SharedLibrary, StaticLibrary, Library( DEF_LINK_TYPE = static ); thus,
> exposing a default linktype variable, but this would be messy and probably
> wouldn't help much because you cannot dynamically set the LIBS list
> correctly. If the original code was intending to build SharedLibraries
> (IE. a bigger liblist), then switching to static might work, but the
> resulting files may be linking the same static archive several times... it
> would get really messy and would only work depending on the SConscript
> files were created. Too much maybe, could, possibly for me.
>
> 2. Or maybe you are talking about whether SCons chooses to link shared
> libraries over static libraries by default? I would be surprised if their
> isn't a switch for that, though I don't know what it is off the top of my
> head, or maybe its linker specific. If its not linker specific, then we
> could add a behavior switch, so that when variables can be added directly
> to the Default environment, this behavior could be control be controlled.
>
> Do other build systems support what are speaking of?
>
>
>> This creates complicated compiling systems. Using a Cray as an example,
>> the underlying compiler might be gcc/gfortran. However, the actual
>> compiler has a name like ftn. It has a section for putting ftn specific
>> flags (first section of the compile line) as well as native compiler flags
>> (middle section of the compile line). Code which is looking for a clue,
>> like mpicc to decide if it is an MPI code is out of luck because the
>> compilers parse the code and auto link in its version of message passing as
>> well as most common math libraries without specific links. Adding in
>> unnecessary links (like -lmpi) only result in an aborted compilation. All
>> of this requires modifications (from my experience) deep in the SCons
>> specific files. When I use the term "head" I am referring to the
>> framework's ability to divine its surroundings and make decisions. Your
>> framework certainly does try to do that. I have read and even hard copied
>> printed your user guide. Were I creating a project and trying to use SCons
>> to write it, your user guide would be very helpful. If I have a package
>> that thinks it has write permissions to /usr/local and needs to have
>> specific flag structures, it is not so much.
>>
>
> Is the /usr/local permission due to the SCons default install
> directory? SCons is python and python needs to write out compiled
> counterparts like pyc and pyo. You may be able to get around this with
> python specific switches:
> http://stackoverflow.com/questions/154443/how-to-avoid-pyc-files
>
>
>> I do not wish to write my own builder or scanner. I just want to invoke
>> a build with something like:
>> scons all INSTALL_ROOT=/somewhere CC=something CFLAGS= "-x -y -z"
>> FC=something_else FFLAGS="-a -b -c"
>> Reading variables like LD_LIBRARY_PATH and CFLAGS rather than making up a
>> new ones like LIBPATH (or at least aliasing one to another) would be a
>> nice touch. I seem to have missed that section of the UG. Again, this is
>> just the impression of someone who uses your product. You can do with
>> these comments as you wish.
>> Ray
>>
>>
> SCons supports more operating systems than just linux, so all variables
> will "probably" not be all the same for all platforms. If SCons only
> supported linux, then we wouldn't have these complaints.
> Developers need to lookup a mapping from native OS to SCons internal but
> it could be scripted making this a 1-time lookup ideally.
>
>
>> On 6/14/2015 2:18 AM, William Blevins wrote:
>>
>> Ray,
>>
>> I might have been mistaken about something which is rather unfortunate.
>> Apparently, commandline defines don't get placed directly into the
>> DefaultEnvironment. I would say this could be done in 1-2 lines of code,
>> but you will run into the issue of which variable are replaced vs which are
>> appended...
>>
>> Maybe we could come up with a scheme for doing this with all internally
>> supported variables...
>>
>> I'm a bit sad now,
>> William
>>
>> On Sun, Jun 14, 2015 at 1:49 AM, William Blevins <wblevins001 at gmail.com>
>> wrote:
>>
>>> Ray,
>>>
>>> Variable examples are C/C++.
>>>
>>> On Sat, Jun 13, 2015 at 11:50 PM, Ray Sheppard <rsheppar at iu.edu> wrote:
>>>
>>>> Hello William,
>>>> I only have experience with the SCons-based packages handed to me. So
>>>> possibly my issues are with the way those were created and not what is
>>>> possible. In general, they assume that they know what directory structures
>>>> to expect and assume they have write privileges where they don't.
>>>>
>>>
>>> I'm not sure how this is an issue with the tool; might be an issue with
>>> where the code was located when executed, build requirements failure, etc.
>>> I would expect general practice to have file writes down the SConstruct
>>> directory tree, but this is not required.
>>>
>>>
>>>> In addition, they usually "see" a compiler that they should not use and
>>>> grab it.
>>>>
>>>
>>> SCons has to make some assumption or it wouldn't do anything.
>>> Compiler can be overridden via variable like CCCOM.
>>>
>>>
>>>> The fact that it is just a piece of a more complicated compiler with
>>>> multi-staged compiler flags goes right over its head.
>>>>
>>>
>>> SCons doesn't have a head, but I assume you just want to add compiler
>>> flags: CCFLAGS?
>>>
>>>
>>>> In general, executables need to be static because execution directories
>>>> are thousands of clock cycles away from either home directories or system
>>>> libraries.
>>>>
>>>
>>> I'm not sure how this is an issue with the tool; might be an issue
>>> with where the code was located when executed, build requirements failure,
>>> etc. I would expect general practice to have file writes down the
>>> SConstruct directory tree, but this is not required.
>>>
>>>
>>>> I have posed MPI questions here before that went unresolved.
>>>>
>>>
>>> SCons doesn't support clustering across machines natively; not sure
>>> why else you might want messaging passing. There might be a 3-rd party
>>> extension, but I haven't heard of one.
>>>
>>> There is a shared caching mechanism that might be helpful:
>>> http://www.scons.org/doc/HTML/scons-user/ch24.html
>>>
>>>
>>>> I am not a big fan of Auto-xxx/M4/Configure. I was happy setting my
>>>> flags and compilers in a Makefile and letting it go.
>>>>
>>>
>>> Unfortunately, how SCons behaves is up to the build script writer in
>>> many cases, but any environment variable can be overridden on the SCons
>>> command line including common ones like LIBPATH, CPPPATH, CCFLAGS,
>>> compilers, or any other variable. Here is a bigger list of entries that
>>> SCons supports: http://www.scons.org/doc/HTML/scons-user/apa.html
>>>
>>> There are cases where this will not work if the script writer has
>>> explicitly env.Replaced the variable in question thus overriding any
>>> cmdline inputs, but I'm not sure how that can be helped other than via
>>> communicating build requirements.
>>>
>>> We can promote good practices, but we cannot enforced them. I find that
>>> many developers learn a tool by learning just enough to do "anything" and
>>> brute forcing the rest. I have on many occasions scolded my own coworkers
>>> for not trying to lookup the API before they "reinvented the wheel".
>>>
>>> However, a simple set of command line flags to override SCons
>>>> assumptions, similar to ./configure --yyy, is not widely publicized.
>>>>
>>>
>>> The User Guide is very publicized.
>>>
>>> V/R,
>>> William
>>>
>>>
>>>> Ray
>>>>
>>>>
>>>> On 6/13/2015 11:28 PM, William Blevins wrote:
>>>>
>>>>
>>>>
>>>> On Sat, Jun 13, 2015 at 5:14 PM, Ray Sheppard <rsheppar at iu.edu> wrote:
>>>>
>>>>> Hi All,
>>>>> For what it is worth, without this functionality, use of SCons on
>>>>> supercomputers is problematic.
>>>>> Ray
>>>>>
>>>>> On 6/13/2015 10:44 AM, Vasily wrote:
>>>>>
>>>>>>
>>>>>> Yes, I was talking about those. I believe you can override their
>>>>>> default values by passing them as arguments, like
>>>>>> $ scons all INSTALL_ROOT=some-path
>>>>>>
>>>>>> That would need some testing, though - I did not try setting it to a
>>>>>> directory *not* under SConstruct's directory.
>>>>>>
>>>>>> Thanks,
>>>>>> Vasily
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>> It's not clear what you are implying.
>>>>
>>>> SCons supports setting variables on the command line, and SCons
>>>> supports installing outside the SConstruct directory tree, and
>>>> supercomputing has nothing to do with either of these items.
>>>>
>>>> V/R,
>>>> William
>>>>
>>>>
>>>>> _______________________________________________
>>>>> Scons-users mailing list
>>>>> Scons-users at scons.org
>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Scons-users mailing listScons-users at scons.orghttps://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>>>
>>>
>>
>>
>> _______________________________________________
>> Scons-users mailing listScons-users at scons.orghttps://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>
>
> _______________________________________________
> Scons-users mailing listScons-users at scons.orghttps://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20150614/8b7b8b62/attachment-0001.html>
More information about the Scons-users
mailing list