[Scons-users] SCons popularity
Ray Sheppard
rsheppar at iu.edu
Sun Jun 14 12:57:23 EDT 2015
Hi again,
I did not know that "Dump" could be run from the command line. I
should read closer I guess.
Ray
On 6/14/2015 12:49 PM, William Blevins wrote:
>
>
> On Sun, Jun 14, 2015 at 12:39 PM, Ray Sheppard <rsheppar at iu.edu
> <mailto:rsheppar at iu.edu>> wrote:
>
> Hi William,
> Sorry, I guess it was late. I read a little about your
> StaticLibrary builder and also know that SharedLibrary is a
> WinDoze only function.
>
>
> Not a WinDoze only function :)
>
> I should not have picked a specific use case. All I meant was
> that if the code is expecting one set of libraries, often I need
> to force it to read an equivalent library with a different
> name/link than expected (a naming example: think
> Atlas/LAPACK/MKL/LibSci etc). I am not sure there is an easy way a
> user could ever change link objects. It sure would be nice, but
> it isn't just a SCons issue. I was more soapboxing to the folks
> that write the "shared" restriction into their builds. Sorry.
> About the main subject, I understand that SCons needs to be very
> adaptable to many environments. On top of that, developers might
> create their own environment. What I meant was that to know the
> values of all the variables (for example CCCOM and LIBPATH) which
> currently set, gives a shorthand snapshot of what might be needed
> to be overridden on the command line. "In the old days" this was
> done by a "dry run." Later, they started calling it a "scan."
> When you were talking about your append and replace switches it
> just struck me how nice to would be to do a dump of them first so
> I could have a quick snapshot of the current state of the world
> before I started monkeying with it.
>
>
> You can indeed dump the environment:
> http://www.scons.org/doc/HTML/scons-user/ch28s02.html
>
> Thanks for considering and responding to my comments.
> Ray
>
>
>
> On 6/14/2015 10:42 AM, William Blevins wrote:
>>
>> Ray,
>>
>> And I think you for your impressions, but it's hard for me to
>> divine the desired behavior from the problem behavior so please
>> excuse my excess questions.
>>
>> On Sun, Jun 14, 2015 at 3:10 AM, Ray Sheppard <rsheppar at iu.edu
>> <mailto:rsheppar at iu.edu>> wrote:
>>
>> Hello again William,
>> I am giving you my impressions of the product. I already
>> said I am not a developer. I will address some of your
>> comments. Today's supercomputers seldom permit serial code.
>> All code is parallelized. Usage of 10's of thousands of ranks
>> is not uncommon. The general build directory is a working
>> scratch file system. Installations must put their
>> executables in a different file system. Several systems do
>> not even have home directory access during the execution of a
>> job. While computer scientists seem to love shared libraries,
>> in a modern system, they needlessly light up the network
>> fabric in order to slow down execution. Often, shared
>> expectations (as in *.so) must be accomplished as a a static
>> import from a *.a library.
>>
>>
>> I'm not sure if I understand the desired behavior for the
>> shared/static comments.
>>
>> 1. Do you mean SCons building shared vs static?
>> In theory, we could update the SCons LibraryAPI so something like
>> SharedLibrary, StaticLibrary, Library( DEF_LINK_TYPE = static );
>> thus, exposing a default linktype variable, but this would be
>> messy and probably wouldn't help much because you cannot
>> dynamically set the LIBS list correctly. If the original code
>> was intending to build SharedLibraries (IE. a bigger liblist),
>> then switching to static might work, but the resulting files may
>> be linking the same static archive several times... it would get
>> really messy and would only work depending on the SConscript
>> files were created. Too much maybe, could, possibly for me.
>>
>> 2. Or maybe you are talking about whether SCons chooses to link
>> shared libraries over static libraries by default? I would be
>> surprised if their isn't a switch for that, though I don't know
>> what it is off the top of my head, or maybe its linker specific.
>> If its not linker specific, then we could add a behavior switch,
>> so that when variables can be added directly to the Default
>> environment, this behavior could be control be controlled.
>>
>> Do other build systems support what are speaking of?
>>
>> This creates complicated compiling systems. Using a Cray as
>> an example, the underlying compiler might be gcc/gfortran.
>> However, the actual compiler has a name like ftn. It has a
>> section for putting ftn specific flags (first section of the
>> compile line) as well as native compiler flags (middle
>> section of the compile line). Code which is looking for a
>> clue, like mpicc to decide if it is an MPI code is out of
>> luck because the compilers parse the code and auto link in
>> its version of message passing as well as most common math
>> libraries without specific links. Adding in unnecessary links
>> (like -lmpi) only result in an aborted compilation. All of
>> this requires modifications (from my experience) deep in the
>> SCons specific files. When I use the term "head" I am
>> referring to the framework's ability to divine its
>> surroundings and make decisions. Your framework certainly
>> does try to do that. I have read and even hard copied printed
>> your user guide. Were I creating a project and trying to use
>> SCons to write it, your user guide would be very helpful. If
>> I have a package that thinks it has write permissions to
>> /usr/local and needs to have specific flag structures, it is
>> not so much.
>>
>>
>> Is the /usr/local permission due to the SCons default install
>> directory? SCons is python and python needs to write out compiled
>> counterparts like pyc and pyo. You may be able to get around
>> this with python specific switches:
>> http://stackoverflow.com/questions/154443/how-to-avoid-pyc-files
>>
>> I do not wish to write my own builder or scanner. I just
>> want to invoke a build with something like:
>> scons all INSTALL_ROOT=/somewhere CC=something CFLAGS= "-x -y
>> -z" FC=something_else FFLAGS="-a -b -c"
>> Reading variables like LD_LIBRARY_PATH and CFLAGS rather than
>> making up a new ones like LIBPATH (or at least aliasing one
>> to another) would be a nice touch. I seem to have missed that
>> section of the UG. Again, this is just the impression of
>> someone who uses your product. You can do with these
>> comments as you wish.
>> Ray
>>
>>
>> SCons supports more operating systems than just linux, so all
>> variables will "probably" not be all the same for all platforms.
>> If SCons only supported linux, then we wouldn't have these
>> complaints.
>> Developers need to lookup a mapping from native OS to SCons
>> internal but it could be scripted making this a 1-time lookup
>> ideally.
>>
>>
>> On 6/14/2015 2:18 AM, William Blevins wrote:
>>> Ray,
>>>
>>> I might have been mistaken about something which is rather
>>> unfortunate. Apparently, commandline defines don't get
>>> placed directly into the DefaultEnvironment. I would say
>>> this could be done in 1-2 lines of code, but you will run
>>> into the issue of which variable are replaced vs which are
>>> appended...
>>>
>>> Maybe we could come up with a scheme for doing this with all
>>> internally supported variables...
>>>
>>> I'm a bit sad now,
>>> William
>>>
>>> On Sun, Jun 14, 2015 at 1:49 AM, William Blevins
>>> <wblevins001 at gmail.com <mailto:wblevins001 at gmail.com>> wrote:
>>>
>>> Ray,
>>>
>>> Variable examples are C/C++.
>>>
>>> On Sat, Jun 13, 2015 at 11:50 PM, Ray Sheppard
>>> <rsheppar at iu.edu <mailto:rsheppar at iu.edu>> wrote:
>>>
>>> Hello William,
>>> I only have experience with the SCons-based
>>> packages handed to me. So possibly my issues are
>>> with the way those were created and not what is
>>> possible. In general, they assume that they know
>>> what directory structures to expect and assume they
>>> have write privileges where they don't.
>>>
>>>
>>> I'm not sure how this is an issue with the tool; might
>>> be an issue with where the code was located when
>>> executed, build requirements failure, etc. I would
>>> expect general practice to have file writes down the
>>> SConstruct directory tree, but this is not required.
>>>
>>> In addition, they usually "see" a compiler that they
>>> should not use and grab it.
>>>
>>>
>>> SCons has to make some assumption or it wouldn't do
>>> anything. Compiler can be overridden via variable like
>>> CCCOM.
>>>
>>> The fact that it is just a piece of a more
>>> complicated compiler with multi-staged compiler
>>> flags goes right over its head.
>>>
>>>
>>> SCons doesn't have a head, but I assume you just want to
>>> add compiler flags: CCFLAGS?
>>>
>>> In general, executables need to be static because
>>> execution directories are thousands of clock cycles
>>> away from either home directories or system libraries.
>>>
>>>
>>> I'm not sure how this is an issue with the tool; might
>>> be an issue with where the code was located when
>>> executed, build requirements failure, etc. I would
>>> expect general practice to have file writes down the
>>> SConstruct directory tree, but this is not required.
>>>
>>> I have posed MPI questions here before that went
>>> unresolved.
>>>
>>>
>>> SCons doesn't support clustering across machines
>>> natively; not sure why else you might want messaging
>>> passing. There might be a 3-rd party extension, but I
>>> haven't heard of one.
>>>
>>> There is a shared caching mechanism that might be
>>> helpful: http://www.scons.org/doc/HTML/scons-user/ch24.html
>>>
>>> I am not a big fan of Auto-xxx/M4/Configure. I was
>>> happy setting my flags and compilers in a Makefile
>>> and letting it go.
>>>
>>>
>>> Unfortunately, how SCons behaves is up to the build
>>> script writer in many cases, but any environment
>>> variable can be overridden on the SCons command line
>>> including common ones like LIBPATH, CPPPATH, CCFLAGS,
>>> compilers, or any other variable. Here is a bigger list
>>> of entries that SCons supports:
>>> http://www.scons.org/doc/HTML/scons-user/apa.html
>>>
>>> There are cases where this will not work if the script
>>> writer has explicitly env.Replaced the variable in
>>> question thus overriding any cmdline inputs, but I'm not
>>> sure how that can be helped other than via communicating
>>> build requirements.
>>>
>>> We can promote good practices, but we cannot enforced
>>> them. I find that many developers learn a tool by
>>> learning just enough to do "anything" and brute forcing
>>> the rest. I have on many occasions scolded my own
>>> coworkers for not trying to lookup the API before they
>>> "reinvented the wheel".
>>>
>>> However, a simple set of command line flags to
>>> override SCons assumptions, similar to ./configure
>>> --yyy, is not widely publicized.
>>>
>>>
>>> The User Guide is very publicized.
>>>
>>> V/R,
>>> William
>>>
>>> Ray
>>>
>>>
>>> On 6/13/2015 11:28 PM, William Blevins wrote:
>>>>
>>>>
>>>> On Sat, Jun 13, 2015 at 5:14 PM, Ray Sheppard
>>>> <rsheppar at iu.edu <mailto:rsheppar at iu.edu>> wrote:
>>>>
>>>> Hi All,
>>>> For what it is worth, without this
>>>> functionality, use of SCons on supercomputers
>>>> is problematic.
>>>> Ray
>>>>
>>>> On 6/13/2015 10:44 AM, Vasily wrote:
>>>>
>>>>
>>>> Yes, I was talking about those. I believe
>>>> you can override their default values by
>>>> passing them as arguments, like
>>>> $ scons all INSTALL_ROOT=some-path
>>>>
>>>> That would need some testing, though - I
>>>> did not try setting it to a directory *not*
>>>> under SConstruct's directory.
>>>>
>>>> Thanks,
>>>> Vasily
>>>>
>>>>
>>>>
>>>>
>>>> It's not clear what you are implying.
>>>>
>>>> SCons supports setting variables on the command
>>>> line, and SCons supports installing outside the
>>>> SConstruct directory tree, and supercomputing has
>>>> nothing to do with either of these items.
>>>>
>>>> V/R,
>>>> William
>>>>
>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org
>>>> <mailto:Scons-users at scons.org>
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org <mailto:Scons-users at scons.org>
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>>>
>>> _______________________________________________
>>> Scons-users mailing list
>>> Scons-users at scons.org <mailto:Scons-users at scons.org>
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Scons-users mailing list
>>> Scons-users at scons.org <mailto:Scons-users at scons.org>
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org <mailto:Scons-users at scons.org>
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org <mailto:Scons-users at scons.org>
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org <mailto:Scons-users at scons.org>
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20150614/9e126114/attachment-0001.html>
More information about the Scons-users
mailing list