[Scons-users] Scons on pypy

Mon Jun 6 11:32:05 EDT 2016

I assume you mean that we can no do a “pip install -e .” to get a developer build working, have them remove the -e to get a full install. 

I always tell people to use pip now if we can do it, then fallback to “setup install” when they are on some system where there seems to be an issue getting pip install easily. 

Do you have better advice on how that should be done?

Jason

From: Scons-users [mailto:scons-users-bounces at scons.org] On Behalf Of Daniel Holth
Sent: Monday, June 6, 2016 10:03 AM
To: SCons users mailing list <scons-users at scons.org>; dl9obn at darc.de
Subject: Re: [Scons-users] Scons on pypy

Of note is that modern versions of pip do not need 'setup.py install'. Instead, they always build a wheel first, then install that.

On Mon, Jun 6, 2016 at 10:58 AM Daniel Holth <dholth at gmail.com <mailto:dholth at gmail.com> > wrote:

I don't have it yet. I would be happy to trade information about the necessary commands and Python packaging metadata for SCons advice :)

List of what pip invokes on setup.py https://pip.pypa.io/en/stable/reference/pip_install/#build-system-interface

Here's an implementation of setup-requires (install requirements before running the bulk of setup.py). https://bitbucket.org/dholth/setup-requires Eventually a future version of pip will do this for you, and this will become a no-op shim.

Here's an experimental waf script that builds a wheel for bdist_wheel itself, in a few lines of code. https://bitbucket.org/dholth/wheel/src/tip/wscript

Wheel also has code to convert between old and new metadata formats.

I haven't measured the impact of __slots__ on PyPy 5.1 very much except that it prevents it from working at all. You would have to work with the PyPy developers on that front. 'local-zip' uses more time and more memory on PyPy, but there may be better SCons benchmarks.

On Mon, Jun 6, 2016 at 10:18 AM Jason Kenny <dragon512 at live.com <mailto:dragon512 at live.com> > wrote:

I was hoping to clean up the setup.py file myself. I had hoped to make it use whl files. I had hoped to test it with pypy myself.

I was hoping to make it simpler like the one I define for a project I am working on here: https://bitbucket.org/dragon512/reusable-gold-testing-system/src/23db072a0d7469e777f8921ca6e7580894587da4/setup.py?fileviewer=file-view-default 

However we seem to have the expert here :) Do you have a better setup.py already, or should I still try to give it a shot and have you give it a review?

Also on the __slots__ issue with pypy. I had defined a test to show the need for the node changes and posted it here some place. The use of __slots__ or __slabs__ in pypy case shows values. I did the test with ironpython, cpython 2 and 3 and pypy. All showed better use of memory. This is critical for Scons to scale better for large build. As a side effect of use less memory for the same information we get a speed boost as well, and for a build system that is good, as user always complain that the “build is slow”.

Jason

From: Scons-users [mailto:scons-users-bounces at scons.org <mailto:scons-users-bounces at scons.org> ] On Behalf Of Daniel Holth
Sent: Sunday, June 5, 2016 1:00 PM
To: dl9obn at darc.de <mailto:dl9obn at darc.de> ; SCons users mailing list <scons-users at scons.org <mailto:scons-users at scons.org> >
Subject: Re: [Scons-users] Scons on pypy

I am the guy who invented wheel (*.whl) for Python packaging, and I am interested in advancing Python packaging generally. I feel that a setuptools monoculture holds us back generally, it hails from the late 90's (distutils) and very few people understand how to extend it. SCons seems like a nice way to experiment with a more flexible build system that is written in Python and easily installed from pypi.

Soon, pip will be able to install dependencies *before* it touches setup.py. See https://github.com/brettcannon/build-deps-pep/blob/master/pep-0NNN.rst. Pip is also working on defining an interface (consisting of a few setup.py commands) that it promises to use when invoking setup.py. That makes it much easier to bring abstractions to setup.py and re-implement just the parts that pip needs. So for any project that runs against the limitations of distutils there should and can be other options.

As it happens I also have a concrete project that does benefit from SCons. The pysdl2-cffi binding to SDL2 is not particularly exceptional, but it needs to automatically generate some *.py code to help wrap the SDL2 library before "setup.py bdist_wheel" can be called. It does this by parsing SDL2's headers with a little compiler that generates a friendlier wrapper compared to using cffi raw. It also calls cffi to generate *.c and compiles that as an extension module (here, it is especially helpful to be running under the target Python, to ask distutils for compiler options). SCons is great at this because there are several build steps that depend on each other's outputs. It would be difficult to do that efficiently with distutils or setuptools.

The other aspect I'm interested in is to implement bdist_wheel in SCons. It should be much easier to prototype new features without having to deal with distutils' and setuptools' baggage.

Thanks,

Daniel Holth

On Sun, Jun 5, 2016 at 12:11 PM Dirk Bächle <tshortik at gmx.de <mailto:tshortik at gmx.de> > wrote:

Hi Daniel,

On 05.06.2016 14:55, Daniel Holth wrote:
> Simply I want to try using scons to implement setup.py. To do this I need to have scons run on whatever versions of Python the
> package is compatible with, obviously. Then I need to write a task to generate a couple of packaging metadata and archive formats.
>

are we really talking about the setup.py from the setuptools? Do you really want to re-implement it, but using SCons?
Please feel free to do so, I'm just wondering what the benefit of this would be. I mean, for installing SCons on my system I'd need
to have the setuptools anyway...so what does your solution bring that setuptools doesn't have? Just curious...

Best regards,

Dirk

_______________________________________________
Scons-users mailing list
Scons-users at scons.org <mailto:Scons-users at scons.org> 
https://pairlist4.pair.net/mailman/listinfo/scons-users

_______________________________________________
Scons-users mailing list
Scons-users at scons.org <mailto:Scons-users at scons.org> 
https://pairlist4.pair.net/mailman/listinfo/scons-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20160606/436b36c7/attachment-0001.html>