[Scons-users] [Scons-dev] Support for Tools via a pip package / python module in sys path

Tue Aug 1 11:29:45 EDT 2017

Once we merge your pull request I'll try building on my system.
I know there is occasionally some issue using lxml vs libxml2/libxslt.

If you like I have a script to build python with libxml2 and libxslt.
Just change the version string at the top and it should build a python
which should work for the doc flow.

https://github.com/bdbaddog/python_build_script

-Bill

On Tue, Aug 1, 2017 at 8:04 AM, RW <garlicbready at googlemail.com> wrote:

> Just to follow on the below
> I've recently found I can do what I want to do without making any
> functional changes to scons
> but I put in a pull request for some changes to the user manual to make it
> clear on how to use toolpath since it doesn't seem to be documented all
> that much
> the trick I found was just to put sys.path into the toolpath argument so
> that it can pick up on tools installed via a package management system like
> pip
>
> https://bitbucket.org/scons/scons/pull-requests/493/
> updates-to-user-manual-tests-for-the-use/diff#comment-None
>
>
> I'm still having problems generating the docs under the head sources though
> I had to copy the xml files to a 2.5.1 tree just to build them into html
> to check how they would come out
>
> I had a root through the list of bugs but I don't think this one is listed
> yet
> Under the latest sources
>
> doc/user/tasks.xml is built to build/doc/generated/examples/
> tasks_ex1_1.xml
>
> tasks.xml is the same / no change between 2.5.1 and 3.0 latest
> but the resultant tasks_ex1_1.xml is different
>
> under 2.5.1 it comes out as
> ```
> <?xml version="1.0" encoding="UTF-8"?>
> <screen xmlns="http://www.scons.org/dbxsd/v1.0" xmlns:xsi="
> http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://
> www.scons.org/dbxsd/v1.0 http://www.scons.org/dbxsd/v1.0/scons.xsd">%
> <userinput>scons -Q</userinput>
> cc -o app main.cpp
> cat < foo.bar2 > foo.cpp
> cc -o app2 main2.cpp foo.cpp
> cat < test.bar > test.h
> </screen>
> ```
>
> under 3.0 latest it comes out as
> ```
> <?xml version="1.0" encoding="UTF-8"?>
> <screen xmlns="http://www.scons.org/dbxsd/v1.0" xmlns:xsi="
> http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://
> www.scons.org/dbxsd/v1.0 http://www.scons.org/dbxsd/v1.0/scons.xsd">%
> <userinput>scons -Q</userinput>
> [?1034hcat < test.bar > test.h
> cc -o app main.cpp
> cat < foo.bar2 > foo.cpp
> cc -o app2 main2.cpp foo.cpp
> </screen>
> ```
>
> I think the difference leads to an error of
> ```
> scons: Reading SConscript files ...
> scons: done reading SConscript files.
> scons: Building targets ...
> __xinclude_lxml(["scons_xi.xml"], ["main.xml"])
> __build_lxml(["scons_ex.xml"], ["scons_xi.xml"])
> scons: *** [scons_exi.xml] XSLTApplyError : Cannot resolve URI
> ../generated/examples/tasks_ex1_1.xml
> scons: building terminated because of errors.
> scons: *** [build/doc/user/index.html] Error 2
> scons: building terminated because of errors.
> ```
>
> Since the source file is the same I think it's something inbetween
> if you want me to file this as a bug then let me know
>
> Many Thanks
> Richard
>
> On 20 July 2017 at 21:21, RW <garlicbready at googlemail.com> wrote:
>
>> Hi Bill,
>>
>> # Current Search Mechanism
>>
>> From what I understand of the toolpath search mechanism it searches:
>> 1. within scons\engine\SCons\Tool for the tools bundled with scons
>> 2. A list of directories specified within the toolpath parameter
>> 3. A variable called DefaultToolpath within the scons source (by default
>> is empty)
>>
>> I checked experimentally and it looks as if the syspath is not included
>> in the search for tools so I don't think it can pick up on modules at
>> present when searching for tools.
>>
>> I've already added in the dot syntax in a prior commit to do something
>> different
>> The dot syntax is currently coded to search sub-directories within
>> toolpaths
>> so if you have a toolname like 'Test1.TestBuilder1'
>> and a list of toolpaths like
>> C:\Lib\toolpath1
>> C:\Lib\toolpath2
>> C:\Lib\toolpath3
>>
>> The places it's going to search will be something like
>> scons\engine\SCons\Tool\Test1\TestBuilder1
>> C:\Lib\toolpath1\Test1\TestBuilder1
>> C:\Lib\toolpath2\Test1\TestBuilder1
>> C:\Lib\toolpath3\Test1\TestBuilder1
>>
>> I could change that around so that the dot means something else
>> and use another character such as forward slash (pythonesque) as the
>> directory separator instead.
>> That might be a better idea
>>
>> # Overrinding
>>
>> For overriding, it's currently just currently adding to the toolpath
>> parameter behind the scenes
>> so the behaviour will be the same as that, I'm not sure how that works
>> currently
>> If toolpath overrides the inbuilt tools then this should work the same way
>>
>>
>> # Suggested syntax
>>
>> There's a few different ways of changing the syntax around
>> To give some examples of what I'm thinking based on what you've said
>>
>> 1. With the below example I've change the dot to a forward slash within
>> the toolname
>> "TestBuilder1" - look for tool in root of toolpath directory
>> "ExampleDir1/TestBuilder1" - look for tool in sub-dir of toolpath
>> directory
>> "ExampleDir1/ExampleDir2/TestBuilder1" - look for tool in sub-dir of
>> toolpath directory
>>
>> 2. This example uses a # prefix in the toolname to specify to use a
>> python module as the base search directory (add to the toolpath)
>> "scons_toollib_example#TestBuilder1"
>> "scons_toollib_example.submod1#TestBuilder1"
>> "scons_toollib_example#ExampleDir1/TestBuilder1"
>> "scons_toollib_example#ExampleDir1/ExampleDir2/TestBuilder1"
>>
>> 3. We could also keep the toolmods option to avoid using a prefix as well
>> all that will do is find the directory where the module is located and
>> add that directory to the toolpath list
>>
>>
>> Any thoughts?
>>
>>
>> Many Thanks
>> Richard
>>
>>
>> On 20 July 2017 at 20:05, Bill Deegan <bill at baddogconsulting.com> wrote:
>>
>>> Wouldn't this work and not require any changes?
>>> Or if it doesn't already work, wouldn't that be easier syntax? no
>>> additional arguments, if it's got a '.' in it then you know it's part of a
>>> package?
>>>
>>> toollist = ['scons_toollib_example.TestBuilder1']
>>> env = Environment(ENV = os.environ, tools = toollist)
>>> env.TestBuilder1('testdest1.txt', 'testsrc1.txt')
>>>
>>> How would you handle overloaded tool names in a module?
>>> For example
>>> scons_toollib_example has a gcc tool?
>>>
>>> -Bill
>>>
>>>
>>>
>>> On Thu, Jul 20, 2017 at 2:53 PM, RW via Scons-dev <scons-dev at scons.org>
>>> wrote:
>>>
>>>> Hi,
>>>> One of the features I'm interested in is the ability to have tools
>>>> loaded externally via a package installed via pip.
>>>> This way if you wanted to, you could ether install tools as a separate
>>>> repo via pip, or move some of the existing tools into separate repo's
>>>>
>>>> I think I've come up with a way to do it which doesn't involve much
>>>> code.
>>>> But before I add in any tests or additions to the man page / user docs
>>>> I wanted to check if this is an okay way to implement this
>>>>
>>>> I've put the commit here:
>>>> https://bitbucket.org/grbd/scons/commits/af3ea9f2d4468479f7f
>>>> 7be73afd455b0f2067409
>>>>
>>>> With this method there's an additional option on the environment
>>>> constructor called toolmods
>>>>
>>>> ```
>>>> toollist = ['TestBuilder1']
>>>> env = Environment(ENV = os.environ, tools = toollist, toolmods =
>>>> ['scons_toollib_example'])
>>>> env.TestBuilder1('testdest1.txt', 'testsrc1.txt')
>>>> ```
>>>>
>>>> This option looks up the directory where the module is located in the
>>>> sys path and adds it to the toolpath
>>>> (e.g. C:\Python27\Lib\site-packages\scons_toollib_example)
>>>>
>>>> A side effect is that the module file gets imported at the same time
>>>> (scons_toollib_example\__init__.py is run)
>>>> I just do this to get the full path to the module when adding to the
>>>> toolpath, although you could probably use that to modify scons or do
>>>> something else when the tool is referenced if you wanted to.
>>>>
>>>> Is this implementation is a good idea? if so I'll look into adding some
>>>> tests and user doc entries for it before doing a pull request
>>>>
>>>> Many Thanks
>>>> Richard
>>>>
>>>> _______________________________________________
>>>> Scons-dev mailing list
>>>> Scons-dev at scons.org
>>>> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20170801/c143da6f/attachment-0001.html>