[Scons-users] Is there a pure-python tar builder?

[Bill Deegan]

Remember though, if a flow depends on creating many independent tarballs,
implementing it in python means that it will no longer run in parallel.

I guess create as tar_python, and then we can poll the users if this would
impact them and should we merge/replace with existing tar tool.

Thanks!

On Mon, Jun 26, 2017 at 12:32 PM, Daniel Holth <dholth at gmail.com> wrote:

> I would be satisfied with replacing the official tar builder with a pure
> Python one. The reward would be consistent cross platform behavior. It
> would be faster whenever the overhead of spawning a subprocess was greater
> than the speed difference between Python and native tar (probably true for
> smaller tar).
>
> It would also be possible to build a directory structure inside the
> .tar.gz without copying the files first as packaging does now, but that
> would probably require deeper changes to the packaging tool.
>
> On Mon, Jun 26, 2017 at 2:51 PM Bill Deegan <bill at baddogconsulting.com>
> wrote:
>
>> Daniel,
>>
>> Would the function be a duplicate of the existing tar tool, just pure
>> python?
>> Or different function?
>>
>> You might take a look at the zip tool which checks for the existence of
>> the zip package and uses it if available, otherwise calls external zip.
>> (Likely that can get dropped now as SCons used to support python back to
>> 1.5.7 (I think)).
>>
>> It might make sense to modify the existing tar tool to do similar?
>>
>> -Bill
>>
>>
>>
>> On Mon, Jun 26, 2017 at 11:35 AM, Daniel Holth <dholth at gmail.com> wrote:
>>
>>> I love reduced parallelism. Anything wrong with writing my own tar
>>> builder env = Environment(tools=['python_tar']) to overwrite the stock
>>> env['BUILDERS']['Tar'] with my own pure Python version? The Python tar
>>> builder will work on any platform and produce exactly the PAX format tar
>>> files I want with configurable permissions & file timestamps. Then
>>> hopefully it will work with the packaging tool which already does 95% of
>>> what I need. Either that or add my new type to SCons.Tool.packaging.__all__
>>>
>>> - Daniel
>>>
>>> On Mon, Jun 26, 2017 at 11:50 AM Bill Deegan <bill at baddogconsulting.com>
>>> wrote:
>>>
>>>> Daniel,
>>>>
>>>> See:
>>>> https://bitbucket.org/scons/scons/wiki/DistTarBuilder
>>>> and
>>>> https://bitbucket.org/scons/scons/wiki/ArchiveBuilder
>>>>
>>>> One note, using python code to implement the builder rather than
>>>> shelling out to run a command will make the builder subject to the GIL and
>>>> thus reduce parallelism.
>>>>
>>>> -Bill
>>>>
>>>> On Mon, Jun 26, 2017 at 8:21 AM, Daniel Holth <dholth at gmail.com> wrote:
>>>>
>>>>> Hello, I would like to generate .tar.gz archives cross platform and on
>>>>> Windows by using the Python tar module instead of the tar command, similar
>>>>> to how the zip builder uses the zipfile module. Has anyone developed such a
>>>>> builder?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Daniel
>>>>>
>>>>> _______________________________________________
>>>>> Scons-users mailing list
>>>>> Scons-users at scons.org
>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>
>>>>>
>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>>
>>> _______________________________________________
>>> Scons-users mailing list
>>> Scons-users at scons.org
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20170626/78c88b17/attachment.html>