[Scons-users] Is there a pure-python tar builder?

[Daniel Holth]

Python -m tar 😀

On Mon, Jun 26, 2017, 18:53 Bill Deegan <bill at baddogconsulting.com> wrote:

> Remember though, if a flow depends on creating many independent tarballs,
> implementing it in python means that it will no longer run in parallel.
>
> I guess create as tar_python, and then we can poll the users if this would
> impact them and should we merge/replace with existing tar tool.
>
> Thanks!
>
> On Mon, Jun 26, 2017 at 12:32 PM, Daniel Holth <dholth at gmail.com> wrote:
>
>> I would be satisfied with replacing the official tar builder with a pure
>> Python one. The reward would be consistent cross platform behavior. It
>> would be faster whenever the overhead of spawning a subprocess was greater
>> than the speed difference between Python and native tar (probably true for
>> smaller tar).
>>
>> It would also be possible to build a directory structure inside the
>> .tar.gz without copying the files first as packaging does now, but that
>> would probably require deeper changes to the packaging tool.
>>
>> On Mon, Jun 26, 2017 at 2:51 PM Bill Deegan <bill at baddogconsulting.com>
>> wrote:
>>
>>> Daniel,
>>>
>>> Would the function be a duplicate of the existing tar tool, just pure
>>> python?
>>> Or different function?
>>>
>>> You might take a look at the zip tool which checks for the existence of
>>> the zip package and uses it if available, otherwise calls external zip.
>>> (Likely that can get dropped now as SCons used to support python back to
>>> 1.5.7 (I think)).
>>>
>>> It might make sense to modify the existing tar tool to do similar?
>>>
>>> -Bill
>>>
>>>
>>>
>>> On Mon, Jun 26, 2017 at 11:35 AM, Daniel Holth <dholth at gmail.com> wrote:
>>>
>>>> I love reduced parallelism. Anything wrong with writing my own tar
>>>> builder env = Environment(tools=['python_tar']) to overwrite the stock
>>>> env['BUILDERS']['Tar'] with my own pure Python version? The Python tar
>>>> builder will work on any platform and produce exactly the PAX format tar
>>>> files I want with configurable permissions & file timestamps. Then
>>>> hopefully it will work with the packaging tool which already does 95% of
>>>> what I need. Either that or add my new type to SCons.Tool.packaging.__all__
>>>>
>>>> - Daniel
>>>>
>>>> On Mon, Jun 26, 2017 at 11:50 AM Bill Deegan <bill at baddogconsulting.com>
>>>> wrote:
>>>>
>>>>> Daniel,
>>>>>
>>>>> See:
>>>>> https://bitbucket.org/scons/scons/wiki/DistTarBuilder
>>>>> and
>>>>> https://bitbucket.org/scons/scons/wiki/ArchiveBuilder
>>>>>
>>>>> One note, using python code to implement the builder rather than
>>>>> shelling out to run a command will make the builder subject to the GIL and
>>>>> thus reduce parallelism.
>>>>>
>>>>> -Bill
>>>>>
>>>>> On Mon, Jun 26, 2017 at 8:21 AM, Daniel Holth <dholth at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello, I would like to generate .tar.gz archives cross platform and
>>>>>> on Windows by using the Python tar module instead of the tar command,
>>>>>> similar to how the zip builder uses the zipfile module. Has anyone
>>>>>> developed such a builder?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Daniel
>>>>>>
>>>>>> _______________________________________________
>>>>>> Scons-users mailing list
>>>>>> Scons-users at scons.org
>>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> Scons-users mailing list
>>>>> Scons-users at scons.org
>>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>>
>>>>
>>>> _______________________________________________
>>>> Scons-users mailing list
>>>> Scons-users at scons.org
>>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>>
>>>>
>>> _______________________________________________
>>> Scons-users mailing list
>>> Scons-users at scons.org
>>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>>
>>
>> _______________________________________________
>> Scons-users mailing list
>> Scons-users at scons.org
>> https://pairlist4.pair.net/mailman/listinfo/scons-users
>>
>>
> _______________________________________________
> Scons-users mailing list
> Scons-users at scons.org
> https://pairlist4.pair.net/mailman/listinfo/scons-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20170626/44618ee9/attachment-0001.html>