[Scons-users] travis ci failing for py3 builds which are succeeding?

[Bill Deegan]

Sounds good!
I'd love to see passing on all the CI's.. :) For all platforms and builds.
(You know.. unless we actually break the tests..)

On Mon, Feb 19, 2018 at 3:46 PM, Daniel Moody <dmoody256 at gmail.com> wrote:

> I meant ignore failures for JobsTest only. Anyways, I agree, if we make an
> issue to track the JobsTest issue and for now just ignore it in the CI.
>
> I think it may even better to ignore the specific part of that test that
> fails intermittantly (parallel execution) and continue with the rest of the
> test as there is still good testing value there.
>
> I'll submit an issue regarding JobsTest in the CI environments (I have
> seen the same issue intermittently in appveyor) and a PR with proposed
> changes to skip the specific failure only when in CI environments.
>
> On Feb 19, 2018 3:24 PM, "Bill Deegan" <bill at baddogconsulting.com> wrote:
>
>> Ignoring failures in the CI environment would negate their value.
>> Ideally we see all pass which we expect to pass and then build is marked
>> successful and we don't need to dig into the details of each build's tests.
>> Otherwise you need to go look at each build * N (where N = platforms  *
>> python versions) to know if the changes introduced any new bugs
>>
>> That said, since we see failure often for that single test, I'd be o.k.
>> with letting the tests pass if it fails, and skip trying to run it 10
>> times.
>> Just run it singly and then run the rest. Ignoring the return value for
>> the single test, but allowing the full test - jobstests.py run to define a
>> passing or failing build.
>>
>> And of course even more ideally we'd figure out why that test fails in
>> that environment and fix it, but it may not be worth the time at this point.
>>
>>
>>
>> On Mon, Feb 19, 2018 at 3:02 PM, Daniel Moody <dmoody256 at gmail.com>
>> wrote:
>>
>>> Actually there are several that are passing in the build history after
>>> the patch.
>>>
>>> The issue for the ones that are failing is that JobsTest.py is
>>> intermittently failing. I tried to do a workaround that retried the test 10
>>> times, but sometimes it just continually fails.
>>>
>>> I think it's because of the VM environment that travis uses.
>>>
>>> I was thinking instead of trying to workaround it, we could ignore
>>> failures while in the CI environments.
>>>
>>>
>>>
>>> On Feb 19, 2018 2:49 PM, "Bill Deegan" <bill at baddogconsulting.com>
>>> wrote:
>>>
>>>> This one:
>>>> https://travis-ci.org/SCons/scons/jobs/343424806
>>>> (or any since we put in the patch to bump the pickle version up in
>>>> order to fix the py3.* builds)
>>>>
>>>> On Mon, Feb 19, 2018 at 2:42 PM, Daniel Moody <dmoody256 at gmail.com>
>>>> wrote:
>>>>
>>>>> The double bar (||) means execute the next command on non zero exit
>>>>> from previous command. So if there is exit 1 or 2, then it checks if 2,
>>>>> which means that it was only no results, which is considered passing. If
>>>>> there is a 1 exitcode then it fails.
>>>>>
>>>>> Do you have a specific travis build in question?
>>>>>
>>>>>
>>>>> On Feb 19, 2018 2:37 PM, "Bill Deegan" <bill at baddogconsulting.com>
>>>>> wrote:
>>>>>
>>>>>> Daniel,
>>>>>>
>>>>>> I think it's this line in the .travis.yml
>>>>>>
>>>>>> What do you expect:
>>>>>> - python runtest.py -a --exclude-list exclude_jobtest || if [[ $? ==
>>>>>> 2 ]]; then true; else false; fi
>>>>>>
>>>>>>
>>>>>> Do?
>>>>>>
>>>>>> Here's what I see with a simple test in my bash shell:
>>>>>> $ echo "BILL"
>>>>>> BILL
>>>>>> $ echo $?
>>>>>> 0
>>>>>> $ if [[ $? == 2 ]]; then true; else false; fi
>>>>>> $ echo $?
>>>>>> 1
>>>>>>
>>>>>>
>>>>>>
>>>>>> -Bill
>>>>>>
>>>>>
>>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20180219/039f8018/attachment.html>