[Scons-users] travis ci failing for py3 builds which are succeeding?

[Daniel Moody]

I meant ignore failures for JobsTest only. Anyways, I agree, if we make an
issue to track the JobsTest issue and for now just ignore it in the CI.

I think it may even better to ignore the specific part of that test that
fails intermittantly (parallel execution) and continue with the rest of the
test as there is still good testing value there.

I'll submit an issue regarding JobsTest in the CI environments (I have seen
the same issue intermittently in appveyor) and a PR with proposed changes
to skip the specific failure only when in CI environments.

On Feb 19, 2018 3:24 PM, "Bill Deegan" <bill at baddogconsulting.com> wrote:

> Ignoring failures in the CI environment would negate their value.
> Ideally we see all pass which we expect to pass and then build is marked
> successful and we don't need to dig into the details of each build's tests.
> Otherwise you need to go look at each build * N (where N = platforms  *
> python versions) to know if the changes introduced any new bugs
>
> That said, since we see failure often for that single test, I'd be o.k.
> with letting the tests pass if it fails, and skip trying to run it 10
> times.
> Just run it singly and then run the rest. Ignoring the return value for
> the single test, but allowing the full test - jobstests.py run to define a
> passing or failing build.
>
> And of course even more ideally we'd figure out why that test fails in
> that environment and fix it, but it may not be worth the time at this point.
>
>
>
> On Mon, Feb 19, 2018 at 3:02 PM, Daniel Moody <dmoody256 at gmail.com> wrote:
>
>> Actually there are several that are passing in the build history after
>> the patch.
>>
>> The issue for the ones that are failing is that JobsTest.py is
>> intermittently failing. I tried to do a workaround that retried the test 10
>> times, but sometimes it just continually fails.
>>
>> I think it's because of the VM environment that travis uses.
>>
>> I was thinking instead of trying to workaround it, we could ignore
>> failures while in the CI environments.
>>
>>
>>
>> On Feb 19, 2018 2:49 PM, "Bill Deegan" <bill at baddogconsulting.com> wrote:
>>
>>> This one:
>>> https://travis-ci.org/SCons/scons/jobs/343424806
>>> (or any since we put in the patch to bump the pickle version up in order
>>> to fix the py3.* builds)
>>>
>>> On Mon, Feb 19, 2018 at 2:42 PM, Daniel Moody <dmoody256 at gmail.com>
>>> wrote:
>>>
>>>> The double bar (||) means execute the next command on non zero exit
>>>> from previous command. So if there is exit 1 or 2, then it checks if 2,
>>>> which means that it was only no results, which is considered passing. If
>>>> there is a 1 exitcode then it fails.
>>>>
>>>> Do you have a specific travis build in question?
>>>>
>>>>
>>>> On Feb 19, 2018 2:37 PM, "Bill Deegan" <bill at baddogconsulting.com>
>>>> wrote:
>>>>
>>>>> Daniel,
>>>>>
>>>>> I think it's this line in the .travis.yml
>>>>>
>>>>> What do you expect:
>>>>> - python runtest.py -a --exclude-list exclude_jobtest || if [[ $? ==
>>>>> 2 ]]; then true; else false; fi
>>>>>
>>>>>
>>>>> Do?
>>>>>
>>>>> Here's what I see with a simple test in my bash shell:
>>>>> $ echo "BILL"
>>>>> BILL
>>>>> $ echo $?
>>>>> 0
>>>>> $ if [[ $? == 2 ]]; then true; else false; fi
>>>>> $ echo $?
>>>>> 1
>>>>>
>>>>>
>>>>>
>>>>> -Bill
>>>>>
>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist4.pair.net/pipermail/scons-users/attachments/20180219/f4fb7d22/attachment-0001.html>