Stability Release

[CHli]

Originally Posted by gds

Well, in an ideal world you'd be totally right. But software is never really error-free, so it's just a matter of defining metrics to say what works (and to what extent it works) and what doesn't work. To make myself clearer, think about Microsoft releasing Windows XP: when they released it, it wasn't certainly bug-free (a couple of Service Packs and some more updates prove I'm not wrong ...) but they let it out of the beta stage when most of the show-stopper bugs were solved for most of the possible HW configurations. It was not perfect but under most conditions it worked. If they had waited for the "perfect" XP to let it come out, it would never come out...
I think it would help the developers to know what always works, what works most of the time and what doesn't work at all for most of the users, so that they can prioritize bug-hunting, working first and foremost to solve show-stopper bugs and leave the least annoying one waiting for their spare time.
And, by they way, I agree that having non binary test results can lead the test results to be conditioned by human judgement, but here comes statistics to help us figure the real priorities.

Generally when a tester, test a feature it can only check it on one computer. So on his computer the feature will work or not. If on someone else's computer it fails then we'll see that it's not stable under some conditions / hardware configuration. This keep the things easy so even non computer scientist can help debugging.

Originally Posted by gds

Well, we are here to define the test structure and I think that the tests should encompass the broader possible range of issues. So, given that the most important goal of a test set is to identify bugs, one other important goal is to identify areas where the program behaves in an inadequate way. By this, I don't mean we should have testers say: "hey, scrolling items in menus is tooo slow", but have them identify areas where the lack of speed can impede using the software in a fruitful way (e.g., if MP records your favorite show but looses a frame here and there cos its code is not optimized, you can say it works, but it cannot be tought it works in an adequate way).
I hope I made myself clear.
Bye.

Ok your point of view is clearer now. But I think the test can be :

Open MyTV, record now, record the entire show, push ! and look at the dropped frame counter. Does it increase ? : yes / no

There is no way for subjectivity.

[infinityloop]

Hello!

The new Media Portal Test Team Forum is now ONLINE!

If you want to help us testing Media Portal and provide the developers with useful informations please register at.
www.elite-squad.net

everyone can help. you just have to follow the TestPlans that will be posted soon.
i have also written a HowTo:use Testplans and Post Results in the new Forum.

If you have experience in developing and/or testing software please send a pm at the new forum (nick is infinityloop). we need your help to setup new testplans.

[igalan]

Wasn't it possible to create the necessary forums here? Creating a new website makes harder to exchange information, users and does not seem to bring any advantage to me. Not to mention the fact that I'm alredy subscribed to countless forums . I appreciate the effort behind this, but I'm looking to make things more efficient and simple.

[infinityloop]

Originally Posted by igalan

Wasn't it possible to create the necessary forums here?

i have tried to get a new subforum here.

the new forum also has advantages:
x create new subforums quickly
x add users to other groups i.e. when the join the Test Team Core
x most important: we need to attach eventlog and dxdiag files to the testresults

[FlipGer]

Hi,

i agree with iagalan in both points.

1) Great, and thanks for your effort!
2) Try to implement the forum here (contact luke or MrMario)

Flip

[FlipGer]

Hi,

ok i see. Thanks Avalon!

Flip

[infinityloop]

Today Media Portal 0.2pre was released.
::download::

:!: And we have the first 4 Testplans Online! :!:
:::Media Portal Test Group Forum:::

We have been working very hard the last days to develop a easy and usefull Testprocedure.

Now that you all see how a testplan looks i know many of you noticed that they can help us with setting up Testplans.

If you want to help us please send a email to infinityloop@gmx.net

Lets get some testing done!

have a nice day
chris aka infinityloop

[idioteque]

the stable version should consist of:

1 skin and 1 language and as few plugins as possible, use its own codecs and only use certified drivers