Bug/Crash submission system

[krntea]

I upgraded to MediaPortal 0.2.3.0 RC1 release the other day, but am now experiencing fairly serious problems: crashing, freezing etc. Computer locks up when TV server is running but without client. Cant shutdown .NET BroadcastEvent lockup problems. I will try and get documentation in on these things as I get a chance. Just a quick suggestion: An automatic crash bug submission system would do the world of good for improving reliability through bug submissions. Usually when it crashes I have kids and more yelling to restart it 5 minutes ago. Its hard to start messing around pulling together evidence at the time, so usually it does not get done later. Typing in quick a quick explanation and press send to deliver dump of everything needed to debug the problem would be beneficial.

[krntea]

A reference:
"Summary
This article provides a good introduction to the "live debugging" story Microsoft has. There is a robust debugging framework present but, unfortunately, it is not widely known in the developer community. Too often, developers as well as managers take a shot in the dark approach and hope to find a resolution this way. It is easy to take crash and hang dumps in production without having to disturb the production environment. It will take some experience to read these dump files. Nevertheless, it is easy to find out a wealth of information about the application at the time of crash or hang. This article is just an introduction. There are many more commands available. You can find more out about each by using the "!help" command. The following blog by Mike Taulty provides more examples as well as more command descriptions. There are two more articles by John Robbins (article one and article two) that also give a good introduction to this topic. Finally, there is a very comprehensive guide by Microsoft how to debug production .NET applications."
http://www.codeguru.com/csharp/.net/....php/c9595__3/

[uggabugga]

This might just be the "best/smartest" suggestion i've read for a while, nice one
Devs, take notice. Might just make your life a bit easier. Or one would expect, under the right conditions, that is!

UggaBugga

[patricon]

I totally agree with you... i hope that devs take notice about this... it could be the most important task to add to MP now... in fact, a few time ago, i developed a system to do this task when i was developer. I did my bug tracking system using ASP and a database, the software was sending all variables, reports, and info about my app when it crashed, so it help me so much to repair all bugs, and no need for end user explanations....

regards,

[infinityloop]

Originally Posted by krntea

Just a quick suggestion: An automatic crash bug submission system would do the world of good for improving reliability through bug submissions.

this would surely make bugreporting for the end-user easy. but anyone thought about the developers?

we do not have enought ppl which would have the time to go though automated reports

1. automatic reports are missing way to much informations (steps how to reproduce, hardware specification-dont tell me about dxdiag now)
2. its not possible to get back to the "reporter" to get more info
3. it is not possible to get back to the "reporter" to tell him how to fix it

The ONLY way how the users can help us to fix problems is to provide us with DECENT bugreports in the forum.

following the ::template:: shouldnt be hard.

Does this mean "much work" for the end-user? -> yes
Will a DECENT report HELP the team to fix issues? -> YES

So ppl, shake off the lazyness, spend some time to do a decent report and contribute someting to this project!

[krntea]

Hi infinityloop,
With all due respect, your post displays a lack of understanding of automatically generated bug submission systems, OR you have been mis-informed, and/or you have not had the pleasure of using one. In my posts here I only wish to try and introduce the MP developers to the pleasure that is an automated bug submission system...

Originally Posted by infinityloop

Does this mean "much work" for the end-user? -> yes

An efficient automated bug reporting system is easy for the end user AND the developer... this thinking that it has to be either hard work for the end user OR the developer is mis-informed.

As a software engineer of 13 years, part of my duties were for many years to support a large complex legacy commercial c++ electrical engineering CAD product with many users spread across many countries. The system has the following crash bug submission system embedded in it: http://www.codeproject.com/debug/crash_report.asp

When the program crashed or hung in some part of the world, the user pressed one key: "send" and all dumps and machine state info are emailed. If the end user felt like it and had the extra 30 seconds, they also could type in a description of what they were doing. The developers inbox receives the automatically generated dumps and user report (here auto scripts can sort the reports by program version and other info into bug reports seen before or something new to save yet more time, but I wont go into that). The point is that the developer takes a minute to click on the dump file, and it loads up into your favorite debugger (Visual Studio, whatever) and the break point positions the on the offending line where the system crashed or hung. You are then usually minutes away from fixing the bug...

Originally Posted by infinityloop

[*]automatic reports are missing way to much informations (steps how to reproduce, hardware specification-dont tell me about dxdiag now)

This is completely incorrect: You have the entire stack trace and all variables that caused and led upto the crash. You also can have system state information at the time of the crash and any third party products that might be influencing the problem - but in 98% of the cases it is usually not necessary to look at this information and neither is it necessary to read the users blurb, that is usually wrong or misleading anyway. All the evidence is before you in the debugger without user explanations necessary in almost all cases.

Originally Posted by infinityloop

[*]its not possible to get back to the "reporter" to get more info[*]it is not possible to get back to the "reporter" to tell him how to fix it

Both Incorrect: The report is emailed: you can email them back if you really wanted to: but this is almost never required.

Originally Posted by infinityloop

So ppl, shake off the lazyness

I am a developer, and as a developer I am lazy: lazy in the way that I want the most fast and efficient algorithm or method of doing things. I do not want to waste time clicking around reading forums full of repeated bug reports all describing the same problem slightly differently, reading long winded reports in bad english from country X etc etc: that support model just does not scale up when you have many users to support. As a developer I want and need crash reports with everything that I need to debug them as fast as possible, so I can spend my time developing rather than reacting to bugs all the time.

Here is a quick simple test you can do to convince yourself and see that I am not inventing all this: Download http://www.codeproject.com/debug/crash_report.asp either run his test program or in 5 minutes you can attach it to any c++ program. Now put in the most nasty hard to find bug you can imagine (or find demonstrated in Dr Dobbs journal), run and crash the program and debug it using the emailed report. You will be shocked how easy and quick it is. Compare that time saving to this forum bug report model... they just cannot compare.

Originally Posted by infinityloop

The ONLY way how the users can help us to fix problems is to provide us with DECENT bugreports in the forum.
Will a DECENT report HELP the team to fix issues?

I agree that a decent bugreport is critical. However MediaPortal does not have a decent bug reporting system, and submitting bug reports in a forum highly inefficient and time wasting for both the users AND the developers. What suffers as a result? The stability of Mediaportal: something everyone here wants to see improve, so we are all together on that one.

Originally Posted by infinityloop

spend some time to do a decent report and contribute someting to this project!

How many people have been watching TV in MP in their relax mode, relaxing time with their family and experienced a MP crash - only to restart and continue their relax mode... rather than get off the couch, interrupt the movie, search the forums and face this large complex barrier to reporting the problem:

Report bugs you have found in this forum.

Your bugreport has to include:

* hardware specification
* MediaPortal Client logs
* MediaPortal TV-Server Logs
* step by step description of the issue
* Installation Type: single seat (Server client on the same PC) | Network - single TV-Server | Network - multiple TV-Servers

note: TV-Server logs are placed in
--> C:\documents and settings\All Users\applicationdata\MediaPortal TV Server\log

Reports without this informations will NOT be processed!

The developers of MP have put the above wall up to reduce bad bug reports - which is normal and understandable since it is very had to process bug reports using forums even when they are very good. The MP core developers are so overworked under this inefficient system that I am sure they will not even have the time to read this thread!

At the moment I use MP at home on an experimental HTPC, had I the tools at home I would debug my crash problems myself (and in the future I hope to have more time to contribute developer skills to the project + get some some development tools on the htpc). However what I have done is looked at all the MP/TV Service generated logs and the source code to try see where MY crash problem is coming from - and I couldn't. Without the debugger and stack trace its extremely hard to do for all but the most obvious problems, as any programmer knows. So how can I or anyone else expect that a written bug report posted in the forum, without dump information and only logs is going to be enough for a MP developer to reproduce the end users environment enough to reproduce the crash, so that they can then use the debugger on it? Its extremely difficult, inefficient and cannot be scaled up. It is also why MP continues to have the same major crash and hang problems that it had months ago when I started using it.
It is also why major open source projects like Firefox, Azureus etc use automatic bug submission systems...

For MP's sake, please consider modernising the bug submission system.

Sincerely,

Keith

[infinityloop]

Originally Posted by krntea

I agree that a decent bugreport is critical. However MediaPortal does not have a decent bug reporting system, and submitting bug reports in a forum highly inefficient and time wasting for both the users AND the developers. What suffers as a result? The stability of Mediaportal: something everyone here wants to see improve, so we are all together on that one.

aahhh, so our mantis bugtracker is not decent for you?

Originally Posted by krntea

The MP core developers are so overworked under this inefficient system that I am sure they will not even have the time to read this thread!

interesting assumption. wrong, but intersting.

Originally Posted by krntea

However what I have done is looked at all the MP/TV Service generated logs and the source code to try see where MY crash problem is coming from - and I couldn't. Without the debugger and stack trace its extremely hard to do for all but the most obvious problems, as any programmer knows. So how can I or anyone else expect that a written bug report posted in the forum, without dump information and only logs is going to be enough for a MP developer to reproduce the end users environment enough to reproduce the crash, so that they can then use the debugger on it?

if we know the steps "how to re-produce the crash", than this is no problem.

And now think about the following.

How often is MediaPortal realy crashing?
And when it crashes how often is the crash caused by MediaPortal?

the major amount of issues which are reported are not related to any crash.
we use "our inefficient" forums to collect the reports, because:

• its easy to talk with the reporter
• other users can help this reporter
• other users see that someoneelse has the same problem (and can confirm it)
• forums are used to "filter" reports, as soon as confirmed they get added to our bugtracker
• we only have one placce to monitor for reports which reduces the workload
• test files can be published very easiely for the effected users

you may not like how we do it here. but for us this is working pretty good.

Originally Posted by krntea

However what I have done is looked at all the MP/TV Service generated logs and the source code to try see where MY crash problem is coming from - and I couldn't.

and you didnt had a look into your windows-eventlog?

[krntea]

infinityloop, I am not personally attacking you. I am sharing knowledge and experience in the forum in a way that should help improve MediaPortal. The snide comments in your response are quite frankly worrying to me: this attitude is not positve nor is it helpful to progressing MediaPortal.

Originally Posted by infinityloop

aahhh, so our mantis bugtracker is not decent for you?

The Mantis bugtracker is not a problem: but it has nothing to do with the bug submission process. Bug trackers are used by the developers to track the bugs that are found after the submission process ends. A very good automatic bug submission system can even automate entry of bugs into the bug tracker, eliminating duplicate submissions before they even reach developer eyes as I mentioned above, saving yet more time for the developers and users but this is not common on very large projects.

Originally Posted by krntea

interesting assumption. wrong, but intersting.

Point taken and no problem. I was merely going by the forum postings and developer responses.

Originally Posted by krntea

if we know the steps "how to re-produce the crash", than this is no problem.

You cant reproduce a large % of crashes because you do not have the end users machine state. Perhaps you did not read this paragraph of my above post:

"However what I have done is looked at all the MP/TV Service generated logs and the source code to try see where MY crash problem is coming from - and I couldn't. Without the debugger and stack trace its extremely hard to do for all but the most obvious problems, as any programmer knows. So how can I or anyone else expect that a written bug report posted in the forum, without dump information and only logs is going to be enough for a MP developer to reproduce the end users environment enough to reproduce the crash, so that they can then use the debugger on it?"

Originally Posted by krntea

And now think about the following.
How often is MediaPortal realy crashing?
And when it crashes how often is the crash caused by MediaPortal?

With automatic bug submission systems, you can pipe the incoming reports info into stats reading scripts so they will show you, over many thousands of submissions and users, the exact answers your asking for any particular version or release. When a buggy release goes out you can see right away as the submissions spike - and you can usually track down the develors that keep submitting buggy code as a result. The answers that gives you are much more useful than asking one person in one forum post.

Originally Posted by krntea

the major amount of issues which are reported are not related to any crash.
we use "our inefficient" forums to collect the reports, because:

• its easy to talk with the reporter
• other users can help this reporter
• other users see that someoneelse has the same problem (and can confirm it)
• forums are used to "filter" reports, as soon as confirmed they get added to our bugtracker
• we only have one placce to monitor for reports which reduces the workload
• test files can be published very easiely for the effected users

you may not like how we do it here. but for us this is working pretty good.

I am always open to new ideas. Are you? Although what I am proposing here is far from new. Your points on the benifits of forums are well taken: I have colleagues on other projects that modified the auto submission system to post relevant forum URLS automatically whenever a repeat bug was submitted. However automatic debugging dump submission a forum cannot do, and as you say if the developer can't reproduce it they (usually) can't fix it.

Originally Posted by krntea

and you didnt had a look into your windows-eventlog?

Do you really think I didn't, or are you just continuing to try to attack me personally for suggesting an improvement, rather than consider the improvement suggestion seriously? Either way I don't see how this is constructive or helping MP progress.

[piranha]

I have agree to a degree with krntea.
When I took over the project I am working on, it had no error handler at all.
The first thing I did was to implement decent error handler and crash/error submission system, and it's so much easier to support the users.
And I am the only one person who is in charge of this project. It doesn't create much of the overhead, actually it works opposite.
One thing I always say, it's better to have information, than not to have it. If you have decent detailed information, that's where you can start filtering out the crap.

You don't have to stop forum submission system. You can join both of them together.

[infinityloop]

Originally Posted by krntea

Do you really think I didn't, or are you just continuing to try to attack me personally for suggesting an improvement, rather than consider the improvement suggestion seriously? Either way I don't see how this is constructive or helping MP progress.

the windows eventlog reveals most of the time where the crash occurs.
so this was a serious question.


you suggest an automatic submission system.
~1year ago the team was thinking about it (as i allready said). the conclusion was that the efforts are to much that we could do it.

so if you want to create such a submission system for us which we can then implement, we would give it a shot.
(also a serious question, not trying to make fun here)

be aware, we simply do not have the required manpower to:

1. create these feature for MediaPortal and TV-Server
2. create the required online facilities to catch and store the reports
3. analyze all submitted reports

all developers are actively busy with working on bugfixes and/or new features. they do not have to waste much time with "trying to reproduce" something because of the lack of informations.
thats something the test team takes care about.

many ppl might think now "i want that submission system. less work for me to report and developers fix it instandly".
the only thing which helps to get more bugs fixed are more developers.
the current "reporting-system/procedure" would even keep twice as much devs as we currently have busy.


what i miss is a inbuild feature which collects all logs and creates a zip file out of it. the user only has to attach it to the report then. a additional crash.dump would be nice then too.

but i prefer manual bugreports in a forum.