利用者:Eibriel/Automated Bug Classification

Proposal

Automatically classify and tag bug reports.

Motivation

Bug review takes a lot of valuable time to developers, that task could be automatized at least in part.

Scope

The software should read bug reports tagged as Needs Triage, apply a statistical model to predict if the developer will tag the report as Incomplete (example) , if the likelihood surpass some threshold it will leave a message like:

Please follow our submission template and guidelines and make a complete, valid bug report, with required info, precise description of the issue, precise steps to reproduce it, small and simple .blend and/or other files to do so if needed, etc.

The developer still needs to set the triage of the bug report, but at that time the user will correct the description of the issue.

The software could also detect when a bug report is in fact a Support Request (example) and could redirect the user to the Community Support.

As a more advanced feature the software could automatically assign bug reports to the corresponding developer.

The software

The software will be a Python3 script coupled with a database. The script will communicate to developer.blender.org through the Conduit API using an user specifically created for this task. The script will use SKLearn to infer the current status of the bug report.

The username, avatar and message should be easily recognizable as a Bot and not a real Developer.

Machine Learning

The inference will be done using Machine Learning techniques, I propose using BaggingClassifier over a bag of words (to investigate:Common Vectorizer).

Dumping the Database

I'll make a dump of the Bug Reports on the last 12 months in such a way that we know what was the report at the moment it received the firs triage.

Features

Then I'll analyze the reports and extract features (words) that better describe the quality of the Bug Report. Also will be useful to know if the report have at least one blend file attached or at least a text file attached.

Training

Then I can train the model to predict the triage received using as feature the count of the words on the "bag". For instance if the Bug don't contains any of the words "Linux", "Windows" or "MacOS" the probability to be an Incomplete bug is high.

Validation

I'll test the model on new bug reports to measure its performance, when the model have a performance higher than some value selected by blender core developers (for example 95%) the model will be ready for production. We can also choose to prefer false positives or false negatives. Maybe is better to have a high performance on classifying correctly all Incomplete bug reports even when the system misclassify some Complete bug reports, or maybe is better on the other way to avoid confusing the users, classifying correctly all Complete bug reports at the expense of misclassify some Incomplete bug reports.

Maintenance

The model can be retrained regularly using new data gathered automatically (and deleting old data) to keep up to date with new ways of write Incomplete bug reports.

Conduit API

https://developer.blender.org/api/conduit.ping

   {"result":"developer.blender.org","error_code":null,"error_info":null}