Validation and Reporting of Models
Facilitated by Andreas Maunz and Harry Sarimveis
The tutorial “Validation and Reporting of Models” on the OpenTox 2011 conference in Munich shall provide insight into validation and reporting of model performance, as well as guide participants in using these tools on actual datasets and algorithms.
Opening Remarks
The tutorial is derived from the “Advanced Validation Tutorial” by M. Gütlein, see http://www.opentox.org/tutorials/advanced-validation and presented by Andreas Maunz (FDM, Uni Freiburg) and Haralambos Sarimveis (National Technical University, Athens).
Materials
Note: Participants are kindly asked to register on the OpenTox website, since many services used require an account.
Draft Program
The following provides an overview of the tutorial content:
Presentation Part
- Validation: What it is, OECD principles, Applicability Domain Estimation.
- Introductory information on OpenTox architecture.
- Validation as a “plugin” for OpenTox applications.
HTML reports
- Example: ToxCreate Classification.
- ROC-Plots.
- Confidence-vs-Accuracy plots.
- Inspection of Individual Folds.
- Example: ToxCreate Regression.
- Scatterplots.
- Confidence-vs-RMSE plot.
QMRF reports
- Standardized Template with Sections.
- Save and print, restore pdf.
- Some information on the QPRF editor.
Interactive Part
- Entry page of the validation service, wrapped by HTML.
- Dataset and Model may be inserted by URI.
- Train / Testset Validation: Build a model, use it on test dataset.
- Participants experimentation, using OpenTox login.
Complete the evaluation form following this link: Evaluation Form