Sections
You are here: Home » Tutorials » Advanced Validation

Validate Your QSAR Model and Create a Report using Web Forms

In this tutorial you will learn how to employ the validation services and explore the reporting facilities, including generating QMRF reports, via web forms.

General Information / Requirements

Description of Activity:

In this tutorial we will demonstrate how to effectively use the validation and reporting web services behind the applications of ToxPredict and ToxCreate. Using web forms, we will contact the web services and validate a model or algorithm using a number of different approaches such as k-fold split, training-test-split, or bootstrapping. Furthermore, we will generate a QMRF report and visualize it using the QMRF Editor web start application. This tutorial is aimed at advanced users and developers, who want to look behind the scenes. 

Datasets and Algorithms/Models:

Provided as online resources: https://docs.google.com/spreadsheet/pub?hl=en_US&hl=en_US&key=0AllBraSK8gV-dFgwUDJHaVJmQTlJRG1iTnJqNXBaZEE&single=true&gid=0&output=html

Required Software:

Web browser.

Evaluation Form

Please provide your feedback after completing this exercise using the Evaluation form at https://www.surveymonkey.com/s/652XY7C

Example 1 - Build and validate a regression model with training-test-set validation

In this use case, we will build a model with a training dataset, apply the model to predict the compounds in the test-dataset, and validate the results.

 

Step 1 - Log in as guest (user: guest, passowrd: guest) at  http://opentox.informatik.uni-freiburg.de/validation/sign_in

We have Authentication & Authorization enabled on our services. The resources used in this tutorial are available to the user guest (password guest).

 

Step 2 - Fill out html form at http://opentox.informatik.uni-freiburg.de/validation/training_test_validation

We will use Multiple Linear Regression for prediction (the service is located in Athens, http://opentox.ntua.gr:8080/algorithm)

The dataset is a well known permeability dataset (http://pubs.acs.org/doi/abs/10.1021/ci049884m). It includes 4 numerical chemical descriptors, and one measured numerical endpoint (caco-2 permeability).

The dataset is available at the Ambit web service (located in Sophia, http://apps.ideaconsult.net:8080/ambit2/dataset)

Visit http://opentox.informatik.uni-freiburg.de/validation/training_test_validation  and fill out the form with the above given links. Click Perform training-set-validation. This will load a task resource, like e.g.: http://opentox.informatik.uni-freiburg.de/task/12251. Reload this page (hitting key F5, or using the reload button of your browser) until the task is completed. The result of the task is given in the section Related links (e.g.: The task result: http://opentox.informatik.uni-freiburg.de/validation/1406).

 

Step 3 - Create validation report

Visiting your validation result directly with a browser (e.g. http://opentox.informatik.uni-freiburg.de/validation/1406) will give you the internal representation of the validation. You can create a report by clicking on the Search for corresponding report link (example link: http://opentox.informatik.uni-freiburg.de/validation/report/validation?validation=http://opentox.informatik.uni-freiburg.de/validation/1406)

A form is provided for creating a report. The parameter field is already filled out with the corresponding validation link, you just have to click the Create validation report button. This will forward you to another task page. Again, reload until the task is completed and follow the task-result link to the validation report.

 

Step 4 - Visit validation report

An example validation report is available here: http://opentox.informatik.uni-freiburg.de/validation/report/validation/147.

  • The first section of the report is the Results section. It contains the most important validation results, like R² and Root mean squared error.
  • The Plots provides the Actual vs Predicted values plot. This plot gives an intuitive overview on the predictions performance of your model.
  • The All Results section gives more details on the validation, like validation runtime, or the variance of the actual and predicted values.
  • Predictions lists all compounds, providing a picture, the actual target value and the predicted target value for each compound.

 

Example 2 - Evaluate classification with crossvalidation

Tutorial will be added soon....

Example crossvalidation report: http://opentox.informatik.uni-freiburg.de/validation/report/crossvalidation/5


Play around and make your own validations

You can follow the links to opentox-resources at the following link to play around with the validation service and make your own validations:

https://docs.google.com/spreadsheet/pub?hl=en_US&hl=en_US&key=0AllBraSK8gV-dFgwUDJHaVJmQTlJRG1iTnJqNXBaZEE&single=true&gid=0&output=html

 

Additional Material

Recording of the online tutorial

Download the Recording of the Online Tutorial held 22 June 2011

API-Definition

Before we start, it is probably an advantage to have a window with the API definitions for the validation services open in a browser: so please open the following link in a browser, preferably Firefox: http://www.opentox.org/data/documents/development/validation/Validation/

Validation Examples using the commandline

The validation web services can be accessed using the command line tool curl (http://curl.haxx.se). The examples are available here: http://opentox.informatik.uni-freiburg.de/validation/examples

 

Evaluation Form

Please don't forget to provide your feedback after completing this exercise using the Evaluation form at https://www.surveymonkey.com/s/652XY7C

 


 

 

 

 

Document Actions