Validation
Related links
Examples: a how-to with curl calls | http://opentox.informatik.uni-freiburg.de/validation/examples |
Background information: validation workflow | Validation and Reporting Overview and Data Flow |
Technical details: statistics computed by the validation | Validation statistics |
Validation
Component description
A validation corresponds to the validation of a model on a test dataset. The results are stored in another dataset. Parameters with default values are optional.
REST operations
Description | Method | URI | Parameters | Result | Status codes |
Get all validations |
GET | / | - | List of validation URIs | 200,404 |
Retrieves a validation representation | GET | /{id} | - | Validation representation in one of the supported MIME types |
200,404 |
Validates a model on a test dataset |
POST | / | model_uri test_dataset_uri test_target_dataset_uri (default = test_dataset_uri) |
Validation URI or Task URI |
200,400,404,500 |
Builds a model on a training dataset and validates it on a test dataset |
POST |
/ | algorithm_uri prediction_feature algorithm_params (string, default="") training_dataset_uri test_dataset_uri test_target_dataset_uri (default = test_dataset_uri) y_scramble (boolean, default=false) y_scramble_seed (integer, default=1) |
Validation URI or Task URI | 200,400,404,500 |
Directly perform a validation by specifying test- and prediction dataset | POST | /validate_datasets | prediction_feature test_dataset_uri test_target_dataset_uri (default = test_dataset_uri) prediction_dataset_uri predicted_feature (.i.e feature in prediction dataset) |
Validation URI or Task URI | 200,400,404,500 |
Splits a dataset into training and test dataset according to a certain ratio, and performs a validation |
POST | /training_test_split | algorithm_uri prediction_feature algorithm_params (string, default="") dataset_uri split_ratio(float, default=0.66) random_seed(integer, default=1) y_scramble (boolean, default=false) y_scramble_seed (integer, default=1) |
Validation URI or Task URI | 200,400,404,500 |
OPTIONAL: Performs a bootstrap validation |
POST | /bootstrap | algorithm_uri prediction_feature dataset_params (string, default="") dataset_uri bootstrap_percentage(float, default=0.66) random_seed(integer, default=1) y_scramble (boolean, default=false) y_scramble_seed (integer, default=1) |
Validation URI or Task URI | 200,400,404,500 |
Deletes a validation. | DELETE | /{id} | - | - |
200,404 |
Validation representation
- Requested MIME type should be set in the requests "Accept" header, e.g. curl -X GET -H "Accept:application/xml" http://{server}/validation/{id}
- RDF representation defined in opentox.owl
- RDF Examples
Supported MIME types:
Mandatory:
- application/rdf+xml (default)
Optional:
- application/xml, see XML schema definition for validation object
HTTP status codes
Interpretation | Nr | Name |
Success | 200 | OK |
Validation not found | 404 | Not Found |
Illegal model/algorithm/dataset/algorithm params | 400 | Bad request |
Validation/prediction error | 500 | Internal Server Error |
Cross-Validation
Component description
Performs a k-fold cross-validation, which results in k validations. Parameters with default values are optional.
REST operations
Description | Method | URI | Parameters | Result | Status codes |
Get all cross-validations |
GET | /crossvalidation | - | List of crossvalidation URIs | 200,404 |
Retrieves a cross-validation representation | GET | /crossvalidation/{id} | - | Cross-Validation in one of the supported MIME types |
200,404 |
Returns all (k) validations that belong to a crossvalidation |
GET | /crossvalidation/{id}/validations | - | List of validation URIs |
200,404 |
Performs a k-fold cross-validation. |
POST |
/crossvalidation | algorithm_uri prediction_feature algorithm_params (string, default="") num_folds (integer, default=10) random_seed (integer, default=1) stratified (boolean, default=true) y_scramble (boolean, default=false) y_scramble_seed (integer, default=1) |
Cross-Validation URI or Task URI | 200,400,404,500 |
Performs a leave-one-out cross-validation. |
POST |
/crossvalidation/loo | algorithm_uri prediction_feature algorithm_params (string, default="") y_scramble (boolean, default=false) y_scramble_seed (integer, default=1) |
Cross-Validation URI or Task URI | 200,400,404,500 |
Deletes a cross-validation. | DELETE | /crossvalidation/{id} | - | - |
200,404 |
Cross-validation representation
- Requested MIME type should be set in the requests "Accept" header, e.g. curl -X GET -H "Accept:application/xml" http://{server}/crossvalidation/{id}
Supported MIME types:
Mandatory:
- application/rdf+xml (default)
Optional:
- application/xml, see XML schema definition for crossvalidation object
HTTP status codes
Interpretation | Nr | Name |
Success | 200 | OK |
Cross validation not found | 404 | Not Found |
Illegal model/algorithm/dataset/algorithm params | 400 | Bad request |
Validation/prediction error | 500 | Internal Server Error |
Document Actions
Validation - Report
Component description
A report visualizes the (prediction) results of algorithms.
REST operations
Description | Method | URI | Parameters | Result | Status codes |
Get all report types |
GET | /report | - | List of available report types |
200,404 |
Get all reports for the particular report type |
GET | /report/{report-type} | - | List of available reports as URI |
200,404 |
Retrieves a report, format according to accept-header | GET | /report/{report-type}/{id} | - | According to accept-header: text/x-yaml, application/rdf+xml -> report object with meta data (see rdf example below) text/xml, text/html, application/pdf -> report document in specified format |
200,404 |
Creates a report |
POST | /report/{report-type} | various params, see below |
Report URI or Task URI | 200,400,404,500 |
Deletes a report. | DELETE | /report/{report-type}/{id} | - | - |
200,404 |
Available (validation-)report types |
|||||
Create ToxPredict report (multiple models, one compound to predict) |
POST |
/report/toxpredict | List of validation URIs |
Report URI or Task URI | 200,400,404,500 |
Create single validation report (one model, one test dataset) |
POST |
/report/validation | validation URIs |
Report URI or Task URI | 200,400,404,500 |
Create cross-validation report (crossvalidation with one algorithm and one dataset) |
POST |
/report/crossvalidation | crossvalidation URIs |
Report URI or Task URI | 200,400,404,500 |
Create report for comparing different prediction algorithms (crossvalidations/validations with multiple algorithms and datasets) |
POST |
/report/algorithm_comparison | List of crossvalidation URIs or list of validation URIs |
Report URI or Task URI | 200,400,404,500 |
Create report for comparing different models | POST |
/report/model_comparison | list of validation URIs | Report URI or Task URI | 200,400,404,500 |
Special report formats |
|||||
Create QMRF report *) |
POST |
/reach_report/qmrf | application/x-form-www-urlencoded model_uri=Model URI or application/qmrf-xml for creating a report with predefined QMRF XML content |
Report URI or Task URI | 200,400,404,500 |
Replaces QMRF report |
POST |
/reach_report/qmrf/{reportid} |
application/qmrf-xml for creating a report with predefined QMRF XML content |
Report URI | 200,400,404,500 |
Update partially QMRF report |
POST | /reach_report/qmrf/{reportid} | validation_uri = a List of crossvalidation URIs and/or validation URIs of the same model <report_section (as defined in qmrf.dtd)> = content as string |
Report URI or Task URI | 200,400,404,500 |
Delete QMRF report |
DELETE |
/reach_report/qmrf/{reportid} | deletes the report |
||
Retrieves the report |
GET |
/reach_report/qmrf/{reportid} |
retrieves the report |
representation, , format specified by MIME type (XML, RDF, HTML, PDF, XLS, where applicable) | |
Start qmrf editor with report |
GET |
/reach_report/qmrf/{reportid}/editor | - |
return jnlp, starts QMRF editor as Java webstart application |
200,404 |
Create QPRF report *) |
POST |
/reach_report/qprf | application/x-form-www-urlencoded model_uri = Model URI One of { dataset_uri = Dataset URI compound_uri = compound uri }, specifying the compounds or application/qprf-format-to-be-defined for creating a report with predefined QPRF content |
Report URI or Task URI | 200,400,404,500 |
Replaces QPRF report |
POST |
/reach_report/qprf/{reportid} | same as above, replaces the content |
Report URI |
|
Updates QPRF report |
PUT |
/reach_report/qprf/{reportid} | same as above, but adds new content to the report |
Report URI |
|
Deletes QPRF report |
DELETE |
/reach_report/qprf/{reportid} |
deletes the report |
||
Retrieves the report content |
GET |
/reach_report/qprf/{reportid} | retrieves the report |
representation, , format specified by MIME type (XML, RDF, HTML, PDF, XLS, where applicable) | |
Report searching facilities |
GET |
/reach_report/{type} |
application/x-form-www-urlencoded any or subset of model_uri = Model URI dataset_uri = Dataset URI compound_uri = Compound URI algorithm_uri = Algorithm URI endpoint_uri= endpoint URI, as defined by the ontology search=any free text, etc. |
Retrieves list of reports, related to the model, specified by any of the parameter URI |
*) see: http://ecb.jrc.ec.europa.eu/qsar/qsar-tools/index.php?c=QRF. Both report formats, QMRF and QPRF, need some GUI functionality to fill in the additional information. The QMRF Editor http://ambit.sourceforge.net/qmrf/jws/qmrfeditor.jnlp could probably be used for the QMRF report.
Output (file) format
Validation reports are returned in 2 different ways according to the accept-header
- Accept header is text/xml, text/html, application/pdf: the actual report document is returned (the core document is an XML format according to the DocBook standard, which allows conversion to the other formats).
- Accept header is application/rdf+xml or text/x-yaml: the report object is returned, containing meta-data. Example (in N3) below
@prefix : <http://www.opentox.org/api/1.1#> . @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . <http://opentox.informatik.uni-freiburg.de/validation/report/validation/8> a :ValidationReport; :date "Wed Jun 09 13:23:28 +0200 2010"^^<http://www.w3.org/2001/XMLSchema#dateTime>; :reportAlgorithm <http://ambit.uni-plovdiv.bg:8080/ambit2/algorithm/LR>; :reportModel <http://ambit.uni-plovdiv.bg:8080/ambit2/model/359152>; :reportType "validation"^^<http://www.w3.org/2001/XMLSchema#string>; :reportValidation <http://opentox.informatik.uni-freiburg.de/validation/20> .
HTTP status codes
Interpretation | Nr | Name |
Success | 200 | OK |
Report type / report not found | 404 | Not Found |
Illegal params | 400 | Bad request |
Error creating the report |
500 | Internal Server Error |