Sections
You are here: Home » Development » API » API 1.1 - archived version » Validation

Validation

— filed under: , ,

Related links

 
Examples: a how-to with curl calls  http://opentox.informatik.uni-freiburg.de/validation/examples
Background information: validation workflow  Validation and Reporting Overview and Data Flow
Technical details: statistics computed by the validation Validation statistics
 

 

Validation

 

Component description

A validation corresponds to the validation of a model on a test dataset. The results are stored in another dataset. Parameters with default values are optional.

REST operations

Description Method URI Parameters Result Status codes
Get all validations
GET / - List of validation URIs 200,404
Retrieves a validation representation GET /{id} - Validation representation in one of the supported MIME types
200,404
Validates a model on a test dataset
POST / model_uri
test_dataset_uri
test_target_dataset_uri (default = test_dataset_uri)
Validation URI or Task URI
200,400,404,500
Builds a model on a training dataset and validates it on a test dataset
POST
/ algorithm_uri
prediction_feature
algorithm_params (string, default="")
training_dataset_uri
test_dataset_uri
test_target_dataset_uri (default = test_dataset_uri)
y_scramble (boolean, default=false)
y_scramble_seed (integer, default=1)
Validation URI or Task URI 200,400,404,500
Splits a dataset into training and test dataset according to a certain ratio, and performs a validation
POST /training_test_split algorithm_uri
prediction_feature
algorithm_params (string, default="")
dataset_uri
split_ratio(float, default=0.66)
random_seed(integer, default=1)
y_scramble (boolean, default=false)
y_scramble_seed (integer, default=1)
Validation URI or Task URI 200,400,404,500
OPTIONAL:
Performs a bootstrap validation
POST /bootstrap algorithm_uri
prediction_feature

dataset_params (string, default="")
dataset_uri
bootstrap_percentage(float, default=0.66)
random_seed(integer, default=1)
y_scramble (boolean, default=false)
y_scramble_seed (integer, default=1)
Validation URI or Task URI 200,400,404,500
Deletes a validation. DELETE /{id} - -
200,404

Validation representation

  • Requested MIME type should be set in the requests "Accept" header, e.g. curl -X GET -H "Accept:application/xml" http://{server}/validation/{id}
  • RDF representation defined in opentox.owl
  • RDF Examples

Supported MIME types:

Mandatory:

  • application/rdf+xml (default)

Optional:

HTTP status codes

Interpretation Nr Name
Success 200 OK
Validation not found 404 Not Found
Illegal model/algorithm/dataset/algorithm params 400 Bad request
Validation/prediction error 500 Internal Server Error

  

Cross-Validation

Component description

Performs a k-fold cross-validation, which results in k validations. Parameters with default values are optional.

REST operations

Description Method URI Parameters Result Status codes
Get all cross-validations
GET /crossvalidation - List of crossvalidation URIs 200,404
Retrieves a cross-validation representation GET /crossvalidation/{id} - Cross-Validation in one of the supported MIME types
200,404
Returns all (k) validations that belong to a crossvalidation
GET /crossvalidation/{id}/validations - List of validation URIs
200,404
Performs a k-fold cross-validation.
POST
/crossvalidation algorithm_uri
prediction_feature
algorithm_params (string, default="")
num_folds (integer, default=10)
random_seed (integer, default=1)
stratified (boolean, default=true)
y_scramble (boolean, default=false)
y_scramble_seed (integer, default=1)
Cross-Validation URI or Task URI 200,400,404,500
Performs a leave-one-out cross-validation.
POST
/crossvalidation/loo algorithm_uri
prediction_feature
algorithm_params (string, default="")
y_scramble (boolean, default=false)
y_scramble_seed (integer, default=1)
Cross-Validation URI or Task URI 200,400,404,500
Deletes a cross-validation. DELETE /crossvalidation/{id} - -
200,404

Cross-validation representation

  • Requested MIME type should be set in the requests "Accept" header, e.g. curl -X GET -H "Accept:application/xml" http://{server}/crossvalidation/{id}

Supported MIME types:

Mandatory:

  • application/rdf+xml (default)

Optional:

HTTP status codes

Interpretation Nr Name
Success 200 OK
Cross validation not found 404 Not Found
Illegal model/algorithm/dataset/algorithm params 400 Bad request
Validation/prediction error 500 Internal Server Error
 
Document Actions


Validation - Report

Component description

A report visualizes the (prediction) results of algorithms.

REST operations

Description Method URI Parameters Result Status codes
Get all report types
GET /report - List of available report types
200,404
Get all reports for the particular report type
GET /report/{report-type} - List of available reports as URI
200,404
Retrieves a report, format according to accept-header GET /report/{report-type}/{id} - According to accept-header:
text/x-yaml, application/rdf+xml
-> report object with meta data (see rdf example below)

text/xml, text/html, application/pdf
-> report document in specified format
200,404
Creates a report
POST /report/{report-type} various params, see below
Report URI or Task URI 200,400,404,500
Deletes a report. DELETE /report/{report-type}/{id} - -
200,404
Available (validation-)report types





Create ToxPredict report
(multiple models, one compound to predict)
POST
/report/toxpredict List of validation URIs
Report URI or Task URI 200,400,404,500
Create single validation report
(one model, one test dataset)
POST
/report/validation validation URIs
Report URI or Task URI 200,400,404,500
Create cross-validation report
(crossvalidation with one algorithm and one dataset)
POST
/report/crossvalidation crossvalidation URIs
Report URI or Task URI 200,400,404,500
Create report for comparing different prediction algorithms
(crossvalidations/validations with multiple algorithms and datasets)
POST
/report/algorithm_comparison List of crossvalidation URIs
or list of validation URIs
Report URI or Task URI 200,400,404,500
Create report for comparing different models POST
/report/model_comparison list of validation URIs Report URI or Task URI 200,400,404,500
Special report formats





Create QMRF report  *)
POST
/report/qmrf application/x-form-www-urlencoded

model_uri
=Model URI

or
validation_uri =
a List of crossvalidation URIs
and/or validation URIs of the same model
additional fields of the report that cannot be filled out automatically (yet to be defined)

or application/qmrf-xml (smth lke)
for creating a report with predefined QMRF XML content
Report URI or Task URI 200,400,404,500
Update QMRF report
POST
/report/qmrf/{reportid}
same as above
Report URI 200,400,404,500
Delete QMRF report
DELETE
/report/qmrf/{reportid} deletes the report


Retrieves the report
GET
/report/qmrf/{reportid}
retrieves the report
representation, , format specified by MIME type (XML, RDF, HTML, PDF, XLS, where applicable)
Create QPRF report *)
POST
/report/qprf application/x-form-www-urlencoded

model_uri
= Model URI
validation_uri = a List of crossvalidation URIs and/or validation URIs of the same model
One of {
dataset_uri = Dataset URI
compound_uri = compound uri
}, specifying the compounds

or application/qprf-format-to-be-defined
for creating a report with predefined QPRF content
Report URI or Task URI 200,400,404,500
Replaces QPRF report
POST
/report/qprf/{reportid} same as above, replaces the content
Report URI

Updates QPRF report
PUT
/report/qprf/{reportid} same as above, but adds new content to the report
Report URI

Deletes QPRF report
DELETE
/report/qprf/{reportid}
deletes the report


Retrieves the report content
GET
/report/qprf/{reportid} retrieves the report
representation, , format specified by MIME type (XML, RDF, HTML, PDF, XLS, where applicable)
Report searching facilities
GET
/report/{type}
application/x-form-www-urlencoded
any or subset of
model_uri
= Model URI
dataset_uri = Dataset URI
compound_uri = Compound URI
algorithm_uri = Algorithm URI
endpoint_uri= endpoint URI, as defined by the ontology
search=any free text,

etc.
Retrieves list of reports, related to the model, specified by any of the parameter URI

*) see: http://ecb.jrc.ec.europa.eu/qsar/qsar-tools/index.php?c=QRF. Both report formats, QMRF and QPRF, need some GUI functionality to fill in the additional information. The QMRF Editor http://ambit.sourceforge.net/qmrf/jws/qmrfeditor.jnlp could probably be used for the QMRF report.

Output (file) format

Validation reports are returned in 2 different ways according to the accept-header

  • Accept header is text/xml, text/html, application/pdf: the actual report document is returned (the core document is an XML format according to the DocBook standard, which allows conversion to the other formats).
  • Accept header is application/rdf+xml or text/x-yaml: the report object is returned, containing meta-data. Example (in N3) below
@prefix : <http://www.opentox.org/api/1.1#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
    
<http://opentox.informatik.uni-freiburg.de/validation/report/validation/8>     
a :ValidationReport;
	:date "Wed Jun 09 13:23:28 +0200 2010"^^<http://www.w3.org/2001/XMLSchema#dateTime>;
        :reportAlgorithm <http://ambit.uni-plovdiv.bg:8080/ambit2/algorithm/LR>;
        :reportModel <http://ambit.uni-plovdiv.bg:8080/ambit2/model/359152>;
        :reportType "validation"^^<http://www.w3.org/2001/XMLSchema#string>;
        :reportValidation <http://opentox.informatik.uni-freiburg.de/validation/20> .


HTTP status codes

Interpretation Nr Name
Success 200 OK
Report type / report not found 404 Not Found
Illegal params 400 Bad request
Error creating the report
500 Internal Server Error

 

Document Actions

GET /report/{report-type}/{id}

Posted by Andreas Karwath at Feb 10, 2010 04:29 PM
I believe that this is inconsistent. One might think that report-type is XML, PDF, HTML, or RTF. However, it should be fasttox, validation, crossvalidation, algorithm_comparison, or model_comparison. So, therefore this method should have an 'output_format' parameter which can be one of the following: XML, PDF, HTML, or RTF

GET /report/{report-type}/{id}

Posted by Jeliazkova Nina at Feb 26, 2010 12:05 PM
The output format should be controlled by HTTP Accept header and corresponding MIME type, to be consistent with the REST style and the behaviour of other services.

Report(s) for a model

Posted by Jeliazkova Nina at Feb 26, 2010 12:09 PM
How does one could get reports for a given model?

What is the URL to retrieve a QMRF report for a given model? QMRF is not particularly a validation report, it is annotation of th emodel itself and could have multiple sections on validation with test datasets. How this is handled with the current API ?