OpenTox API 1.2 Validation

From ToxBank API Wiki
Revision as of 16:21, 29 August 2011 by Mrautenberg (Talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Contents

Related links

Examples: a how-to with curl calls http://opentox.informatik.uni-freiburg.de/validation/examples
Background information: validation workflow Validation and Reporting Overview and Data Flow
Technical details: statistics computed by the validation Validation statistics
Currently running validation web service
http://opentox.informatik.uni-freiburg.de/validation
Algorithm and Dataset Examples to validate

https://spreadsheets.google.com/spreadsheet/ccc?key=0AllBraSK8gV-dFgwUDJHaVJmQTlJRG1iTnJqNXBaZEE&hl=en_US&authkey=CJGPxMgB

Validation

Component description

A validation corresponds to the validation of a model on a test dataset. The results are stored in another dataset. Parameters with default values are optional.

REST operations

Description Method URI Parameters Result Status codes
Get all validations
GET / [subjectid] List of validation URIs 200,404
Retrieves a validation representation GET /{id} [subjectid] Validation representation in one of the supported MIME types
200,404
Validates a model on a test dataset
POST /test_set_validation [subjectid]
model_uri
test_dataset_uri
test_target_dataset_uri (default = test_dataset_uri)
prediction_feature (default = dependent variable of model)

[../AsyncTask#creating-a-task-post Validation URI or Task URI]

200,400,404,500
Builds a model on a training dataset and validates it on a test dataset
POST
/training_test_valdiation algorithm_uri
prediction_feature
algorithm_params (string, default="")
training_dataset_uri
test_dataset_uri
test_target_dataset_uri (default = test_dataset_uri)
y_scramble (boolean, default=false)
y_scramble_seed (integer, default=1)
[subjectid]

[../AsyncTask#creating-a-task-post Validation URI or Task URI]

200,400,404,500
Splits a dataset into training and test dataset according to a certain ratio, and performs a validation
POST /training_test_split algorithm_uri
prediction_feature
algorithm_params (string, default="")
dataset_uri
split_ratio (float, default=0.66)
random_seed (integer, default=1)
y_scramble (boolean, default=false)
y_scramble_seed (integer, default=1)
[subjectid]

[../AsyncTask#creating-a-task-post Validation URI or Task URI]

200,400,404,500
Performs a bootstrap validation
POST /bootstrapping algorithm_uri
prediction_feature
dataset_params (string, default="")
dataset_uri
bootstrap_percentage (float, default=0.66)
random_seed (integer, default=1)
y_scramble (boolean, default=false)
y_scramble_seed (integer, default=1)
[subjectid]

[../AsyncTask#creating-a-task-post Validation URI or Task URI]

200,400,404,500
Directly perform a validation by specifying test- and prediction dataset POST /validate_datasets prediction_feature
test_dataset_uri
test_target_dataset_uri (default = test_dataset_uri)
prediction_dataset_uri
predicted_feature (.i.e feature in prediction dataset)
[subjectid]

[../AsyncTask#creating-a-task-post Validation URI or Task URI]

200,400,404,500
Deletes a validation. DELETE /{id} [subjectid] -
200,404

Validation representation

  • Requested MIME type should be set in the requests "Accept" header, e.g. curl -X GET -H "Accept:application/xml" http://{server}/validation/{id}
  • RDF representation defined in [../../../../data/documents/development/RDF files/OpenToxOntology opentox.owl]
  • [../../../../data/documents/development/RDF files/Validation RDF Examples]
  • subjectid (optional) parameter that contains the OpenSSO A&A token needed to access protected services.

Supported MIME types:

Mandatory:

  • application/rdf xml (default)

Optional:

  • application/xml, see [../../../data/documents/development/validation/ValidationResult.xsd XML schema definition for validation object]

HTTP status codes

Interpretation Nr Name
Success 200 OK
Validation not found 404 Not Found
Illegal model/algorithm/dataset/algorithm params 400 Bad request
Validation/prediction error 500 Internal Server Error

Cross-Validation

Component description

Performs a k-fold cross-validation, which results in k validations. Parameters with default values are optional.

REST operations

Description Method URI Parameters Result Status codes
Get all cross-validations
GET /crossvalidation [subjectid] List of crossvalidation URIs 200,404
Retrieves a cross-validation representation GET /crossvalidation/{id} [subjectid] Cross-Validation in one of the supported MIME types
200,404
Returns all (k) validations that belong to a crossvalidation
GET /crossvalidation/{id}/validations [subjectid] List of validation URIs
200,404
Performs a k-fold cross-validation.
POST
/crossvalidation algorithm_uri
prediction_feature
algorithm_params (string, default="")
num_folds (integer, default=10)
random_seed (integer, default=1)
stratified (boolean, default=true)
y_scramble (boolean, default=false)
y_scramble_seed (integer, default=1)
[subjectid]

[../AsyncTask#creating-a-task-post Cross-Validation URI or Task URI]

200,400,404,500
Performs a leave-one-out cross-validation.
POST
/crossvalidation/loo algorithm_uri
prediction_feature
algorithm_params (string, default="")
y_scramble (boolean, default=false)
y_scramble_seed (integer, default=1)
[subjectid]

[../AsyncTask#creating-a-task-post Cross-Validation URI or Task URI]

200,400,404,500
Deletes a cross-validation. DELETE /crossvalidation/{id} [subjectid] -
200,404

Cross-validation representation

Supported MIME types:

Mandatory:

  • application/rdf xml (default)

Optional:

  • application/xml, see [../../../data/documents/development/validation/CrossValidation.xsd XML schema definition for crossvalidation object
    ]

HTTP status codes

Interpretation Nr Name
Success 200 OK
Cross validation not found 404 Not Found
Illegal model/algorithm/dataset/algorithm params 400 Bad request
Validation/prediction error 500 Internal Server Error
Document Actions

[#validation-report Validation - Report]

Component description

A report visualizes the (prediction) results of algorithms.

REST operations

Description Method URI Parameters Result Status codes
Get all report types
GET /report [subjectid] List of available report types
200,404
Get all reports for the particular report type
GET /report/{report-type} [subjectid] List of available reports as URI
200,404
Retrieves a report, format according to accept-header GET /report/{report-type}/{id} [subjectid] According to accept-header:
text/x-yaml, application/rdf xml
-> report object with meta data (see rdf example below)

text/xml, text/html, application/pdf
-> report document in specified format
200,404
Creates a report
POST /report/{report-type} various params, see below
[subjectid]

[../AsyncTask#creating-a-task-post Report URI or Task URI]

200,400,404,500
Deletes a report. DELETE /report/{report-type}/{id} [subjectid] -
200,404
Available (validation-)report types





Create single validation report
(one model, one test dataset)
POST
/report/validation validation_uris (comma separated list of validation URIs)
[subjectid]

[../AsyncTask#creating-a-task-post Report URI or Task URI]

200,400,404,500
Create cross-validation report
(crossvalidation with one algorithm and one dataset)
POST
/report/crossvalidation validation_uris (comma separated list of crossvalidation URIs)
[subjectid]

[../AsyncTask#creating-a-task-post Report URI or Task URI]

200,400,404,500
Create report for comparing different prediction algorithms
(crossvalidations/validations with multiple algorithms and datasets)
POST
/report/algorithm_comparison validation_uris (comma separated list of crossvalidation URIs)
identifier (comma seperated list of string-ids, one for each crossvalidation-uri)
[subjectid]

[../AsyncTask#creating-a-task-post Report URI or Task URI]

200,400,404,500

Output (file) format

Validation reports are returned in 2 different ways according to the accept-header

  • Accept header is text/xml, text/html, application/pdf: the actual report document is returned (the core document is an XML format according to the DocBook standard, which allows conversion to the other formats).
  • Accept header is application/rdf xml or text/x-yaml: the report object is returned, containing meta-data. Example (in N3) below
@prefix : <http://www.opentox.org/api/1.1#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .

<http://opentox.informatik.uni-freiburg.de/validation/report/validation/8>
a :ValidationReport;
	:date "Wed Jun 09 13:23:28  0200 2010"^^<http://www.w3.org/2001/XMLSchema#dateTime>;
        :reportAlgorithm <http://ambit.uni-plovdiv.bg:8080/ambit2/algorithm/LR>;
        :reportModel <http://ambit.uni-plovdiv.bg:8080/ambit2/model/359152>;
        :reportType "validation"^^<http://www.w3.org/2001/XMLSchema#string>;
        :reportValidation <http://opentox.informatik.uni-freiburg.de/validation/20> .

http://www.opentox.org/data/documents/development/validation/ValidationResult.xsd

HTTP status codes

Interpretation Nr Name
Success 200 OK
Report type / report not found 404 Not Found
Illegal params 400 Bad request
Error creating the report
500 Internal Server Error
Personal tools