BioInfo.PL conducts in collaboration with other partners several evalution experiments of structure prediction methods. Methods must be completely automated and must be accecible for the Structure Prediction Meta Server to participate in the experiments. The main programs are:

The Live Bench Project is a continuous benchmarking program. Not more than 6 hours after release new PDB proteins are submitted to participating fold recognition servers. The results are collected and continuously evaluated using automated model assessment programs. Result are calculated weekly and a summary is produced after several months of data collection.

The ToolShop program offers the possibility to compare a fold recognition server with other standard servers before releasing it to the public. The targets used in previous LiveBench experiments are used for this test and the results of the evaluated server are compared with previously obtained results. Develeopers are asked to contact us if they whish to conduct an assessment experiment. Only servers which indicate clearly the template used for structure prediciton are eligible.

The CAFASP (Critical Assessment of Fully Automated Structure Prediction) experiment is conducted in parallel with the biannual CASP experiment. In contrast to LiveBench it is a blind experiment. However the test set is smaller and the assesment of the predictions is available months after the predictions are collected. The main advantage over LiveBench is that the automated predictions can be compared with manual predictions made by human experts.

The PDB-Cafasp experiment is a hybrid between LiveBench and CAFASP. It submits the pre-released sequences available from the PDB as targets once a month. Predictions can be treated as blind and the sample size is larger than in the case of CASP/CAFASP. Evaluation is conducted periodically when the coordinates of the proteins are released in PDB. EVA is a related experiment which also provides secondary structure prediction assessment.

If You whish to register a server for the evaluation experiments the server should be coupled to the Structure Prediction Meta Server first. Please contact us so we can help you futher.