LiveBench program has two main goals:

• The program provides simple evaluation of the structure prediction servers from the point of view of a potential user. The evaluation of sensitivity and specificity of the available servers can help the user develop sequence analysis strategies and assess the confidence of the obtained predictions.

• The program offers a simple weekly procedure for the prediction service providers, which can help to locate possible problems and tune the methods for best performance.

The main advantage of LiveBench is the fast evaluation cycle. The assessment is conducted maximum a week after release of targets. Each week provides several new targets that can be used to check differences in the performance of servers.

Latest LiveBench sets:

The detailed LiveBench procedure is as follows:

• Every 6 hours sequences of newly released proteins with known structure (targets) are downloaded from the Protein Data Bank (PDB).

• A target is skipped if it is shorter than 24 resiudes or longer than 400 residues.

• The sequences of the target is compared with the previous set of PDB proteins (templates) using BLAST. A target is skipped if there is a hit to any protein from the previous set with an E-value lower than 0.001 (this removes mainly trivial fold recognition targets).

• The targets are submitted to participating fold recognition servers using a server specific queueing system, which in most cases sends only one request per server at a time.

• The results from fold recognition servers are collected by the meta server and translated into uniform formats.

• The structure of the target protein may also be submitted to participating structure comparison servers to evaluate the structural similarity of the target with template proteins in the PDB database.

• Prediction results are translated into structural models (the Ca atoms of aligned residues of the template proteins are used as model). The native structure is compared with the generated models using various evaluation methods.