A total of 348 DNA sequences 16-30 mers long with experimental Tm, salt and oligonucleotide concentrations available were used in this benchmark: 37 sequences were obtained from the work of Owczarzy et al., 1998; 11 sequences were obtained from NTDB database (Chiu et al., 2003); and the remaining 300 sequences were obtained from Owczarzy et al., 2004. The full table containing all the experimental values and the theoretical predictions by using the various methods is available here. The melting temperatures were predicted with the basic method (BAS), the salt adjusted method (SAL), and the NN model with the thermodynamic parameters of Breslauer et al., 1986 (BRE), SantaLucia et al., 1986 (SAN), and Sugimoto et al., 1986 (SUG). The melting temperature was also predicted using the consensus method (CON) proposed by us (Panjkovich and Melo), which is based on the results obtained in the comparative study and shown in the following Figure. The consensus melting temperature corresponds to the average melting temperature of those methods that exhibit similar results at a given grid point of the oligonucleotide feature space. In those cases where no similarities are observed among methods (black regions of the Figure), the average of all melting temperature values was used (top values within each cell). The results of the benchmark using the 281 sequences that map in zones 1, 2 and 3 (excluding those sequences from the black regions in the Figure) are shown between parentheses (bottom values within each cell). Four different accuracy measures are reported in this table. First, the percentage of cases where the method gives the closest prediction to the experimental Tm (BEST); then the percentage of cases where the method gives a prediction within 5 and 3 Celcius degrees from the experimental Tm (ERROR); and finally, the average of the absolute differences between the method prediction and the experimental Tm for all the cases considered (AVE. ERROR).

Accuracy Measure BAS SAL BRE SAN SUG CON
BEST (%) 0.57
(0.00)
5.17
(6.41)
3.45
(2.14)
40.80
(38.08)
26.15
(27.05)
23.85
(26.33)
ERROR < 5° C (%) 11.21
(9.61)
31.03
(35.23)
26.15
(24.56)
83.33
(83.99)
83.62
(84.34)
83.91
(86.12)
ERROR < 3° C (%) 3.74
(2.85)
14.94
(17.44)
14.37
(12.46)
60.92
(57.30)
60.06
(62.63)
61.49
(64.06)
AVE. ERROR (° C) 12.28
(12.56)
7.11
(6.69)
8.45
(8.60)
2.85
(2.98)
2.86
(2.73)
2.84
(2.65)
 

Breslauer,K.J., Frank,R., Blöcker,H. and Marky,L.A. (1986) Predicting DNA duplex stability from the base sequence. Proc. Natl. Acad. Sci. USA, 83, 3746-3750.
SantaLucia,J.,Jr, Allawi,H.T. and Seneviratne,P.A. (1996) Improved nearest-neighbor parameters for predicting DNA duplex stability. Biochemistry, 35, 3555-3562.
Sugimoto,N., Nakano,S., Yoneyama,M. and Honda,K. (1996) Improved thermodynamic parameters and helix initiation factor to predict stability of DNA duplexes. Nucleic Acids Res., 24, 4501-4505.
Owczarzy,R., Vallone,P.M., Gallo,F.J., Paner,T.M., Lane,M.J. and Benight,A.S. (1998) Predicting sequence-dependent melting stability of short duplex DNA oligomers. Biopolymers, 44, 217-239.
Chiu,W.L., Sze,C.N., Ma,N.T., Chiu,L.F., Leung,C.W. and Au-Yeung,S.C. (2003) NTDB: Thermodynamic Database for Nucleic Acids, Version 2.0. Nucleic Acids Res., 31, 483-485.
Owczarzy,R., You,Y., Moreira,B.G., Manthey,J.A., Huang,L., Behlke,M.A. and Walder,J.A. (2004) Effects of sodium ions on DNA duplex oligomers: improved predictions of melting temperatures. Biochemistry, 43, 3537-3554.
Panjkovich, A. and Melo, F. (2004) Comparison of DNA melting temperature calculation methods for short DNA sequences. Bioinformatics (in press).

 
dnaMATE