AMBER 7
The benchmarks below are for Amber 7 MD simulations of DHFR (Joint Amber Charmm Benchmark and Liver Alcohol Dehydrogenase).
If you cannot reproduce these numbers and find the scaling worse than you expected the following information may be of help.
LADH job = 12A cutoff, ca 76000 atoms, PME, 1fs step, no shake, TIP3P water
DHFR Job = Direct from the Amber7 benchmarks directory
An excel spreadsheet with the numerical data is here.
LADH
DHFR-JAC
The hardware we are using is as follows:
Redhat 7.3, kernel 2.4.20
Dual Athlon 2200MP chips
8 machines (16 cpus)
512MB DDR PC2100 ECC ram in each machine
Head node disk = Maxtor 80GB 7200rpm UDMA133 IDE
Motherboard = MSI K7D Master Dual AMD Athlon Mobo DDR
Network Cards = Scan Etan 64 bit gigabit adaptors - http://web6.scan.co.uk/Products/Info.asp?WPID=15143 (cheap :-))
These use the ns83820 module that is part of kernel 2.4.20.
Also, make sure you are using Cat5E cables and not just Cat5 since if the network card detects a Cat5 cable it will default to 100MBps instead of gigabit.
The switch is a 24 port 3COM gigabit switch - SuperStack 3 - 4924 http://www.3com.com/prod/en_UK_EMEA/detail.jsp?tab=features&pathtype=purchase&sku=3C17701-US
The important thing with this switch is that it is non-blocking across all ports. A lot of the cheaper gigabit switches on the market only have a 10GBps back plane and so not all ports can talk to each other at full speed at the same time. Some things you might want to check / try.
1) Make sure you compile Amber 7 with -DRLE and -DROWAT
2) Recompile LAM MPI from source using the intel compilers
3) Optimise Head node harddrive options with hdparm (you can test the speed with "hdparm -tT /dev/hda") I use the following in rc.local:
#c1 = 32bit
#d1 = DMA
#S0 = No Power Down
#W1 = Write Cache
#X70 = UDMA 133
#u1 = multiple IRQ mode
#m16 = 16Kb prefetch
hdparm -c1 -d1 -S0 -W1 -u1 -m16 -X70 /dev/hda >/dev/null
4) Increase the network send and receive buffers (in rc.local):
echo "262144" >/proc/sys/net/core/wmem_max
echo "262144" >/proc/sys/net/core/rmem_max
echo "131072" >/proc/sys/net/core/wmem_default
echo "131072" >/proc/sys/net/core/rmem_default