projects techreports press lab location staff
citi top.2 top.3
citi mid.3
bot.1 bot.2 bot.3

Projects: ASCI

ASCI Cluster Information

Node Chart

Fast Ethernet Nodename Fast Ethernet IP Gigabit Nodename Gigabit IP Same Same Same Same

Iota Hardware

  • Dual 1.7 GHz Pentium 4 processors with a 256 KB cache
  • 2 GB of RAM
  • Seagate 80GB, 7200 RPM hard drive with an Ultra ATA/100 interface and a 2 MB cache (Nodes 5-8 have an additional 120GB drive)
  • 3Com 3C996B-T gigabit card (Nodes 6-8 have 2 cards) (64Mbit/66Mhz)
  • PCI-X(133Mhz) risers for the gigabit cards
  • 2 64bit/66Mhz PCI slots
  • Front Side Bus 400Mhz

LLNL Hardware

  • 1.4 GHz Intel Xeon processors with a 256 KB cache
  • 1GB RAM
  • Adaptec 40GB, 10K RPM SCSI hard drive using Ultra 160
  • AceNIC gigabit Ethernet card (64Mbit/66Mhz)
  • Myrinet card running at 66Mhz
  • Note: I'm not sure if the cards are 66Mhz, or if it is the bus, or if only 1 card is 66Mhz and the other is 133Mhz. The only way to find out is to open up the machine. Type 'lspci -vvv' for more info.


  • All iota and llnl machines are currently running redhat 9 with various kernels


  • IntraCore 35160 gigabit switch


  • APC 16 Port Powerbar
  • URL:
  • IP:


  • IP:
  • To connect type: ssh username:iota<#>@ccc


  • Currently, GPFS is installed over iSCSI to fas960-1 on iota4-8 on sda
  • I used the following script to install the rpms
  • It is currently set for testing, not performance as the block size is 16KB and the pagepool is only 32MB.
  • I used the following script to create it
  • You can use the following script script to delete it

Using the machines

  • Type 'mount /nfs' to retrieve plymouth home directories
  • Here are sample iota .config files:2.4 2.6
  • Here are sample llnl .config files:2.4 2.6
  • Use 'authconfig' to set level of security access (Kerberos,etc)
  • For test purposes, if you want to control the CPU frequency, you must enable 'CPU Frequency Scaling' in the kernel and then modify the value in /proc/sys/cpu/0/speed
  • 'hdparm /dev/hdX' adjusts disk parameters. Need to ensure DMA, etc are enabled. 'hdparm -[t|T] /dev/hdaX' (noncached/cached) will perform tests.
  • The best driver for the 3Com cards is the Broadcom BCM5701 driver (not the Tg3 driver). It is available here
    • This driver is currently installed as a module. A patch exists for the 2.6.0 kernel which should work with later kernels, but modules are easier with the rapid number of kernels used in testing.
    • To manually install a module, you must first create it (.o for 2.4 and .ko for 2.6), put it in /lib/modules/'uname -r'/kernel/drivers/net/ and then add it to the /lib/modules/'uname -r'/modules.dep file. The eth1/eth2 module must then be modified in the /etc/modules.conf (for 2.4) or /etc/modprobe.conf (for 2.6)
  • Some very basic kernel profiling help. I recommend reading the README too.
  • Some useful performance monitoring tools are:
    • 'top' -individual processor statistics and memory
    • 'sar' -Interrupts(-I), network(-n DEV), per cpu (-U ALL or -P ALL)
    • 'vmstat' -A wonder tool
    • 'iostat' -x is useful for disk info
  • Setting up the serial port
  • ISCSI 3.4.1 is installed on iota4-8. Use /etc/init.d/iscsi stop/start. The config file /etc/iscsi.conf is here.
  • A sample /etc/sysconfig/network-scripts/ifcfg-eth1 file.
  • /usr/local/bin/iozone_ssh is installed on each machine. It is the same as iozone, but uses ssh instead of rsh for throughput tests
  • Here is a lilo.conf file for iota4-8
  • Here is a lilo.conf file for iota1-3
  • Here is a lilo.conf file for llnl1 machines
  • Here is a lilo.conf file for llnl2 machines
  • If you have comments or suggestions about our NFSv4 work with ASCI, email asci @ projects | techreports | press | lab | location | staff Email address
    or call +1 734 763 2929
    Copyright © 1996-2013
    The Regents of the University of Michigan