[NOTE: this document describes much of the installation and configuration required
[      by the NTAP portal, but it doesn't describe Kerberos realm setup, and much
[      of the LDAP information is elsewhere.


[i recommend a tab width of 3]
________________________________________________________________________________
0.  a big acknowledgement goes to andy adamson for writing the 
      original CITI portal setup document, a subset of which 
      provided the basis for this installation.
		
________________________________________________________________________________
1.	 starting out
	. beginning with dell 2650 with Red Hat FC1 installed, 
		openssl 0.9.7a, kerberos5 1.3.2.

	. download (source) and install OpenLDAP and its prerequisite, BerkeleyDB.  
		these are practically automatic.  we'll need both the `ldapsearch` 
		commandline utility and, if you'll also host your AdminRealm's LDAP
		directory on the webserver, all the rest of the stuff it installs :)
		please see the NTAP FAQ for information specific to setting up the
		LDAP directory [http://www.citi.umich.edu/projects/ntap]
		  
	. download apache 1.3.31
		[http://apache.towardex.com/httpd/apache_1.3.31.tar.gz]
		
	. download mod_ssl 2.8.18-1.3.31
		[http://www.modssl.org/source/mod_ssl-2.8.18-1.3.31.tar.gz]
		
	. download kx509
		[http://www.citi.umich.edu/projects/kerb_pki/kx509.dist20031111.tar.gz]
	
	. get the webserver example docroot bundle
		[http://www.citi.umich.edu/projects/ntap/webserver_docroot.tar.gz]
			
	. get the webserver modules bundle (see NOTE just below)
		[http://www.citi.umich.edu/projects/ntap/webserver_modules.tar.gz]
		
		NOTE: there are still some open questions about how CITI will go about
		distributing the source code to the webserver modules.  At this point
		in time (June 30, 2004 [dmr]), things are still up in the air.  Please
		contact CITI directly and we'll figure something out -- just until we
		get a more permanent solution.  Sorry for the hassle :)
		

	. get GARA 1.2.2 (CITI's modified version)
	
	. get Globus 2.4 resource manager (client-side) and GPT 3.0.1
		[http://www-unix.globus.org/ftppub/gt2/2.4/2.4-latest/bundles/src/globus-resource-management-client-2.4.3-src_bundle.tar.gz]
		[http://www-unix.globus.org/ftppub/gt2/2.4/2.4-latest/gpt/gpt-3.0.1-src.tar.gz]

	. get PHP 4.3.7 from a mirror off of:
		[http://www.php.net/get/php-4.3.7.tar.gz/from/a/mirror]
		e.g., http://us2.php.net/distributions/php-4.3.7.tar.gz
		
	. i put everything in /opt/ for this install, btw.
		. unpack apache.  it'll be e.g. /opt/apache_1.3.31, which i'll call $APACHE
		. unpack mod_ssl. it'll be e.g. /opt/mod_ssl-#.#.#-#.#.#, which i'll call $MODSSL
		. unpack kx509.   it'll be e.g. /opt/kx509, which i'll call $KX509
		. unpack webserver_modules.  now, do:
			% cd /opt && mv webserver_modules/* $APACHE/src/modules/
			
			that puts grid/, kct/, and kx509/ into apache's src/modules/ dir.
			
			
________________________________________________________________________________
2.  modules makefile template
	. this is lifted effectively-verbatim from andy's document:

		this next step is used simply to have the apache build create a makefile
		in $APACHE/src/modules/extra, which can then be used as a template for
		the kct, kx509, and grid module Makefiles. after the apache build, the mod_kct.c
		copied into the modules/extra directory will be removed.

		% cp $APACHE/src/modules/kct/mod_kct.c $APACHE/src/modules/extra


________________________________________________________________________________
3.  webserver host certificate and privkey
	. create the host cert and have it signed by your KCA.
	. i'll refer to these files as hostcert.pem and hostkey.pem and they
	   should be put in $APACHE with perms 444 and 400, respectively.
		

________________________________________________________________________________
4.  build mod_ssl
	. cd $MODSSL
	. ./configure --with-apache=$APACHE --prefix=$APACHE \
	  --add-module=$APACHE/src/modules/extra/mod_kct.c \
	  --enable-module=kct --enable-shared=kct --enable-shared=ssl \
	  --with-crt=$APACHE/hostcert.pem \
	  --with-key=$APACHE/hostkey.pem
	  
	. now, `configure` will have TOLD you the following:
			Now proceed with the following commands (Bourne-Shell syntax):
			 $ cd /opt/apache_1.3.31
			 $ SSL_BASE=/path/to/openssl ./configure ... --enable-module=ssl
			 $ make
			 $ make install
	
	. cd $APACHE		 
	. ./configure --with-layout=Apache --prefix=$APACHE \
	  --add-module=$APACHE/src/modules/extra/mod_kct.c \
	  --enable-module=kct --enable-shared=kct --enable-shared=ssl \
	  --enable-module=ssl
	. make
	
	. the make should FAIL.  that's okay.  we're really setting up modules now.
		. rm $APACHE/src/modules/extra/mod_kct.c  (we just wanted its Makefile generated)
		. vi $APACHE/src/modules/extra/Makefile   (remove all mod_kct references)
		. cp $APACHE/src/modules/extra/Makefile $APACHE/src/modules/kct/
		. cp $APACHE/src/modules/extra/Makefile $APACHE/src/modules/kx509/
		
		
________________________________________________________________________________
5.  build mod_kct:
	. pushd $APACHE/src/modules/kct
	. vi Makefile:
		from
		EXTRA_INCLUDES=
		to
		EXTRA_INCLUDES= -I/usr/kerberos/include -I/opt/apache_1.3.31/src/modules/ssl

		from 
		CFLAGS=$(OPTIM) $(CFLAGS1) $(EXTRA_CFLAGS)
		to
		CFLAGS=$(OPTIM) $(CFLAGS1) $(EXTRA_CFLAGS) $(SSL_CFLAGS)

	. make
		

________________________________________________________________________________
6.  build mod_kx509		
	. first, we'll have to build libkx509.so.
		. cd $KX509
		. src/configure --enable-libkx509
			. mine couldn't find krb5.h or libkrb, so i had to add
				the flag  --with-krb5=/usr/local

		. make
			. mine had an error because a placeholder ("@EXTRA_KX_LIBS@")
				was hanging around in /opt/kx509/client/Makefile.  removed it.
			. a second `make` worked.

		. sudo cp src/client/libkx509.so /usr/lib	

	. now, mod_kx509 will want to use that.
		. pushd $APACHE/src/modules/kx509	
		. vi Makefile:
			from
			SHLIBS= mod_kct.so
			OBJS_PIC= mod_kct.lo

			to
			SHLIBS= mod_kx509.so
			OBJS_PIC= mod_kx509.lo


			from
			EXTRA_INCLUDES=

			to
			EXTRA_INCLUDES= -I/usr/kerberos/include -I/opt/kx509/src/lib


			then add to LIBS_SHLIB:
			LIBS_SHLIB= -L/usr/kerberos/lib -lkrb5 -lcom_err -lk5crypto -lssl -lcrypto -lkx509 -lresolv

		. make
		. ldd mod_kx509.so (just to make sure yours worked right)

________________________________________________________________________________
7.  need Globus 2.4 (libs) so we can build the grid module.		

	. first, we have to build GPT, which handles building Globus stuff.
		. unpack GPT into /opt
		. export GPT_LOCATION=/opt/gpt-3.0.1
			. this isn't for convenience -- GPT needs this variable
		. cd $GPT_LOCATION
		. ./build_gpt		
		
	. next, create the Globus directory
		. mkdir /opt/globus-2.4
		. export GLOBUS_LOCATION=/opt/globus-2.4
			. this isn't for convenience -- GPT needs this variable

	. assuming that the Globus resource manager tarball is in /opt/ ...
		. cd /opt
		. $GPT_LOCATION/sbin/gpt-build globus-resource-management-client-2.4.3-src_bundle.tar.gz gcc32dbgpthr
			. the "gcc32dbgpthr" is the Globus "flavor"; the build will take awhile
		. $GPT_LOCATION/sbin/gpt-postinstall
		. if you don't have a /etc/grid-security directory, run the setup-gsi
			script that gpt-postinstall tells you to do, but say "yes" to proceeding
			and then quit at the next menu.  this might be superfluous for the webserver.
		
	. there should now be a ton of libs in $GLOBUS_LOCATION/lib/  
	. we'll come back to those
			
________________________________________________________________________________
8.  also need GARA 1.2.2 so we can build the grid module.
		
		. actually, we only need libglobus_gara_client.so, so we don't need,
			e.g., KeyNote. 
		. checked out from cvs; /opt/gara-1.2.2
		. moved stock libtool out of the way
		. moved /opt/gara-1.2.2/libtool into my path
		. need the following older tools to build gara without big hassles (don't
			forget that these tools have helper dirs in /usr/share/autoconf, e.g.,
			that need to match the version of the script you're using):
			. libtool  1.4.2
			. autoconf 2.13
			. automake 1.4-p6
			. aclocal  1.4??  -- this might just mirror the version of automake?  
	
		. this should make things "easier":
			(sh)  export CFLAGS="-DGENERIC_RSL -DCLIENT_INFO -DCHECK_AUTHORIZATION -DRES_CHECK -DQUATER_TIMER"

		. make sure that the configure script will be able to find some Globus utils; i'm lazy:
			(sh)  alias doglobus-2.4='GLOBUS_LOCATION=/usr/local/src/globus-2.4; \
			                          GPT_LOCATION=/usr/local/src/gpt-3.0.1; \
											  export GLOBUS_LOCATION GPT_LOCATION;'

		  ... then i just do 'doglobus-2.4' all the time.

		. did the configure:
			./configure --prefix=/opt/gara-1.2.2 --with-globus-prefix=/opt/globus-2.4 \
			  --with-globus-flavor=gcc32dbgpthr --enable-lram_diffserv --enable-rm_diffserv \
			  --enable-gara_diffserv --enable-rm_network --enable-slot_manager

		. the version in CVS should "do the good thing" now.
		. cd $GARA/gara/libraries
		. make
		
________________________________________________________________________________
9.  build the apache grid modules

	. cd $APACHE/src/modules/grid
	. tailor the file Makefile.mod_grid_init for your system and build 
		mod_grid_init.so with it.
		
	. tailor the file Makefile.globus_client (needs the Globus-2.4 source dir,
		the GARA source dir, maybe extra stuff and libglobus_gara_client.so 
		for sure) and build globus_client with it.
		
			. may need to try (remember $GLOBUS is just my name for where 
				you installed it):
					GLOBUS_SRC=$GLOBUS 
					GLOBUS_INCLUDES=-I$(GLOBUS_SRC)/include/gcc32dbgpthr

	. cp $APACHE/src/modules/grid/globus_client      $APACHE/bin/
	. cp $APACHE/src/modules/grid/mod_grid_init.so $APACHE/libexec/
	. cp $APACHE/src/modules/kx509/mod_kx509.so    $APACHE/libexec/
	. cp $APACHE/src/modules/kct/mod_kct.so        $APACHE/libexec/
	

________________________________________________________________________________
10.  the apache webserver  :)

	. cd $APACHE
	. keeping in mind your configure settings from before, do a make install  
	
	. now, you'll need to set up a directory, $APACHE/.globus/ (note period),
		that contains the self-signed KCA cert -- a sample .globus directory
		(with the cert for CITI's KCA) is included in the webserver tarball.

________________________________________________________________________________
11.  php stuff

	. i unpacked into /opt, making /opt/php-4.3.7; call it $PHP
	. cd $PHP
	. ./configure --prefix=$PHP --with-apxs=$APACHE/bin/apxs
	. make && make install
	. drop libphp4.so into $APACHE/libexec if PHP didn't already do it for you.

	... actually, on the new flash installation, i configured with:
	
	./configure  --enable-sockets --with-mysql --with-apxs=/usr/local/apache_1.3.31/bin/apxs --with-ldap
	
	
________________________________________________________________________________
12.  docroot

	. inside the webserver_docroot tarball, there should be a directory called 
		'root' -- it contains some PHP script samples from our test environment at
		CITI.  drop its contents into whatever directory's apache's document root.

	. edit the file from that tarball called root/contants.inc and make sure
		WEB_PREFIX, PARAM_PREFIX, and SHELL_COMMAND are good.

________________________________________________________________________________
13.  httpd.conf

	. this step has several parts; for those conversant with httpd.conf, this 
		may be too much detail.  i needed the help ;)  anyway, the following is
		copied almost verbatim (with permission) from Andy Adamson:
		

	. add the new modules:

		# Example:
		# LoadModule foo_module libexec/mod_foo.so
		LoadModule php4_module         libexec/libphp4.so
		LoadModule grid_init_module    libexec/mod_grid_init.so
		LoadModule kx509_module        libexec/mod_kx509.so
		LoadModule kct_module          libexec/mod_kct.so
		<IfDefine SSL>
		LoadModule ssl_module          libexec/libssl.so
		</IfDefine>

	
	. now, just below the above "LoadModule" section in httpd.conf, find the
		"AddModule" section and append this to the end (the ordering of the
		modules matters):
		
		AddModule mod_php4.c
		AddModule mod_grid_init.c
		AddModule mod_kx509.c
		AddModule mod_kct.c
		<IfDefine SSL>
		AddModule mod_ssl.c
		</IfDefine>


	. remove port 80 from SSL:

		# Port: The port to which the standalone server listens. For
		# ports < 1023, you will need httpd to be run as root initially.
		#
		#Port 80

		##
		##  SSL Support
		##
		##  When we also provide SSL we have to listen to the
		##  standard HTTP port (see above) and to the HTTPS port
		##
		<IfDefine SSL>
		#Listen 80
		Listen 443
		</IfDefine>


	. set document root to point to the root created in step [12 above]

		# DocumentRoot: The directory out of which you will serve your
		# documents. By default, all requests are taken from this directory, but
		# symbolic links and aliases may be used to point to other locations.
		#
		DocumentRoot "/opt/apache_1.3.31/root"

	... and ...

		#
		# This should be changed to whatever you set DocumentRoot to.
		#
		<Directory "/opt/apache_1.3.31/root">


	. we added this just above the SSL Global Context... this disables the 
		following modules for port 80.

		##
		## HTTP Virtual Host Context
		##
		## - turn off SSL as well as our modules
		<VirtualHost _default_:80>
		<IfModule mod_ssl.c>
		SSLEngine               off
		</IfModule>
		<IfModule mod_kct.c>
		kctengine               off
		</IfModule>
		<IfModule mod_kx509.c>
		kx509engine             off
		</IfModule>
		<IfModule mod_grid_init.c>
		gridengine              off
		</IfModule>
		</VirtualHost>


	. add the module configuration under SSL Virtual Host Context:

		##
		## SSL Virtual Host Context
		##

		<VirtualHost _default_:443>

		<IfModule mod_kct.c>
		kctengine               on
		kctverifyclient         on
		kctkeytab               /opt/apache_1.3.31/conf/foo.web_server.keytab
		kctwsname               web_server/foo.citi.umich.edu
		kctservicename          kca_service/babble.citi.umich.edu
		kcttgsservicename       kct
		kctkdcfullname          krbtgt/CITI.UMICH.EDU
		kcttgsserverhostname    babble.citi.umich.edu
		kcttgsserverport        5039
		kcttktcachename         /tmp/web_server
		kcttgslifetime          10h
		kctrealm                CITI.UMICH.EDU
		kctclientcacheprefix    FILE:/tmp/modkct_
		</IfModule>

		<IfModule mod_kx509.c>
		kx509engine             on
		kx509ccstore            on
		kx509certfileprefix     /tmp/x509_proxy_
		kx509keyfileprefix      /tmp/x509_proxy_
		kx509certkeyfileprefix  /tmp/x509_proxy_
		</IfModule>

		<IfModule mod_grid_init.c>
		gpiExecname             /opt/globus-2.4/bin/grid-proxy-init-bin
		gridengine              on
		X509_CERT_DIR           .globus
		X509_CERT_FILE          .globus/078314af.0
		X509_USER_PROXY         /tmp/x509_proxy_
		</IfModule>


	. general setup for the virtual host:

		DocumentRoot "@@ServerRoot@@/root"
		ServerName foo.citi.umich.edu
		ServerAdmin bar@citi.umich.edu
		ErrorLog logs/error_log
		TransferLog logs/access_log


	. uncomment the SSLCACert file:

		#   Certificate Authority (CA):
		#   Set the CA certificate verification path where to find CA
		#   certificates for client authentication or alternatively one
		#   huge file containing all of them (file must be PEM encoded)
		#   Note: Inside SSLCACertificatePath you need hash symlinks
		#         to point to the certificate files. Use the provided
		#         Makefile to update the hash symlinks after changes.
		#SSLCACertificatePath @@ServerRoot@@/conf/ssl.crt
		SSLCACertificateFile @@ServerRoot@@/conf/ssl.crt/ca-bundle.crt


	. require client authenitcation:

		#   Client Authentication (Type):
		#   Client certificate verification type and depth.  Types are
		#   none, optional, require and optional_no_ca.  Depth is a
		#   number which specifies how deeply to verify the certificate
		#   issuer chain before deciding the certificate is not valid.
		SSLVerifyClient require
		SSLVerifyDepth  10


	. handle the PHP mime type:

		# AddType allows you to tweak mime.types without actually editing it, or to
		# make certain files to be certain types.
		#
		AddType application/x-tar .tgz
		AddType application/x-httpd-php .php

________________________________________________________________________________
14.  extra setup

	. webserver principal credentials -- have your friendly KCA admin generate 
		a key and signed cert for the webserver.  these should be dropped in 
		$APACHE/conf/ssl.key/server.key  and  $APACHE/conf/ssl.crt/server.crt, 
		respectively.
		
	. create (or copy over) the .globus directory in $APACHE.  in .globus/, install 
		the self-signed cert for your KCA and set the perms, e.g.:
		
			% ls -lart $APACHE/.globus
			-rw-r--r--    1 nobody   nobody       1422 Jun 29 15:43 078314af.signing_policy
			-rw-r--r--    1 nobody   nobody       1017 Jun 29 15:43 078314af.0

	. obtain the bundle of CA certificates that this webserver will trust; call that
		ca-bundle.crt.  install it into $APACHE/conf/ssl.crt/ca-bundle.crt
		
	. obtain a Kerberos v5 keytab for your kx509 realm -- this is used by mod_kct and
		mod_kx509.  given the path to "kctkeytab" specified in the step above, put the
		file there and set its permissions:
		
			% ls -lart $APACHE/conf/*.keytab
			-rw-------    1 nobody   nobody        204 Jun 29 16:18 krb5.keytab

	. have your admin add your webserver principal to the KCT's ACL file so that
		the webserver can access the KCA service.  make sure they restart the KCT
		thereafter :)
		
	. tailor the pilot script -- set ithe variables "ldapIP", "ldapDefaultBase", and
		"exec_bin" to their correct values.  "ldapIP" is the LDAP directory server for the
		AdminRealm; "ldapDefaultBase" is the DN suffix for the directory; and "exec_bin"
		is the globus_client binary used to issue remote commands (should be in 
		$APACHE/bin/globus_client when you're done setting up the grid modules).