# Installation Guide

12 February 2007 - Document version 2.3

## Overview

### Choose an Install Type

Sakai downloads are available for three different installation methods on http://sakaiproject.org/release:

Demo:

  This archive includes the Tomcat container and pre-built webapps, so that the only prerequisite for getting started is Java (v. 1.5). While not intended for large-scale implementations, it is suitable for evaluating the software and small-scale pilot implementations on single servers.

Binary:

  This archive is a kind of Tomcat overlay. If you have a working Tomcat appropriately configured - including the database connection - the binary installation provides a quick way to drop the Sakai application bundle into place without requiring a maven build. This may be a good choice if you know you don't need to make any code-level changes.

Source:

  This archive includes only the source code to build Sakai, and therefore calls for preparatory installations of Tomcat and the Maven build tool, along with (most likely) a preferred database. The installation and configuration of these various components comprises the bulk of this installation guide.

### Demo Installation

A demo installation is brief and straightforward. The most technically challenging aspect of a demo install is making sure that your Java environment is squared away in advance. The steps are as follows:

- Verify Java Installation and Environment Variables (see the first two sections of Set up Environment).

- Download and unpack the Demo archive

  Windows:

    demo archive

  Mac/*nix:

    demo archive

- Start Tomcat with the following commands from the root Tomcat directory:

  Windows:

    `start-sakai.bat`

  Mac/*nix:

    `start-sakai.sh`

Once Tomcat is started up, you can point your browser to http://localhost:8080/portal. This will bring you to the Sakai gateway site, from which you can create new accounts and log in to the syste. For more orienting tips on how to begin using Sakai, skip ahead to Getting Started.

To stop the demo:

Windows:

  `stop-sakai.bat`

Mac/*nix:

  `stop-sakai.sh`

Data storage: | The data for a demo installation is stored in a HypersonicSQL database (HSQLDB) which is stored in memory and then written out to files when Tomcat is shut down. These files are located at `sakai-demo/sakai/db/sakai.db.*`.
Email not configured: | The Sakai demo is configured to neither send nor receive email.

### Binary Installation

The Binary installation of Sakai provides a shortcut for those that already have Tomcat in place and configured as needed (including the database connection: see Set up Environment for details on Tomcat configuration, and Database Configuration for database details), and it does so by providing a pre-built Sakai that can simply be dropped into place. All you need to do is unpack the binary archive at the root Tomcat directory, and the appropriate wars, jars, etc., for the Sakai application will be deposited in the correct locations.

If you wish to configure a binary installation you'll need to manually create a `sakai` directory in `$CATALINA_HOME` to hold your `sakai.properties` file, as described in Post-Installation Configuration.

### Source Installation

The Demo and Binary installs described above provide quick ways to get up and running, but assume that no code-level changes are called for. The remainder of this installation guide devotes itself to the more rigorous and customizable Source installation, although some of its details (e.g. the post-installation configuration) may also be pertinent for more sophisticated uses of the demo and binary installs. The full set of steps for a Source installation may be organized into the following phases:

1. Set up Environment, including Java, Tomcat, and Maven.

2. Build and Deploy Sakai by running Maven.

3. Perform appropriate Post-installation Configuration of the application.

4. If you want to configure Sakai for either MySQL or Oracle, perform the requisite Database Configuration.

### Migrating from a Previous Release

Migration from an earlier version of Sakai typically involves a database conversion (for which scripts are supplied with the release), an update to any custom skins, and possibly changes to any custom code (see the Release Notes to learn if this is the case).

For links to the conversion scripts, see the Database Configuration section of this guide.

## Set up Environment

### Verify Java Installation

Check to see if you have Java (version 1.5) installed on your system by running the following command (sample output from a *nix system):

    $ java -version
    java version "1.5.0_06"
    Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
    Java HotSpot(TM) Server VM (build 1.5.0_06-b05, mixed mode)

You'll want to make sure that you have also installed the full J2SE, and not just the JRE (Sakai includes .jsp and .jws files which require some compilation).

JRE vs. JDK vs. J2SE vs. J2EE

If you're unfamiliar with the Java world, you may find the acronyms for various Java environments a little mysterious. The JRE is simply the "runtime environment" which allows you to execute compiled Java bytecode, and for many platforms this is all you usually get by default - for the very good reason that the average user just wants to run programs, and doesn't need to be building code.

To run Sakai, however, you'll also need the JDK (Sakai includes some .jws and .jsp files which are not pre-compiled), which stands for "Java Developer's Kit." To both build and run Sakai you'll therefore need a combination of both the JDK and the JRE, and, happily, Sun makes this available as a unit: the J2SE (the "Java 2 Standard Edition").

If you go to the Sun site for a download, you may also run across the J2EE (the "Enterprise Edition"), or even the J2ME (the "Micro Edition"). Don't trouble yourself with these, unless you know of a particular reason why you may want them - neither is necessary for a Sakai installation.

### Verify Java Environment Variables

There are three Java environment variables you'll want to set for everything to function as it should, and though some of them may have already been set by your Java installation (depending on your platform), you'll want to double-check them all. You can also look at the Sun Java Installation Instructions page at the Java web site for further details.

First you'll need to set the `JAVA_HOME` environment variable to point to the base directory of your Java installation, which will enable Tomcat to find it. In UNIX operating systems you typically modify a startup file like `~/.bash_login` to set and export these shell variables, while for Windows XP you would go toStart -> Control Panel -> System -> Advanced -> Environment Variables and then set them to point to the correct locations through the text fields of the GUI (samples below):

Windows:

  Set the (or create a new) environment variable `JAVA_HOME` to `C:\jdk1.5.0_06`.

Mac/*nix:

  `export JAVA_HOME=/usr/java/java-current`

JRE_HOME

If your machine has already set JRE_HOME for any reason, or if you want to be sure to be using a particular JRE - in the event that you have more than one JRE installed on your machine - then you'll want to set the JRE_HOME variable as well. JRE_HOME is what Tomcat uses when it starts up, but it defaults to use JAVA_HOME if JRE_HOME is not set. For most people, then, setting JAVA_HOME will cover both cases sufficiently.

Next you'll want to append to the PATH variable the location of Java commands:

Windows:

  Append the string `;C:\jdk1.5.0_06\bin` to the end of the system variable named Path.

Mac:

  Not necessary.

*nix:

  `export PATH=$PATH:$JAVA_HOME/bin/`

You should test that these variables are set correctly. In both Windows XP and *nix operating systems you can simply start a new shell and type the `set` command to see your environment variables.

### Install Tomcat

Sakai 2.3.1 was QA'ed with Tomcat 5.5.17, and so that's the version we recommend, although the latest stable version will most likely not present problems. Tomcat can be downloaded as a binary from http://tomcat.apache.org/download-55.cgi. The distribution you want is the one labeled Core, along with the JDK 1.4 Compatibility Package (even though you're using Java 1.5, some Sakai dependencies will require the compatibility package, and Sakai won't start up without it).

Tomcat on Windows

Windows users have the option of either downloading an .exe Tomcat installer (for installing Tomcat as a service) or a .zip binary. Although either may serve, they are however not entirely equivalent when it comes to operational details, and for control and brevity's sake the main instructions will assume that you grabbed the .zip. You can turn the .zip install into a service install later, however, by running `service.bat` from the bin directory, e.g.

    C:\tomcat\bin> service.bat install

You can add a service name as a second argument to that script. The default is "Tomcat5". You can likewise uninstall the service by replacing "install" with "remove". Then to get the configuration window, just run this at the command prompt:

    C:\tomcat\bin> tomcat5w //ES//Tomcat5

Replace "Tomcat5" with whatever service name you chose for the install. You'll want to set the service to startup automatically ("Startup Type" under the General tab).

Also be sure to read the Windows note in the JVM Tuning section of Post-Installation Configuration.

Unpack the Tomcat archive at your desired location, and unpack the compatibility package directly on top of it. From this point forward these instructions will refer to the top-level Tomcat directory (e.g. `/usr/local/apache-tomcat-5.5.17`) as `$CATALINA_HOME`. You may set this as an environment variable for convenience's sake, but this is not required. Make sure that you have write permissions to the Tomcat files and directories before proceeding, or you may later run into errors during the build phase.

### Configure Tomcat

Sakai supports UTF-8, allowing for non-Roman characters, but this requires that Tomcat be configured to accept UTF-8 URLs since it ships with ISO-8859-1 as the default URL encoding. To change this setting, edit `$CATALINA_HOME/conf/server.xml`. Add the attribute `URIEncoding="UTF-8"` to the <connector> element. For example:

    <Connector port="8080" maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
    enableLookups="false" redirectPort="8443" acceptCount="100" debug="0"
    connectionTimeout="20000" disableUploadTimeout="true" URIEncoding="UTF-8"/>

If you want to run Tomcat on different ports than the defaults, this would also be a good time to make those changes in the `server.xml` file. See Tomcat configuration documentation for more details.

If you're going to run Tomcat as a standalone web server (i.e. if you're not going to connect it to the Apache web server) then you'll want to make a further minor change that may spare some confusion later. The `ROOT` webapp is the one served up when a request is made to your Tomcat's root URL, and so if you want users to be re-directed automatically to the Sakai application, you'll want to insert an `index.html` file into `webapps/ROOT` that prompts this re-direction. It should look something like:

    <html>
        <head>
            <meta http-equiv="refresh" content="0;url=/portal">
        </head>
        <body>
            redirecting to /portal ...
        </body>
    </html>

If you don't make this change you (and your users) will need to append '`/portal`' to the URL entered to access Sakai each time. If you intend to connect Tomcat with Apache, you can handle this issue as a matter of Apache configuration, which is however outside the scope of this document.

### Install Maven

Maven is the build tool used by Sakai, and the latest stable release (currently 1.0.2) can be downloaded from http://maven.apache.org/maven-1.x/start/download.html

Newer versions of Maven

It's important that Maven 1.0.2 be used. There are betas for Maven 1.1 and Maven 2.0 currently available, but neither will work for Sakai 2.2. Many plugins have not yet been adapted for the new Maven architecture.

Choose a location for Maven, and unpack the archive there. You will have a top-level directory named `maven-1.0.2`.

### Configure Maven

To use Maven you'll need to set two more environment variables and then create a local repository using a script provided by Maven. Just as with the Java variables, `MAVEN_HOME` should be set, and then Maven's binary commands should be added to the Path:

Windows:

  Create a new `MAVEN_HOME` environment variable to `C:\maven-1.0.2`. Then append to the PATH variable `;C:\maven-1.0.2\bin`

Mac/*nix:

  `export MAVEN_HOME=/usr/local/maven; export PATH=$PATH:$MAVEN_HOME/bin`

Next you'll want to create your local maven repository in your home directory by running the following command:

Windows:

  `install_repo.bat %HOMEDRIVE%%HOMEPATH%\.maven\repository`

Mac/*nix:

  `install_repo.sh $HOME/.maven/repository`

Home Directories in Windows

You may not often deal with the notion of the home directory on Windows systems, but it's still there. By default its location is `C:\Documents and Settings\yourusername`. Windows also establishes it through environment variables which you've just seen above - the combination of your "home drive" location and your "home path" location: `%HOMEDRIVE%%HOMEPATH%` (The %-sign is how Windows brackets environment variables).

Finally, you'll need to create a build properties file in your home directory which will define some basic parameters for your Sakai build. Simply create a new text file with the filename `build.properties` in your home directory, and paste in the following contents:

    maven.repo.remote=http://source.sakaiproject.org/maven/
    maven.tomcat.home=/usr/local/tomcat/

build.properties syntax

Maven is very sensitive to the syntax of the build.properties file. Be sure not to omit the trailing slashes as shown above, and be sure to change the value of maven.tomcat.home to match the path to your Tomcat installation.

Further, if you are running on Windows special care is needed in identifying your tomcat home. Maven wants Unix-style forward slashes, "/", and is confused by Windows-style backslashes "\". If you have your tomcat located in c:\tomcat, for example, you need to identify it like this:

    maven.tomcat.home=c:/tomcat/

Finally, to confirm that you can start Maven, run the command `maven -v`. This should start maven and cause it to report its version. At this point your environment is prepared to build and deploy the Sakai source code.

## Build and Deploy

Sakai uses Apache's Maven build tool to compile and deploy its modules.

How Sakai uses Maven

The steps below are limited to what you need to get started, but you may feel the need to learn more about how Sakai uses Maven. To find the most detailed such documentation, look for `sakai_maven.doc` in `reference/docs/architecture` of the source archive or on subversion at: https://source.sakaiproject.org/svn/reference/tags/sakai_2-3-1/docs/architecture/sakai_maven.doc

### Download Source

If you haven't yet, download the Source archive from http://www.sakaiproject.org/release

Getting the Code from Subversion

Alternatively, you may check out the source code from subversion. You might wish to do this, for example, in order to grab the maintenance branch, which will include the latest set of bug fixes. To check out the maintenance branch, you can use the following command:

    svn export https://source.sakaiproject.org/svn/sakai/branches/sakai_2-3-x/

In which case you could skip the unpacking step below, and your root directory would be sakai_2-3-x instead of sakai-src.

### Unpack Source

Choose a location to unpack the Sakai source (your home directory is fine), and when you unpack it you will see a directory named `sakai-src`.

### Run Maven

From within the `sakai-src` directory, run the command `maven bld dpl`.

Subsequent builds

Subsequent builds would be advised to add the "clean" phase by using the command `maven cln bld dpl`, or the equivalent shortcut: `maven sakai`. See the sakai_maven.doc for more details.

This will run for quite a few minutes with fairly verbose output, particularly when it's your first build. Maven will download any dependencies into the local repository, compile the Sakai code, and then deploy Sakai to Tomcat in the form of .war files in the `$CATALINA_HOME/webapps` directory. If Maven completes with the message `BUILD SUCCESSFUL`, you should be able to move on to the next step. If you are greeted with the report `BUILD FAILED` read the accompanying error message carefully to troubleshoot (see the Troubleshooting section).

Maven and Memory

You may find that your machine runs out of memory while it is building Sakai. As it turns out, maven does not read `JAVA_OPTS` when it starts up, and so if your build output complains of "Out of Memory" errors you'll need to set an additional environment variable, namely, `MAVEN_OPTS`. The value in the sample Unix command below should be more than enough:

    export MAVEN_OPTS="-Xmx384m -XX:PermSize=48m"

You will probably be able to start up Tomcat and run Sakai with the default configuration at this point, but it would be better to take care of some basic post-installation configuration first.

## Post-Installation Configuration

### Create folder for properties file

Sakai runs with a default set of configuration properties for its various components. To override them you'll want to specify them in a `sakai.properties` file which should be located in `$CATALINA_HOME/sakai` by default. This directory is not created by maven, so you'll have to do so manually.

Choosing a different location for sakai.properties

You may find it preferable to store Sakai's configuration files outside of the Tomcat file hierarchy. In a development environment, for example, you may find yourself frequently blowing Tomcat away, and you'll likely want to avoid recreating the sakai folder and its contained file(s) each time.

To accomplish this, you need only modify the java startup command (or the `JAVA_OPTS` environment variable) to set the system property `sakai.home`:

    -Dsakai.home=/path/to/desired/sakai/home/

You'll need to make sure that this location is readable and writable to Tomcat.

### The sakai.properties file

The main configuration file for Sakai is called `sakai.properties`, and you can either create it from scratch or copy in a known working copy. A sample `sakai.properties` file which self-documents many of the standard properties in its comments can be found in the source under `sakai-src/reference/docs/sakai.properties`, while the properties file that the OOTB software uses as its default may be found in `sakai-src/kernel/component/src/config/org/sakaiproject/config/sakai.properties.`

New value settings can be freely added to the `sakai.properties` file. Since any component property can in principle be overridden here, any sample `sakai.properties` will show only a small fraction of all the possible settings.

Learning more about sakai.properties

Settings in `sakai.properties` govern everything from setting your institution name to configuring your database. The essential settings for installation are covered here, but to find the most detailed documentation on the full variety of possible `sakai.properties` settings, look in sakai_properties.doc in subversion or `reference/docs/architecture/sakai_properties.doc` of the source archive.

### HTTP proxy

In environments where local network policy or firewalls require use of an upstream http proxy / cache, Sakai needs to be configured to use this. Otherwise, components or services which use http requests, such as BasicNewsService for RSS feeds in the News tool, cannot retrieve data from the target URLs. RSS feeds then do not display in the news tool, and it is not possible to add new news channels.

This can be fixed by adding the following JAVA_OPTS arguments to the Sakai local startup script (e.g. `/etc/rc.d/sakai`), or to a Sakai startup script such as `startup.sh` or `catalina.sh`:

    export JAVA_OPTS="-DproxySet=true -DproxyHost=cache.some.domain -DproxyPort=8080"

You'll need to set additional JAVA_OPTS for memory tuning (see below), and the appropriate ones can simply be appended.

### Email configuration

Sakai needs to be set up for two email functions: receiving email sent to Sakai sites, and sending out email notifications. For sending mail Sakai needs the address (name or IP) of an SMTP server that will accept mail from Sakai, and this needs to be set in your `sakai.properties` file:

    smtp@org.sakaiproject.email.api.EmailService=some.smtp.org

To enable Sakai to receive mail you'll need to set the following:

    # dns addresses used for incoming email
    smtp.dns.1 = 255.255.255.1
    smtp.dns.2 = 255.255.255.2

    # SMTP port on which our SMTP server runs. Default is 25.
    #Recommend running on 8025, and using a standard mailer on 25 to forward mail to Sakai.
    smtp.port = 25

    # flag to enable or disable our SMTP server for incoming email (true | false)
    smtp.enabled = true

To disable the SMTP server for incoming email, use this line in `sakai.properties`:

    smtp.enabled=false

Sakai's SMTP server is James, and to run with the above configuration which runs James on the standard SMTP port 25 you must be running with admin privileges. Most admins won't want to let Tomcat run with those permissions, and would rather run a standard mailer like postfix on port 25 and configure it to forward requests to Sakai. You might also already have a mailer service running on port 25 (Linux usually has it running by default), and so you'd want to change the James port simply to avoid a conflict. For example:

    smtp.port = 8025

### JVM Tuning

The default Java virtual machine is not adequate to run an application of Sakai's size, and several JVM parameters must be increased to a certain threshold for Sakai to run, while some may be further adjusted for optimal performance on your machines.

Disclaimer

JVM tuning is, as a general rule, something of a black art, and we recommend that you take the time to experiment with different memory and garbage collection settings and see what works best in your environment. We can make some minimal recommendations that should serve as a foundation for further experimentation and testing, but the following details are however offered only as samples or suggestions, and we recommend that you consult a systems administrator or local Java guru before making any such changes to a production system. See the Programmer's Cafe on our Confluence wiki for more discussion of the details.

The standard way to control the JVM options when Tomcat starts up is to have an environment variable `JAVA_OPTS` defined with JVM startup options. Since any given Sakai instance may be deployed for a variety of purposes - ranging from developers doing private testing to large-scale deployments - it's hard to recommend a single set of such options as the preferred ones for every case. We can, however, start with a bare minimum which will at least avoid "Out of Memory" errors, and be suitable for developer installs of the software:

    JAVA_OPTS="-server -Xms512m -Xmx768m -XX:PermSize=128m -XX:MaxPermSize=196m -XX:NewSize=192m -XX:MaxNewSize=384m"

This is an adequate - if minimal - starting point: it selects server mode, sets an adequate heap size, and sizes the permanent generation to accommodate more longer-persisting objects. These settings will allow things to be functional for testing and development, but will hardly be adequate for serving large numbers of concurrent users. A more suitable production environment on a 32-bit machine with multiple processors might use a set of options like:

    JAVA_OPTS="-server -Xms1500m -Xmx1500m -XX:NewSize=400m -XX:MaxNewSize=400m -XX:PermSize=128m
    -XX:MaxPermSize=196m -verbose:gc -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC"

This is better: a larger heap size is set for smoother performance (we have found the best results when the min and max memory settings - Xms and Xmx, respectively - are set to be the same values), and garbage collection messages are turned on. Another important consideration is the ratio of NewSize to the size of the heap. We want as large a NewSize as we can fit in the total heap, while keeping the total heap significantly bigger than NewSize in order for Java to properly garbage collect. The last two arguments above can produce a performance improvement if you have two or more processors on your machine, however your performance will be better if you don't include them for single-processor machines.

As you can see, there's a lot to think about here, and in practice each implementing institution uses their own particular `JAVA_OPTS` that they've settled on for their deployment and hardware, and they often use more option arguments than we've shown here. This discussion is meant only as a head start, and there is no replacement for doing your own testing.

Once you set `JAVA_OPTS` Tomcat will see this environment variable upon startup and use it. Instead of putting this in an environment variable you might create your own startup script to do so.

JVM Tuning of Tomcat as a Windows Service

Windows users that have installed Tomcat as a service may set most Java options through the Tomcat service manager GUI, but not all of them are as straightforward as inclusion in a single environment variable.

To achieve the equivalent of the "-server" option, for example, you'll need to change the Java Virtual Machine path from `[..]\bin\client\jvm.dll` to `[..]\bin\server\jvm.dll`. Then be sure to put the remaining JAVA_OPTS on separate lines in the Java Options field of the GUI. e.g.:

    -Xmx512m
    -Xms512m
    -XX:PermSize=128m
    -XX:MaxPermSize=128m

Finally, clear out the Initial Memory Pool and Maximum Memory Pool values, as those might conflict with the options you're putting in the Java Options field. Then click Apply, restart the service, and double-check the service manager to verify that the values have changed.

### Test Sakai

At this stage your installation of Sakai has not yet been configured to use your preferred database (it will use its own HSQLDB by default), but you should now be able to bring it up as a working web application by simply starting Tomcat, and it can be helpful at this point to know if any problems exist before you try to connect it to another database. Tomcat will take a minute or so to start up, depending on the speed of your machine, and it's a good idea to watch the Tomcat log as it comes up to catch any errors (see Troubleshooting).

From `$CATALINA_HOME` you can start up Tomcat with the command:

Windows:

  `bin/startup.bat`

Mac/*nix:

  `bin/startup.sh`

Once Tomcat has loaded the Sakai application (again, this can take a minute or so) point your browser to http://localhost:8080/portal. If the gateway page does not come up, check the Tomcat logs (see the Troubleshooting section) for any errors and stack traces. If the gateway page does come up, log in with the default admin account (the username and password are both 'admin'). If you can log in without errors you should be able to stop Tomcat and proceed with Database configuration, if needed.

Windows:

  `bin/shutdown.bat`

Mac/*nix:

  `bin/shutdown.sh`

## Database Configuration

### Migrating from an Earlier Version

A database conversion is typically required between Sakai versions. Database conversion scripts - in distinct versions for MySQL and Oracle, respectively - are found in the `reference/docs/conversion` folder of the release or on subversion:

MySQL:

  https://source.sakaiproject.org/svn/reference/tags/sakai_2-3-1/docs/conversion/sakai_2_2_1-2_3_1_mysql_conversion.sql

Oracle:

  https://source.sakaiproject.org/svn/reference/tags/sakai_2-3-1/docs/conversion/sakai_2_2_1-2_3_1_oracle_conversion.sql

In the same directory you'll also find conversion scripts for earlier Sakai versions. Migration from an earlier version will require the successive application of all intermediate scripts. You cannot, for example, move from 2.2.0 to 2.3.1 by applying a single script. You will need to move first from 2.2.0 to 2.2.1, and then from 2.2.1 to 2.3.1 (there was no database conversion between 2.2.1 and 2.2.2, and while there is a conversion script from 2.3.0 to 2.3.1 for those who may have already migrated to 2.3.0, the 2.2.1-2.3.0 conversion had a significant gradebook error, and is no longer made available).

Examine before using

As a general rule, be sure to read through these conversion scripts before applying them. They do not take into account any special customizations you may have made - such as new roles, or the deployment of additional tools - and they may complicate your migration with unintended consequences if you execute them blindly.

### Deploy Drivers

The supported production-grade databases include MySQL 4.1.12+ (but MySQL 5.0 is not yet recommended for production - see the Configuration for Performance section below) and Oracle 9i+. The version of the JDBC driver (or connector) is also important: for MySQL a 3.1.12+ connector should be used, while for Oracle the 10g driver must be used, even if the database is Oracle 9i. These drivers should be copied into your `$CATALINA_HOME/common/lib` directory, and they are available from the official sites:

MySQL:

  http://dev.mysql.com/downloads/connector/j/3.1.html

Oracle:

  http://www.oracle.com/technology/software/tech/java/sqlj_jdbc/index.html

Driver Versions

Database driver versions are a common source of problems. It's worth emphasizing again that the Oracle 10g driver must be used for Sakai installations running against Oracle, even when the database is Oracle 9i.

Problems have been reported for both the 3.1.10 and 3.1.11 MySQL drivers. 3.1.12+ will be adequate, but 3.1.14 is the latest recommended version.

### Create Database and User

A Sakai database and privileged user must be prepared for Sakai's use. Consult your database documentation for details, but below are sample commands for MySQL.

    C:\sakai\reference\sql\legacy\mysql\>
    mysql -u root -p
    Enter password: ******
    Welcome to the MySQL monitor. Commands end with ; or \g.

    Your MySQL connection id is 51 to server version: 4.1.5-gamma-nt

    Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

    mysql> create database sakai default character set utf8;
    Query OK, 1 row affected (0.00 sec)

    mysql> grant all on sakai.* to sakaiuser@'localhost' identified by 'sakaipassword';
    Query OK, 0 rows affected (0.00 sec

    mysql> grant all on sakai.* to sakaiuser@'127.0.0.1' identified by 'sakaipassword';
    Query OK, 0 rows affected (0.00 sec)

    mysql> quit

UTF-8 Character Set

When the database is created you must be sure to create it to use the UTF-8 character set, just as Tomcat was configured to use UTF-8. If you don't you may run into a range of issues when attempting to use Unicode characters in Sakai, and this goes for both MySQL and Oracle. Consult your DB documentation or a local DBA for instructions on how to do this.

If you're not certain how your database is currently configured, you can check with a query. Here is a sample query from Oracle showing the correct value:

    SQL> select value from nls_database_parameters where parameter = 'NLS_CHARACTERSET';

    VALUE
    --------------------------------------------------------------------------------
    AL32UTF8

Converting a database from one character set to another is non-trivial, particularly if it's a large production database, and so it's strongly recommended that you verify this aspect of your database creation before deploying Sakai.

### Database Properties

There are settings in sakai.properties that also define the database technology and connection information. Appropriate `sakai.properties` settings for Oracle and MySQL, respectively, are listed below, and you need only modify them with your local particulars:

MySQL:

      hibernate.dialect=org.hibernate.dialect.MySQLDialect
      vendor@org.sakaiproject.db.api.SqlService=mysql
      driverClassName@javax.sql.BaseDataSource=com.mysql.jdbc.Driver
      url@javax.sql.BaseDataSource=jdbc:mysql://SERVER:3306/DB?useUnicode=true&characterEncoding=UTF-8
      username@javax.sql.BaseDataSource=USER
      password@javax.sql.BaseDataSource=PASSWORD
      validationQuery@javax.sql.BaseDataSource=select 1 from DUAL
      defaultTransactionIsolationString@javax.sql.BaseDataSource=TRANSACTION_READ_COMMITTED

Oracle:

      hibernate.dialect=org.hibernate.dialect.Oracle9Dialect
      vendor@org.sakaiproject.db.api.SqlService=oracle
      driverClassName@javax.sql.BaseDataSource=oracle.jdbc.driver.OracleDriver
      url@javax.sql.BaseDataSource=jdbc:oracle:thin:@SERVER:1521:DB
      username@javax.sql.BaseDataSource=USER
      password@javax.sql.BaseDataSource=PASSWORD
      validationQuery@javax.sql.BaseDataSource=select 1 from DUAL
      defaultTransactionIsolationString@javax.sql.BaseDataSource=TRANSACTION_READ_COMMITTED

### Configuring for Performance

Larger institutions have found the database to be the bottleneck when it comes to Sakai performance. Some additional DB configuration settings may be worth considering.

#### Oracle Performance

Oracle may have performance problems with some of the SQL settings that work for HSQL and MySQL. Sakai installations using Oracle should strongly consider the following settings in `sakai.properties` to avoid these problems:

    # For improved Oracle performance (from the University of Michigan)
    validationQuery@javax.sql.BaseDataSource=
    defaultTransactionIsolationString@javax.sql.BaseDataSource=
    testOnBorrow@javax.sql.BaseDataSource=false

Oracle should be set to the proper settings for the first two items (above) automatically; setting them with each use may affect performace. In addition, validating the connection on each transaction caused problems in at least one large production environment (University of Michigan).

#### MySQL Performance

MySQL performance can be considerably improved by caching queries (Oracle does this already, without any extra configuration). You can set this in sakai.properties by altering the connection string (i.e. this will replace the simpler value indicated above) as below, all on one line.

         url@javax.sql.BaseDataSource=jdbc:mysql://localhost:3306/sakai?useUnicode=true&characterEncoding=UTF-8
    &useServerPrepStmts=false&cachePrepStmts=true&prepStmtCacheSize=4096&prepStmtCacheSqlLimit=4096

(all one line)

The parameter that enables use of the query cache is 'useServerPrepStmts=false', while the others (cachePrepStmts=true, prepStmtCacheSize=4096, and prepStmtCacheSqlLimit=4096) are for caching the parsing of prepared statements on the server side. Based on production experience, we'd recommend a query cache size of 64M, configurable in /etc/my.cnf like this:

    [mysqld]
    query_cache_size = 64M

There are some other settable properties for the query cache, but there doesn't seem to be a need to change the defaults. To learn more, you can visit:

- http://dev.mysql.com/doc/refman/4.1/en/query-cache-configuration.html

- http://dev.mysql.com/doc/refman/4.1/en/connector-j-reference-configuration-properties.html

MySQL 5

The above configuration properties work for MySQL 4.1 only. The connection parameters that allow MySQL caching on MySQL 4 cause problems with bit types on MySQL 5. We will likely have a better answer for the next Sakai release, but the issue was not resolved in time for Sakai 2.3. In the interim the basic, non-caching string must be used for MySQL 5, and since this means poorer performance, MySQL 5 is currently not recommended for production.

#### Disk I/O on Linux (SLES9/RHEL4)

A Linux database server may see big disk-hanging delays (especially if the DB and the web server are on the same machine) under load. This appears to be a side effect of the default SLES9 I/O scheduler, CFQ. The Deadline scheduler, which has a maximum latency for serving requests, is therefore a better choice for database operations - although pages may still render slowly under load, performance will degrade more gracefully, avoiding hangs, and provide some feedback to users.

To switch to the Deadline scheduler, the boot parameter elevator=deadline must be passed to the kernel that's being used. You can do so by editing the `/etc/grub.conf` file and adding the parameter to the kernel that's being used:

    title Red Hat Enterprise Linux AS (2.4.21-32.0.1.ELhugemem)
        root (hd0,0)
        kernel /vmlinuz-2.4.21-32.0.1.ELhugemem ro root=/dev/sda2 elevator=deadline reboot=warm
        initrd /initrd-2.4.21-32.0.1.ELhugemem.img

Make sure to reboot the system to activate the new scheduler. You can also see these links for more information:

- http://www.puschitz.com/TuningLinuxForOracle.shtml

- http://nextre.it/oracledocs/ioscheduler_01.html

- http://www.open-mag.com/features/Vol_104/SLES9/SLES9.htm

### Startup

Once you've configured the database appropriately, you need only stop and restart Tomcat. As Tomcat is coming up you can watch its log to see if there are any database connection errors (see the Troubleshooting section).

Oracle and Tests & Quizzes

If you're running Oracle you should check the datatype of the `MEDIA` column in the `SAM_MEDIA_T` table. Hibernate tries to choose the right data type for a field, but has a habit of choosing the wrong one for Oracle. The correct types for each database are:

HSQL:

  varbinary

MySQL:

  longblob

Oracle:

  blob

If you need to change this type for your database, this will also involve finding the primary key constraint, dropping it and then recreating it. Contact your local DBA for further information on making this change. Below is some sample Oracle SQLplus output to better illustrate (SYS_C0064435 is the example constraint; replace it with yours):

    SQL> alter table SAM_MEDIA_T modify MEDIA BLOB;

    Table altered.

    SQL> select constraint_name from user_constraints where table_name='SAM_MEDIA_T'
    and CONSTRAINT_TYPE='P';

    CONSTRAINT_NAME
    ------------------------------
    SYS_C0064435

    SQL> alter table sam_media_t drop constraint SYS_C0064435;

    Table altered.

    SQL> alter table SAM_MEDIA_T add constraint SYS_C0064435 primary key (MEDIAID);

    Table altered.

    SQL> desc SAM_MEDIA_T;

    [table with BLOB type]

    SQL> select constraint_name from user_constraints where table_name='SAM_MEDIA_T'
    and CONSTRAINT_TYPE='P';

    CONSTRAINT_NAME
    ------------------------------
    SYS_C0064435

    SQL> commit;

    Commit complete.

## Getting Started with the Software

You should at this point have a working Sakai installation. Now it's time to get started with adding users, creating work sites, and otherwise playing around with the tools. We won't try to present a full user's guide here, but we can offer some pointers to get you oriented and on your way, and link you to more exhaustive sources of information elsewhere.

### The Gateway Page

When Tomcat has come up successfully, you should be able to direct your browser to its gateway page at http://localhost:8080/portal (or replace 'localhost' with the name of the server where it's installed). From the gateway page you can create new accounts or browse for public site content. You could start by creating a new account, but that can also be done as an admin, and since the admin functions are needed to allow this account the right permissions, it's just as well to start by logging in as the admin user. Sakai's out-of-the-box admin account is simply named 'admin' (with password also 'admin'), so use those credentials to log in.

### My Workspace

Every user on the system - including the admin - has a private site called My Workspace. It's the landing point upon logging in, and it's the first site tab visible at the upper left. Running vertically along the left-hand side of the screen are links to the various different tool pages within a given site, and the admin's My Workspace has a different set of such options here than most (each different type of account can be configured to have a different set of tools in its My Workspace by altering a template - see below).

### Other Sites

Each new (accessible) site becomes visible as a tab along the top, to the right of My Workspace. For most users, they initially only have access to one site - their My Workspace. The admin user is a little different, in that it has access to two.

The second admin site (which you can enter by clicking on its tab) is entitled Administration Workspace which, strangely enough, looks exactly the same as the admin My Workspace. It is.

Why the redundancy? Because you'll likely want to make these admin tools available to a particular user who doesn't have access to the admin's My Workspace (no one has access to other people's My Workspace on the system). To allow anyone access to the admin tools, you need only add them to the list of users of the Administration Workspace, and then promote them in the site to the "admin" role.

### Admin Tools

#### Creating Users

The first thing you may want to do is to change the admin password to something secure, and to start creating a few sample users on your system. You can do both of those tasks through the Users tool on the left. To change the admin user's password, simply click on the 'admin' username in the list of users, and edit the fields on the subsequent page. To create users, click on the New User action link at the top of the tool page.

#### Creating Sites

If you're itching to create your first worksite, you may be tempted to dive directly into the Sites tool. That would probably be a mistake. The Sites tool is a powerful way to construct an entire site from the ground up, with fine-grained control over its every page, tool, and configuration detail. But this flexible power comes with a price, making for an intimidating interface and epic-scale workflow. The Sites tool is therefore best used as a way to tweak an existing site after the fact, once the standard pieces have been more expediently assembled.

The best way to start creating sites, therefore, is to use the Worksite Setup tool. Click on the New link at the top of the tool page, and then, for simplicity's sake, choose the Project site type, which will allow you to avoid issues of academic term, etc., that are provoked by a "course" site - issues which are probably unnecessary if you just want to start playing with the tools. Either type of site will serve, however: both types of sites have all the tools available to them.

Step through the remaining site creation pages, making your preferred selections. Be sure to click the Create Site button at the end of the process. After doing so, you should see the site title visible as a new tab along the top of the screen.

#### Adding Users to Sites

Since you set this site up as an admin, the admin is technically the owner of this site, and its only member at first. If you want to add other sample users to this site in different roles, you can do so through the Site Info tool of the site itself.

Click on the tab of your new site (which should now be visible) to enter it, and then click on the Site Info link along the lefthand side. Site Info has a number of site maintainer functions available as action links across the top, and Add Participants is the one that will allow you to connect other users to the site. These users will of course need to have been previously created.

As long as we're here in Site Info, it's worth pointing out that the Edit Tools link at the top will allow you to remove and add tools from the site.

### Experimenting with Tools

You may have noticed an extra tool appear in your site - one which you didn't explicitly choose - labeled Help. This tool provides online documentation of the various bundle tools, and other facets of the system. This should be your companion as you learn more about the software's functionality.

This Help tool is also reached in a context-sensitive way by clicking on the question mark icons at the upper right of any particular tool frame. Clicking on those question marks will open up the precise content of the tool you happen to be in at the time.

At some later point, when you become comfortable with the standard tools, you may wish to see other, more experimental tools that are available for Sakai. The Sakai distribution includes provisional tools that are still maturing, but can already serve needs in innovative ways that the standard ones do not. These extra tools require additional steps to enable, so that system users will not stumble across them inadvertently if that's not desired, but you are encouraged to evaluate them for your own deployment. See http://bugs.sakaiproject.org/confluence/display/DOC/2.3+Sakai+Tools for more details.

### Where to Learn More

Sakai information has been unfortunately a little fragmented to date. Attempts are underway to consolidate and present this information in a more uniform way, but in the interim a tourist's guide of the landmarks may be in order.

#### About the Project

The sakaiproject.org site offers the best background and orientation to the project and its goals, but chances are you've already seen it. For more particular information on the progress and roadmaps of certain tools or other initiatives, see the Confluence space named Management/Project Coordination. That coordinating page will also link you out to Confluence spaces for particular tools, and documentation about them.

#### Community Participation

The Sakai community hosts its own Sakai installation to serve up collaboration sites for its various sub-groups, and for direct interaction with community members you should join sites there that may be of interest. You can create an account there by visiting http://collab.sakaiproject.org and clicking on the New Account button. Once your account is created you can log in, and with the Membership tool you may join any worksites that may seem appropriate. Joining a site automatically adds you to its email list, and unjoining a worksite similarly removes you (while, it should be noted, sending an "unsubscribe" email will not).

#### About Technical Matters

Reference materials for various technical issues are best found in Sakai's subversion repository (where these installation documents are also stored) or in the project's Confluence wiki. The docs in subversion can be found in the reference module, e.g. https://source.sakaiproject.org/svn/reference. See in particular the `docs/architecture` folder contained there, which contains a wide-ranging collection of technical white papers.

The Confluence wiki is a more informal source of information, but also valuable and broader in scope. New developers in particular should visit the Programmer's Cafe, while the Sakaipedia provides nuggets of insight on a wide array of issues. You're encouraged to create an account there and participate.

But often the most incisive information comes from direct interaction with your peers. As mentioned above, the Collab Server is the place to go for this. For technical questions the best (and most active) such group is DG: Development, also commonly referred to by its alias, "sakai-dev."

## Troubleshooting

Below are a number of common issues encountered while building and starting up Sakai, but you may also benefit from the expert opinions that reside on the sakai-dev list. To join, visit collab.sakaiproject.org, create a new account, log in to your My Workspace, and then use the Membership tool to join the group named DG: Development. Then you can begin to send emails to (and receive them from) sakai-dev@collab.sakaiproject.org. To later unsubscribe, you can simply use the same Membership tool to unjoin the site.

### Build Issues

#### JAR download failures

A first build of Maven on a fresh installation may warn of numerous jar download failures if you include the clean phase. This is a nuisance, but not otherwise a problem. Even when doing a clean maven tries to download all dependencies, including those Sakai jars that may not be built until the later build phase. Once they are built and placed into the repository, maven is perfectly happy for the next "clean build" cycle. The upshot is that you may see these errors for your first build, but you shouldn't see them for subsequent builds. It also means that you're better off not including the clean phase during your first build of a new version of the code.

If this does not account for the download failures you're seeing, then you may want to double-check your `maven.remote.repo` setting. See the Maven Configuration section, and follow it precisely.

#### Out of Memory errors

You may find that your machine runs out of memory while it is building Sakai. As it turns out, maven does not read `JAVA_OPTS` when it starts up, and so if your build output complains of "Out of Memory" errors you'll need to set an additional environment variable, namely, `MAVEN_OPTS`. The value in the sample Unix command below should be more than enough:

    export MAVEN_OPTS="-Xmx384m -XX:PermSize=48m"

#### Uninformative failure messages

If you need more detailed output to track down a build error, maven can be run in debug mode by adding the -x argument, e.g. `maven -x`.

#### Building only a single module

If you'd like to have the Sakai build targets and goals available for building only a single module (i.e. instead of rebuilding the entire source tree again to accomplish this), then you'll want to have the maven plugin for Sakai installed. Installing the plugin makes it available whenever you use maven, not just for those projects that declare a dependency on the plugin. You will need to do this once each time you upgrade a minor point version of sakai.

    maven plugin:download -DgroupId=sakaiproject -DartifactId=sakai -Dversion=2.2

Note that the version you should use in the above command is the one found as the value of the `sakai.plugin.version` property in the `sakai/master/project.properties` file. It is not necessarily (and in fact often isn't) the same as the version of Sakai you're using.

### Tomcat Startup Issues

#### Finding the Tomcat logs

Once you have Sakai installed, configured and started, you can monitor Sakai by watching the logs. The log level for the standard Sakai source code and the demo is set to show info and warnings only. Watch for the WARN: messages. There are going to be some "normal" ones at startup, and some will likely slip by at runtime, but any warning is potentially something you might want to check out.

Changing the Log Configuration

To change the logging for Sakai, you need to change Sakai source code and re-deploy sakai. The file you need to change is `sakai-src/kernel/log-configure/src/conf/log4j.properties`, and the relevant property is:

    log4j.logger.org.sakaiproject=INFO

To turn on debug logging for all of Sakai, change that value from INFO to DEBUG. In order to turn on debug logging for just a single component of Sakai, add a line such as in the following example, which will leave most of Sakai at INFO, but generate DEBUG level messages for the SQL service:

    log4j.logger.org.sakaiproject=INFO
    log4j.logger.org.sakaiproject.component.framework.sql.BasicSqlService=DEBUG

The logging controls are part of the new LogConfigurationManager, implemented as a component, the `Log4jConfigurationManager`, in the util module. It can be disabled, if we really don't want it to do anything, with an entry in `sakai.properties`:

    enabled@org.sakaiproject.log.api.LogConfigurationManager = false

Logging levels are first established in the log4j configuration file we deploy from Sakai. The source for this file is in: `sakai/util/util-impl/log/src/conf/log4j.properties`. This is made into a jar and deployed to common (this is a requirement of log4j configuration in Tomcat). We can set logging levels in this file. For example:

    # Application logging options
    log4j.logger.org.sakaiproject=INFO

    # Ignore erroneous MyFaces variable resolver warnings
    log4j.logger.org.apache.myfaces.el.VariableResolverImpl=ERROR

This sets any logger with "org.sakaiproject.*" to INFO level, and the "org.apache.myfaces.el.VariableResolverImpl" to ERROR level. The problem with this approach, however, is that it requires a change to the source, and a redeploy. Hardly convenient.

Happily, logging levels can now also be specified in `sakai.properties`. This augments and overrides the levels set in the default config file. Example:

    log.config.count=3
    log.config.1 = ALL.org.sakaiproject.log.impl
    log.config.2 = OFF.org.sakaiproject
    log.config.3 = DEBUG.org.sakaiproject.db.impl

This uses the established (but awkward) method of having a name.count followed by name.1, name,2, etc. to form an array of strings for the value "name". In this case, the name is "log.config". The values are of the form `LEVEL.logger`, and the possible levels are: OFF TRACE DEBUG INFO WARN ERROR FATAL ALL.

As you can see, Sakai uses log4j for logging. See the official log4j documentation for more information about how to configure it if you have questions.

For Mac and *nix systems, the most important log is found in `logs/catalina.out` of Tomcat. It can be instructive to watch this log as Tomcat is starting up, by using a startup command like the following:

    bin/startup.sh; tail -f logs/catalina.out

Tomcat on Windows tends to be a little more puzzling about its logs, and it includes more of them, but its default behavior is to open `catalina.out` in a new window as soon as you start Tomcat. If you need more information about the logs in Windows, we'll refer you to the official Tomcat documentation.

Other Logs

The SMTP server logs from Sakai will be written to the `$CATALINA_HOME/sakai/logs` directory.

#### "Unsupported major.minor version 49.0"

If this message appears in catalina.out during startup, it means that you've built Sakai for Java 1.5, but your JRE is 1.4. Go back to the Set up Build Environment section and double-check your Java version and environment variables.

#### "java.net.BindException: Address already in use"

If this message appears in catalina.out during startup, it means that some other application is already running on the port you've assigned for this Tomcat instance. It's most likely another Tomcat. It may be that you didn't stop Tomcat from a previous run, or you may already have Tomcat running on that machine.

If you need to run more than one Tomcat on the same machine, see Tomcat documentation for instructions on how to configure to run Tomcat on a different port.