Lincoln D. Stein, lstein@cshl.org
To use the socket server version of AceDB (www.wormbase.org, port 2007) you will need the experimental Jadex library. Otherwise, for the older RPC version of AceDB, use jade version 1.0j.
The Jade class libraries and installation guide can be downloaded from the following locations:
Jade is still a work in progress. The underlying database interface layer is stable and well documented, but the graphical layers are liable to change without notice. Some of this instability is due to the labile nature of Java itself, which has undergone dramatic changes to the Abstract Windows Toolkit (AWT) in recent months.
This document describes the steps required to install Jade over an ACEDB database.
Jade consists of several software components, some of which are optional:
Because only the Unix version of the ACEDB server is currently supported, you must have a Unix machine available to install the database itself. In addition, jade2ace is designed to operate under the supervision of the Unix inetd super daemon. It may work with Windows and Macintosh versions of inetd, but this has not been tried.
The ACEDB server and jade2ace are part of the standard ACEDB 4_5e release. However jade2ace is only available in source code form. Therefore you will need an ANSI-compliant C compiler as well.
The Jade class library and the Web server can run on any platform that supports the TCP/IP protocol.
You will need a Java interpreter in order to run Jade. Many modern Web browsers, including Internet Explorer and Netscape Navigator, have built-in Java interpreters that are suitable. However, as the Java language has been evolving rapidly, you may find that not all features work correctly on older Java-capable browsers.
We provide the Jade class libraries in precompiled form as well as in source code. If you wish to use the ACEDB browser applet and class libraries as is, you only need the precompiled libraries. However, if you wish to extend the applet or libraries, you will need a Java compiler. A freely-distributable compiler is available in the form of the Java Development Kit, which has been ported to multiple platforms (see below for a partial list). Any of the commercial development environments, such as Visual Age or Java Studio should work too.
| OS | Site |
|---|---|
| Solaris | http://java.sun.com/ |
| Windows NT | http://java.sun.com/ |
| Macintosh | http://java.sun.com/ |
| Linux | http://www.blackdown.org/ |
| Digital Unix (OSF/1) | http://www.osf.org/mall/web/JDK/ |
| HPUX | http://www.osf.org/mall/web/JDK/ |
| AIX | http://www.osf.org/mall/web/JDK/ |
ACEDB is an object oriented database system which was first distributed in 1990 to maintain the genetic and molecular data of the nematode C.elegans. It was written in collaboration with Richard Durbin and with the help and contribution of many other programmers. In 1995, ACEDB was rewritten to follow a client/server paradigm in which aceserver, the database server, uses the network to respond to requests for data from several database clients. The client/server version of ACE is fast, scaleable and reliable. It runs on everything from 486 portables to high-end DEC servers.
ftp://ncbi.nlm.nih.gov/repository/acedb/
ftp://ftp.crbm.cnrs-mop.fr/pub/unix/acedb/
Decompress the distribution with uncompress, and then
unpack it with tar. One safe way to do this is:
% mkdir ace % cd ace % zcat ../ace.4_5e.tar.Z | tar xvf -
The installation directions can be found in the file wdoc/README, along with a script to automate the build and install process in wtools/INSTALL. Follow the directions to create and install the standard X-windows graphical database and associated utilities.
The INSTALL script does not, by default, install the aceserver and jade2ace binaries. However, they should be created in the bin.your_machine_architecture directory that is created during the build process. If for some reason these two programs have not been built, follow these directions:
setenv ACEDB_MACHINE SOLARIS_4_OPT (solaris with optimization)
You'll now want to copy these three files into the ACEDB home directory, which you selected during the execution of the INSTALL script. We will call this home directory $ACEDB throughout the rest of this document:
% cp bin.$ACEDB_MACHINE/{jade2ace,aceserver,aceclient,tace} $ACEDB/bin
% chmod +x $ACEDB/bin/{jade2ace,aceserver,acelient,tace}
Now run aceclient by issuing the command:% $ACEDB/bin/aceserver $ACEDB 20000 1200:1200:0 Database directory: /usr/local/acedb
% $ACEDB/bin/aceclient localhost -port 20000
acedb@localhost> status
// Reponse: 1628 bytes.
//
************************************************
// ACEDB version 4.5, linked Mar 3 1998
// Data directory /usr/local/acedb/, release 4-0
// Session : 2, User , Write Access No
// Global Address 29
// Stream level 2
// Disk: 442 blocks used, 3526 blocks available.
// : 0 blocks allocated and 0 freed in this session.
lots more information deleted for brevity
************************************************
// 0 Active Objects
acedb@localhost> shutdown
// Reponse: 24 bytes.
// 0 Active Objects
acedb@localhost> quit
//Bye
If you have a choice of machines to install aceserver on, you should choose a fast machine with plenty of local disk space available for the databases. Although possible, it is not recommended that you have the server read its database from a network mounted (e.g. NFS) disk.
To have aceserver launched automatically, you will need to add it to the /etc/rpc list of registered services, and to convince the inetd super-daemon to launch it when necessary.
aceserver uses the Sun portmapper service to dynamically assign it a port to use for its RPC calls. Some, but not all, implementations of portmapper expect the programs under portmapper's control to begin with the prefix rpc. To be safe, you should create a symbolic link to alias aceserver to rpc.acedb like this:
Next you need to choose a RPC service number for aceserver to listen on. This is a bit of black magic. Pick a service number greater than 100000 that isn't already mentioned in /etc/rpc. In this document, we'll assume port 500000, which is well out of the range used by other services. Open /etc/rpc and add a line like the following to the bottom:lrwxrwxrwx 1 lstein lstein 9 Apr 1 13:59 rpc.acedb -> aceserver*
If you wish to run multiple ACEDB databases, just declare them, using a different name and port number for each one.acedb 500000
Next you must tell inetd to automatically launch aceserver when it is needed. Open /etc/inetd.conf and add a line like the following (you will need to be root to do this):
acedb/1 stream rpc/tcp wait acedb /usr/local/acedb/bin/rpc.acedb
rpc.acedb /usr/local/acedb/elegans 500000 1200:1200:0
Now test the inetd side of the equation by running aceclient localhost -port 500000. You should get a prompt and be able to type queries and other commands (try typing "help").kill -HUP `ps -aux | grep inetd | grep -v grep | perl -nae 'print $F[1]," "'`
If this doesn't work, you may have to send a HUP signal to the portmapper daemon as well (Linux systems sometimes show this behavior). Do so in the same you you did with inetd. Now, if you run the command rpcinfo -p localhost you should see that the server is registered with portmapper:
This confirms that portmapper is correctly reconfigured.% rpcinfo -p localhost program vers proto port 100000 2 tcp 111 portmapper 100000 2 udp 111 portmapper 100005 1 udp 666 mountd 100005 2 udp 666 mountd 100005 1 tcp 669 mountd 100005 2 tcp 669 mountd 100003 2 udp 2049 nfs 100003 2 tcp 2049 nfs 500000 1 udp 1136 acedb 500000 1 tcp 5687 acedb
The format of inetd.conf is, blessedly, unchanged.acedb 500000/tcp
Solaris does not have a portmap daemon, so don't worry about trying to restart it.
There are also some problems in certain versions of Solaris relating
to apparent bugs in Sun's implementation of RPC. Quoting from the ACE
SERVER.INSTALLATION guide:
Known SOLARIS bug:
ON solaris 5.5, the server does not connect when registered with inetd
this is due apparently to a bug in rpcgen
Symptom:
if you start the server in foreground:
rpc.acedbd $ACEDB 600:600:0
all works ok and the client works
but started by inetd, the server can be seen as running:
ps -elf | grep ace
but you cannot connect with the client
To fix this:
setenv ACEDB_MACHINE SOLARIS_4_OPT
make aceserver gifaceserver
cd bin.SOLARIS_4_OPT
rm -f rpcace_svc.c
cp ../wrpc/rpcace_svc.crick rpcace_svc.c
touch rpcace_svc.c
rm aceserver gifaceserver rpcace_sp.o
cd ..
make aceserver gifaceserver
**************
problem is the following:
rpcace_svc.c is a c code automatically generated by rpcgen
this code is bugged
rpcace_svc.crick is a version that was generated under solaris.5.5.1
rpcace_svc.c is actually included in rpcace_sp.c at compile time
this is why you must rm rpcace_sp.o, then force recompilation
The file $ACEDB/server.log will contain a record of the server activity. Since it grows without limit, you will have to empty it from time to time using a command like:
cp /dev/null $ACEDB/server.log
You'll need to declare a TCP service port for jade2ace in /etc/services, then edit /etc/inetd.conf to launch jade2ace to handle incoming transactions.
Pick a TCP port for jade2ace to run on. Again, this is a bit of sorcery. Choose a port that is greater than 1000, not already mentioned in /etc/services, and not one of the undocumented ports in the 6000-7000 range that the X Windows systems likes to tromp on. Also, because of limitations in the Java representation of an integer, the port you choose should be lower than 32,767. We will use 5000 for our examples.
Open /etc/services, and add a line like the following:
If you are planning to run multiple servers side by side, you can add additional names and ports at this time.jade2ace 5000/tcp
Open /etc/inetd.conf and add a line like the following:
Please note that the line above has been folded to fit on the page. Keep the actual entry on a single line.jade2ace stream tcp nowait acedb /usr/local/acedb/bin/jade2ace jade2ace -host localhost -port 500000
Things to notice about this entry:
% telnet localhost jade2ace Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 200 Ace server ready and waiting help 302 - Commands: 302 - FIND SHOW FOLLOW TABLE NTABLE MODEL GIF 302 - LIST IS QUERY UNDO QUIT HELP 302 - You need to know the Ace query language to use this 302 - service. Sorry. 302 End of HELP info quit 201 A bientot Connection closed by foreign host.
jade.tar.Z to a directory with at least 20 MB of
free space, uncompress and untar it with the command:
This will create a directory jade.version_number containing:zcat jade.tar.Z | tar xvf -
The file JadeBrowser.class is an applet file for the generic ACEDB browser. TestJade.class is a standalone application that runs a variety of tests on the Java to ACEDB connectivity layer and can be used to verify an installation.
If you are intending to use the ACEDB browser applet, then you must install the jade tree into some part of the Web server document tree. We keep all our Java class libraries and applets in a java subdirectory. Adjust according to your preferences.
If the connection to France is slow, this test may take a few minutes to complete. Please be patient. If you are in the United States, you might wish to try a closer server located in New York. The command-line invocation will be:
java TestJade stein.cshl.org 20006
If the test fails immediately with a "can not connect" error, then chances are that the remote database is temporarily down. Try again later. See the caveats about the range of port numbers at the end of the next section for another common problem.
The file install/tace.in contains the appropriate commands to update the database using tace. Run it this way:
Previous versions of the Jade installation guide instructed you to paste the contents of models.jade.ace into the main wspec/models.wrm file. This is no longer necessary, as the Jade model is now hard-coded into ACEDB.% cd install % $ACEDB/bin/tace < tace.in Database directory: /usr/local/acedb Type ? for a list of options acedb> // 6 objects read with 0 errors // New models read, save your work before quitting // 0 Active Objects acedb> // 1 objects read with 0 errors// 1 Active Objects acedb> // 8 objects read with 0 errors// 8 Active Objects acedb> // 8 Active Objects // A bientot
Create an HTML file containing an <APPLET> tag following this model:
<applet codebase="/java" code="jade.JadeBrowser" width=400 height=160> <param name="title" value="Welcome to ACEDB"> <param name="netclient" value="stein.cshl.org"> <param name="port" value="20006"> </applet>Replace codebase with the location of your Java class libraries and applets. Replace the value of the netclient parameter with the host name of the machine that jade2ace is running on (currently it must be the same host as the Web server). Replace the value of the port parameter with the port that jade2ace is listening to (see below for details).
When you fetch this page with a Java-capable Web browser, you should see a welcome banner. The banner will be orange while the applet attempts to establish a connection to ACEDB, and turn green when the connection is successful.
If the applet fails for some reason, the banner will turn bright red. Open the Java console window to see what might have gone wrong. Typical errors include connection timeouts to a non-running server and security violations due to an attempt to connect to a jade2ace running on a different machine than the Web server. Another common gotcha is that Java doesn't have unsigned integers, only signed ones. On Digital Unix/alpha machines (which Jade was developed on) this doesn't pose a problem because integers are four bytes long. But on some other architectures they are two bytes, leading to unexpected problems when a high-numbered port is opened. If you are unable to establish a connection and are using a high-numbered port, try using a port number that is within Java's byte range (i.e., no higher than 32,767).
Although it is tempting to use these Java objects directly as the basis for graphical displays, our experience with ACEDB development has convinced us that a less tightly-integrated approach is desirable. Data displays should be decoupled from the schema as much as possible. Instead, the correspondence between a database object and its graphical representation should be determined by an explicit, documented linkage. The user interface should also take care that any graphical element that appears on the screen is associated with a corresponding object in the database [Jean, is this what you meant?].
To implement this behavior, all the data that drives our graphic displays is retrieved from the database in the form of simple tables declared by name on the server side. The tables are essentially simplified "views" on the more complex object-oriented ACEDB schema. This system has the following advantages:
At any time, data retrieved from the first server can indicate the address of some other server to which Jade can establish independent connections (see below). Depending on the applications, the data retrieved from different servers can be merged or maintained separately.
Jade is also capable of representing and retrieving ACEDB objects in their full object-oriented tree form, translating ACEDB pointers into Java-style references. We use this possibility in the plain Tree-display of the objects, which implements hypertext facility. However, to write a graphic display, like a genetic map, in the object-oriented form, sacrifices some of Jade's portability, because it assumes a certain database structure and schema.
In the section on updating the ACE database models, we showed you how to modify the ACEDB database to support this indirect linkage between displays and objects. To add your own displays, you will need to do three things:
The Jade object supports two tag types, StandaloneDisplay and Display. StandaloneDisplay is used for "top level" display classes that are not associated with a single database object, such as those responsible for selecting among multiple objects. For example, in the grass database demo database, you will find the definition:
The jade.grassplot.MapSetPick display code is responsible for finding all Oxford grid maps in the database and displaying them in a selection list for the user to choose among. By following the Jade graphical class conventions, the browser is able to automatically determine its name and createf a menu entry labeled "Oxford Grid Selector" in the Display menu of the Jade main window. When the menu item is selected, the class is invoked. [XXX ? Jean, can we distribute the full grassdb demo, along with the appropriate Jade and table .ace files so that people can see? ]Jade : default StandAloneDisplay jade.grassplot.MapSetPick
When there is a one-to-one correspondence between a database object and a display, you should use the Display tag. Consider this definition:
There are two Display tags here. In the first case, objects of type "Chromosome" are mapped to the Java class "jade.maps.Vmap" (vertical genetic map). Like standalone displays, this Java class is expected to implement the Jade graphical interfaces described below. Whenever the browser needs to fetch a Chromosome from the database, the class's constructor will be called and passed the database object. It is free to make additional database queries in order to fetch other information that it might need.Jade : default Display Chromosome jade.maps.Vmap Display Sequence jade.maps.Vmap map.interval link.interval
The second Display line, which maps Sequence objects to jade.maps.Vmap, is more interesting. Internally, jade.maps.Vmap invokes a table query named "map.interval" to fetch the list of intervals between all the loci on the genetic map. However, in this case we wanted to reuse the genetic map display to show the relationship between contigs across a large assembled sequence. However, the ACEDB genetic and physical map models are very different. In this case, two optional fields remap all requests for the "map.interval" table query to requests to the "link.interval" table query. The two tables have a very different definition on the server side, but export semantically equivalent tables to the display code. Thus the genetic map display is tricked into displaying a physical map, with little programming effort on our part.
The important feature on the server (jade2ace) side is to be able to answer a request of the form:
The table name refers to a predefined query in the database, and the remaining arguments are optional run time parameter values to be bound to the query at run time.NTABLE table_name [param1 param2 param3...]
The format of the ACEDB table definition file Jade.tables.ace is undocumented. However, table definitions are easily constructed using the acedb xace graphic interface, starting the Table-Maker from the main menu, and saving the definition as an ace object via the new "Save Def" button. Unfortunatelly, the current ace.4_3 interface does not correctly allow the handling of run-time parameters. One must finish the preparation by viewing the resulting table definition in tree-update mode and adding the desired parameters by using the values "%1", "%2", "%3" to indicate the first, second, third parameters, and so forth. The resulting Table class object should then be exported as an .ace file.
All Jade displays must implement the Displayable interface and declare the following method:
public String nickName () { return "Some descriptive string" ; }
Each display class should declare a constructor which takes a single AceObject argument. When the class is instantiated, it can use the AceObject to fetch information about the object it is responsible for displaying. It can also use the AceObject to access other objects in the database that it needs information about.
Please refer to the (incomplete) programmer's API, and to the code itself for examples of how to do this. More detailed information will be forthcoming in future releases.
These lines associate the names ricegenes, maizedb... to the ports number 20211 and 20212... corresponding on the same host to the jade2ace services of the rice, maize... servers.<applet code=jade.maps.Welcome.class width=400 height=150> <param name="title" value="Grass@beta"> <param name="netclient" value="alpha.crbm.cnrs-mop.fr"> <param name="port" value="50000"> <param name="otherdbs" value="ricegenes 20211 maizedb 20213 graingenes 20212 sorghumdb 20214">
The first connection is established with the web host on the principal port, 20210 in this example. Later on, the display may require additional connections to other servers.
Inside the Java display code, the need to call the secondary servers is hard coded, including a name recognition system, and results in a call:
jade.db.MultiAccess.display("ricegenes", "Map", locus.name)
The relevant point is that although the need to call the rice server is part of the display code, the actual location of the server is part of the configuration of the main server.
The grass demo is running at http://alpha.crbm.cnrs-mop.fr/jade/grass.html,
You will see a number of deprecation warnings. This is nothing to be concerned with.% javac -d . src/jade/*.java src/jade/*/*.java
If you have the GNU make utility, you can make use of it to achieve a slightly quicker and more intelligent type of recompilation. Here's an example:
Each of the subdirectories, db, grassplot, etc. has its own Makefile. To rebuild just the contents of that subdirectory, change to it and type make.% cd jade % make
This document and software may be redistributed as long as it is distributed in full and the above copyright statement remains intact. Any additions or modifications should be expressly labeled as such.
THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.