NASA Jet Propulsion Laboratory

Ogre

Oracle-Grok Interface


User's Guide

Robert Keller
Jeffrey Polakow

15 October 1997


Table of Contents

Introduction

Ogre Data Model
Relational Model
Object-Oriented Model
Mapping a Single Table to Object
Mapping Multiple Tables to Object
Using Ogre
Buttons
Viewing Relation Structure
Connecting Attributes
Generating Objects
Constraining Selections
Quitting Ogre
Object-Oriented Database Implementation

GrokDB Application Programming Interface

CORBA-Compliance of the Object-Oriented Database

Future Work

Acknowledgment

Glossary

Mouse Button Summary


Introduction

Ogre is a tool for converting information stored in an Oracle® relational database to an object-oriented database (OODB). The acronym Ogre (Oracle-Grok Interface) alludes to Grok, a tool for representing knowledge graphically and storing that knowledge in an OODB. The commonality between Grok and Ogre is currently only in the form of the database which each produces. Ogre functions as a tool independently from Grok, but is to be considered to be a part of the Grok suite of tools. The current version of Ogre provides two options for the target OODB: (i) an ObjectStore® database, or (ii) a database encoded as an ASCII file. Ogre is currently implemented in C++, ProC® (for the Oracle aspects), and Tcl/Tk, more precisely, in an object-oriented extension of Tcl/Tk available from ftp://ftp.tns.lcs.mit.edu/pub/otcl.

Ogre provides the following functions:

The GUI enables a user to prescribe the conversion without requiring any knowledge of SQL, the Oracle query language. Based upon a graphical representation constructed on the monitor, Ogre generates the necessary SQL automatically and populates the OODB with the results of the SQL query.

Ogre Data Model

Ogre is based on a relational data model, an object-oriented data model, and a mapping between them. We describe these components next.

Relational Model

The relational model is the traditional one:

For example, consider a military database with the following tables, which have been generated from an Oracle session:

Commander table
DIVISION CO
-------- --------
       1 Jones
       2 Smith
       3 Thomas
       4 Williams
Division table
DIVISION TYPE           QUANTITY
-------- ------------ ----------
       1 armor                20
       2 mechanized infantry 100
       3 infantry            500
       4 armor                30
Location table
DIVISION LOCATION
-------- ---------
       1 peninsula
       2 foothills
       3 plains
       4
Rank table
OFFICER      RANK
------------ ---------------
Jones        major
Smith        brigadier general
Thomas       colonel
Williams     lt. colonel
Controlled_by table
LOCATION         FORCES
---------------- ------
peninsula        red
foothills        blue
plains           blue

Object-Oriented Model

The object-oriented model is one derived from Grok's original data representation: The set of attributes for a type of object is referred to as the
schema for the object. The structure of the resulting object-oriented database is determined by the set of schemas, as configured using Ogre.

It is not required that the same attribute names be used in the objects Ogre creates as were used in the relations. Ogre allows the user to rename the attributes. In some cases, renaming is necessary, as ambiguity would otherwise result. Ogre provides visual indications of those cases.

We now present the relational-to-object mapping in two steps. The first step is the mapping of a single relation to objects. The second step is a mapping of multiple relations, sharing common attributes, to objects.

Mapping a Single Table to Objects

For example, suppose our table is

Division table
DIVISION TYPE           QUANTITY
-------- ------------ ----------
       1 armor                20
       2 mechanized infantry 100
       3 infantry            500
       4 armor                30

Then the schema will consist of

schema:

the name of the schema (generated by Ogre, but modifiable)

id:

the logical identifier or name for the object

division, type, quantity:

data values derived from the original relations

and the corresponding set of objects will be:

        id: ob1
            division: 1
            type: armor
            quantity: 20

        id: ob2
            division: 2
            type: mechanized infantry
            quantity: 100

        id: ob3
            division: 3
            type: infantry
            quantity: 500

        id: ob4
            division: 3
            type: armor
            quantity: 30
Note that each object has been given an id for the system, which uniquely identifies that object.

Each object also has a schema attribute which is not shown above. The name of the schema is system-generated, although it can be set by the Ogre user.

One or more attributes in a row of a table may have unspecified values, formally known as NULL values. By design, Ogre does not create attribute-value pairs for such attributes, since part of the motivation is to allow condensed representation of data in which the tables are fairly sparse. For example, in the Location table presented earlier,

Location table
DIVISION LOCATION
-------- ---------
       1 peninsula
       2 foothills
       3 plains
       4

the location of division 4 is NULL, representing unknown. If this relation were converted to objects, only three of the objects would have a value for attribute LOCATION. The fourth object would be created, but would not have an attribute LOCATION.

Clearly our model is only one possible model which could be used. For example, another model would preserve the attribute in all objects, but create some kind of NULL value for the attribute in the object. As part of our objective is to reduce the space required for sparse relations (ones in which NULL's are frequent), we chose the current model.

Mapping Multiple Tables to Objects

A degenerate form of mapping for multiple tables would involve simply repeating the above single-table relation mapping once for each relation. However, this may not capture the intent of the original relational database. Typically relational databases distribute the description of single objects over multiple tables (through a design process known as "normalization"), and the attribute values of one or more tables would then be linked. We would like to recapture those linkages in creating objects. As the exact nature of the linkage cannot be inferred from the tables alone, it is up to the user to specify the linkage, using Ogre's GUI.

Suppose there are two tables R and S with a common column attribute A and we intend that rows of R and S having the same value of A really designate the same object. We can indicate this intention in Ogre's GUI by making a connection between nodes representing these attributes. In relational database terminology, this is conceptually a join of the two tables.

For example, consider the two tables which have a common attribute DIVISION:

Commander table
DIVISION CO
-------- --------
       1 Jones
       2 Smith
       3 Thomas
       4 Williams
Division table
DIVISION TYPE           QUANTITY
-------- ------------ ----------
       1 armor                20
       2 mechanized infantry 100
       3 infantry            500
       4 armor                30

Identifying the attributes DIVISION in the two tables, Ogre could create the following more comprehensive objects:

        id: ob1
            co: Jones            
            division: 1
            type: armor
            quantity: 20

        id: ob2
            co: Smith
            division: 2
            type: mechanized infantry
            quantity: 100

        id: ob3
            co: Thomas
            division: 3
            type: infantry
            quantity: 500

        id: ob4
            co: Williams
            division: 3
            type: armor
            quantity: 30

Ogre makes it possible to link any number of relations in this way. When the linkages describe the user's desired interpretation of the relations as objects, Ogre will present a query to the relational database to extract that information. Ogre will then generate the corresponding objects. The user also can inspect both the original relations and the resulting objects using the GUI.


Using Ogre

Currently Ogre runs on the Sparcstation jake.jpl.nasa.gov, and is configured to access a sample database derived as an excerpt of the ASAS database. Currently Ogre only runs on this database, which is an excerpt of the full ASAS database, with other test relations added.

To leave the application, select close on the main Ogre window.

Buttons

The following is the list of buttons at the top of the Ogre window and their meanings:

Tables
Enables viewing or loading of tables from the relational database.

Schemas
Show the schemas (list of attributes) for the object-oriented databases to be created, as defined by the current screen contents. Schemas can be renamed by double-clicking the left mouse button on the schema name in this menu.

Get Objects
Causes creation of the object-oriented database from the relational one according to the current schemas.

Show DB
Shows the objects currently in the Grok database.

For example, if Show DB is clicked, a window will open displaying whatever is in the Grok database initially. This could be empty, or data which is there from a previous session. As additional objects are created, they will also be displayed in the window. Clicking Clear will clear all data from the database, therefore this button should be used with great discretion.

Viewing Relation Structure

Once the Ogre window appears, click the Tables button in the menu bar. A list of the tables in the relational database should appear, as shown below:

To view the contents of a relation, or to load the relation for conversion to objects:

Suppose, for example, that we click the tables COMMANDER, CONTROLLED_BY, DIVISION, LOCATION, and RANK, as shown next:

If we then click View, we will get a window for each relation chosen. For example, the window for relation DIVISION might show as follows:

If we click Load, we will get the following display:

Each oval corresponds to one relation. The boxes correspond to the attributes of that relation. The labels in the boxes are initialized to the attribute names from the relations. However, these labels may be changed, for example by joining attributes with different names or by explicit intent. The original names of the attributes can be displayed on the links between the ovals and boxes, by double-clicking the left mouse button. This action toggles the link labels. The original link labels cannot be changed, as they are part of the Oracle database.

At this point, we may choose to move the relations to suit our taste. To move an entire relation, with its attributes attached, double-click with the right mouse button to highlight the relation, then drag with the left mouse-button. Moving the CONTROLLED_BY and LOCATION relations results in the following display:

Connecting Attributes

When an attribute is dragged using the right button, it should be dragged to where it is over another attribute of similar type (numeric, string, etc.). Doing so causes the two attributes to become linked, via a single box. As discussed above, this means that the rows of the linked relations represent a single object.

Attributes can be unlinked by clicking the left mouse button over the box. One then has a choice of:

change name
change the name of the attribute (which will become the attribute name of any generated object) to one of the user's choice.

hide
hide the attribute. This attribute will not appear in the resulting objects. However, any linkages will remain intact and used to constrain the creation of objects from multiple tables in the same way as if they were to be included in the result.

unlink
remove the most currently created linkage

unlink all
remove all linkages for this attribute

Note that when a linkage is made, if there are other attributes with common names, these will be highlighted, since they are regarded as conflicting. In other words, attempting to create objects using this naming scheme would cause ambiguities. These ambiguities must be resolved before Ogre will allow objects to be created.

In our example, suppose we consider attribute DIVISION to be the same in the COMMANDER and DIVISION relations. We can join these two attributes. First we drag (using the middle mouse button) the DIVISION attribute of COMMANDER away from the relation to distinguish it. Then we drag (using the right button) of the DIVISION attribute of DIVISION atop the DIVISION attribute of COMMANDER. Using the right button, rather than the middle one, tells Ogre that we wish to join the two relations on that attribute. The window would now appear as follows:

Let's now also join the DIVISION attribute of the LOCATION relation atop the joined DIVISION attributes, again using the right mouse button. Let's also join the two LOCATION attributes, first using the middle mouse button to reposition, then the right mouse to join. Finally, let's similarly join the two OFFICER attributes. The result is:

If the user decides he/she did not wish to link two or more attributes, the left mouse button may be clicked over the attribute box while holding the shift key. This will bring up a menu of options which includes unlink, selecting which will cause the attributes to be no longer identified.

Note that it is considered an error to drag with the right mouse button when no linking of attributes is intended. If the user does this, the following dialog will appear, until dismissed:

This is a standard Tcl dialog box, which will be replaced with something a little friendlier in the production version.

Generating Objects

Once the relations are properly linked to suit the user, objects may be created. There will be one type of object for each connected sub-graph in the window. In the present example, there is only one sub-graph, so there will be one object type. There will be one attribute of the object for each distinct attribute showing in the window. The set of attributes of an object is referred to as the Schema. Thus the number of schemas also corresponds to the number of connected sub-graphs. By clicking the Schemas button at the top of the window, a list of current schemas and their attributes will be shown. The system will add one attribute to each schema which names the schema. (This system-generated name can be changed by the user.) For the current example, opening the Schemas window produces something like the following, where the actual schema name will depend on prior activities:

To add objects to the current Grok database, the user clicks Get Objects. If the GrokDB window is open, the objects created will appear in the window, as shown:

Addition to the Grok database is cumulative. If the user reconfigures the window, e.g. by removing some relations and bring in others, then clicks Get Objects, those objects will be added to the database.

Note that it is possible to create ambiguous schemas if two relations are linked but have one or more pairs of attributes with the same name. This would be tantamount to creating an object with having two attributes of the same name, which would not be helpful. In this event, the boxes of the identical attributes in question are highlighted in red. These should be either joined or one attribute renamed before proceeding. Ogre will not generate objects until the ambiguity is resolved.

Constraining the Selection of table entries

The user may add constraints to the selections from tables used to create objects. These are expressed graphically, but translate into SQL selection conditions.

Examples are:

These additional options are achieved by pressing the left mouse button on the background, to get a menu, then making selections for drawing appropriate constraints or constants.

Quitting Ogre

Ogre is left by selecting Close from the menu which pops up from the box in the very upper-left corner of Ogre's main window.

Object-Oriented Database Implementation

The object-oriented database is implemented in C++ using Keller's
Polya package for the data structuring. Polya is structurally richer than is required for this application, but it was useful in staging the development. We first created an in-memory database which we call grokDB using Polya, which replaced the original now-dysfunctional (for Grok) Shore database. We then developed Ogre by targeting this grokDB, and concurrently made an ObjectStore® version of Polya and grokDB. The ObjectStore version was then installed into Ogre.

The structure of grokDB is quite simple: Each object is represented as a list of attribute-value pairs. The list is open-ended, rather than being a fixed schema, so that new pairs can be added after an object is initially created, and existing pairs can be removed.

Because of Polya's generality, the values and attributes can even be list structured themselves. While Ogre does not currently generate databases which exploit this feature, the feature might be useful for adding additional structure later on.


GrokDB Application Programming Interface

Our implementation of GrokDB has a separable application programming interface. All objects created by Ogre are done with respect to this
API. Since other applications may wish to ask the data created by Ogre, we present the API here. In the section to follow, a CORBA version of the API is presented. Here we use C++, with the actual headers are defined in source file grokDB.hh.

Type Poly is a polymorphic object, one which can be either a string, an integer, a float, or a list of Polys. A LID (logical id) is a Poly used as the identifier of an object. Every object in a grokDB has a unique LID.

Methods and Constructors for Grok Databases

grokDB() constructor

constructs a Grok database

int findObject(LID lid, grokObject &found) method

determines, by return value, whether an object identified by lid exists in this database. If so, then found is set to that object.

int removeObject(LID lid) method

removes the object identified by lid from this database. The return value indicates whether the object exists.

grokObject & newObject(LID lid) method

creates a new object with indicated LID in this database.

grokObject & newObject() method

creates a new object with system-generated LID in this database.

Polylist all() method

returns a Polylist of the attribute-value lists of all objects in this database.

void pretty(ostream &out) method

creates a pretty-printed representation of the database on out.

 
 
Methods for Grok Objects

int get(Poly attribute, Poly &value) method

determines, by return value, whether this Grok object has an attribute, and if so sets value to the value of that attribute.

void set(Poly attribute, Poly value) method

sets the value of this Grok object's attribute to value, whether or not the attribute had a value before.

Poly getLID() method

returns the logical ID of this object.

Polylist all() method

returns a list of all of this objects attribute-value pairs.

void pretty(ostream &out) method

creates a pretty-printed representation of this object on out.


CORBA-Compliance of the Object-Oriented Database

It was desired to make the databases created by Ogre to be CORBA-compliant.
CORBA stands for Common Object Request Broker Architecture, a protocol specified for distributed object-oriented access by the Object Management Group, a consortium of vendors. CORBA compliance is illustrated by the following IDL (Interface Definition Language) specification of the grokDB. This specification was correctly compiled using the Visigenic® Orbeline IDL compiler. Note that the current version does not actually use CORBA; this would require selecting an appropriate ORB (Object-Request Broker), then reconstructing the GrokDB to make CORBA calls on the ORB.

//
// IDL for grokDB
// 

module GROK
{

// These are the supported variants of a Poly, 
// the type of object from which a GrokDB is constructed

enum Polytype {Long, Double, String, List};


// This defines a Poly 

union Poly
  switch(Polytype)
    {
    case Long:   long           nv;
    case Double: double         dv;
    case String: string         sv;
    case List:   sequence<Poly> lv;
    };


// This defines an object in a Grok database

struct grokObject
  {
  Poly id;
  sequence<Poly> pairs;
  };


// This defines the available transactions for a Grok database

interface grokDB
  {
  // Create a new object with a given id

  void newObject(in Poly id);


  // Create a new object with a system-generated id

  void newOb();


  // Add an attribute-value pair to an object

  void add(in Poly id, in Poly attr, in Poly value);


  // Get the value corresponding to a particular attribute

  Poly get(in Poly id, in Poly attr);
  

  // Drop the specified object from the database

  void drop(in Poly id);


  // Exception raised when a specified object doesn't exist

  exception noSuchObject{};
  };
};


Future Work

It may be useful to provide additional ways to structure the created objects within Ogre, such as Ogre's GUI also provides a natural way to show the integration of several relational databases simultaneously, since the relations displayed by Ogre are not attached in principle to their databases. This would require significant additional programming beyond the current code, in the underlying system however. In particular, the current Ogre model deals with the necessary joining by generating the proper SQL; but an SQL is relative to a given database. In order to join across databases, one would have to either replicate all of the machinery of an SQL query processor or, alternatively, create a temporary database into which are copied each of the desired relations from the original databases.


Acknowledgment

The authors are uncertain of the origins of Grok. Our first exposure to it was through Walter Bunch at JPL, who apparently ported it to UNIX® using Tcl/Tk. A previous version of Ogre originated as a Harvey Mudd College Computer Science Clinic project in 1996-97, performed by Elizabeth Coleman, Chris Sloan, and Kevin Sours, with Robert Keller as faculty supervisor and Walter Bunch and William Lincoln as liaisons. The current version is a complete rework of the ideas discussed there, as well as a new implementation. The GUI of the current version builds on the GUI ideas of another HMC CS Clinic project in 1995-96 known as
Condor, which was performed by Will Ballard, Ingo Muschenetz, Jeff Polakow, and Amy Ward, with Robert Keller and Josh Hodas as faculty supervisors.

The authors wish to thank Lamark Johnson for his advice on use of ObjectStore and its installation, and Dr. Bill Lincoln for his assistance with Grok and compiler installation. The advice and steering of James Hauge and Dr. Nevin Bryant were invaluable throughout the project.


Glossary

API
Application Programming Interface

ASAS
All Source Analysis System

ASCII
American Standard Code for Information Interchange, essentially an ordinary text file.

attribute
A quality of a data item to which a specific value may be associated. Attributes appear in relational databases as column headers and in object-oriented databases as variables in each object.

Condor
A graphical query interface for relational databases conceived and prototyped at HMC as part of the Grok project. The Condor prototype is implemented in XPCE (X-Window version of the Portable Computing Environment from SWI in the Netherlands) and Prolog.

constraints
A condition imposed on the selection of objects from a table in addition to join conditions.

CORBA
Common Object Request Broker Architecture

Grok
Graphical representation knowledge, a software tool for extracting information from textual articles and creating graphical objects representing entities in the article and connections between those objects.

GrokDB
An object-oriented database derived from the way in which Grok stored information. The current GrokDB is implemented in C++ using Polya, although other implementations are possible. An IDL definition of GrokDB is defined in this document.

GUI
Graphical User Interface

IDL
Interface Definition Language, a language used to specify a CORBA interface. IDL does not provide an implementation.

join
a relational database term, meaning to create a table from two or more tables by creating rows where the attributes have common values. Ogre provides for joining on operators <, >, IN, etc. as well as equality.

LID
Logical Identifier, used to access to objects in a GrokDB

null values
relational databases allow the values of some attribute to be missing. These are called null and show up as blank in the tables. When Ogre creates objects, it suppresses the attributes corresponding to null values. Rows of a table are not joined if either joining attribute has a null value.

Ogre
Oracle-Grok Interface, the topic of this report, is a software tool for creating an object-oriented database from an Oracle database.

OODB
object-oriented database, a database in which the primary unit of structuring is an object. In our case, each object consists of a set of attributes with corresponding values of the attributes.

Poly
polymorphic object, a general data type used in Polya and the basis for the data in the Grok database created by Ogre.

Polya
A software library for creating polymorphic lists and arrays in the C++ and Java languages.

relation
Another name for a table of data. The data items are identified with rows of the table. Each item specifies values of attributes, which are identified with the columns of the table.

relational database
A database in which the primary unit of structuring is the relation or table, in contrast to an OODB.

schema
In this document the schema may be regarded as the set of attributes for each type of object which will be created.


Ogre Mouse Button Summary

The following conventions are used in Ogre:
Left Button
Shift Key and Left Button
Used to give menus of options for the user
  • If an item is selected, gives a menu of options for the item.
  • If no item is selected, but the cursor is over an item, gives options for the item.
  • If neither of the above, gives a menu of options for creating constants and constraining relations (used to perform a selection of objects from the relational database).

Double-Click Left Button
Used to select an attribute of a relation, or an entire relation.
  • Clicking over a link (between a relation and attribute) toggles the display of the name of the attribute, which identifies the attribute's name in the original relation.

  • Double-clicking on the schema name brings up a dialog to allow the user to enter a new name.

Drag Left Button
Create and resize a selection box. When the button is released, items inside are selected.

Middle Button
Drag to move the item to which the mouse is pointing.

If the mouse is not pointing to any item, drag to scroll the canvas.

Right Button
Drag an attribute box to another attribute box for purposes of joining. The join takes place when the button is released.

Note: If no joining is actually done when this button is used, an error dialog will appear to warn the user that no connection was actually made.