Escolar Documentos
Profissional Documentos
Cultura Documentos
Document Version
Doc6.4.0a RGUIDEv1 8/18/04
Copyright
Copyright 1998-2004 Art Technology Group, Inc. This publication may not, in whole or in part, be copied, photocopied, translated, or reduced to any electronic medium or machine-readable form for commercial use without prior consent, in writing, from Art Technology Group (ATG), Inc. ATG does authorize you to copy documents published by ATG on the World Wide Web for non-commercial uses within your organization only. In consideration of this authorization, you agree that any copy of these documents which you make shall retain all copyright and other proprietary notices contained herein.
Trademarks
Dynamo is a registered trademark of Art Technology Group, Inc. ATG Dynamo Application Server, ATG Relationship Management Platform, ATG Scenario Personalization, ATG Portal, ATG Commerce, ATG Publishing, ATG Data Anywhere Architecture, and ATG Control Center are trademarks of Art Technology Group, Inc. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries. This product includes software developed by the Apache Software Foundation (http://www.apache.org/). All other company and product names are the trademarks or registered trademarks of their respective companies.
No Warranty
This documentation is provided as is without warranty of any kind, either expressed or implied, including, but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. The contents of this publication could include technical inaccuracies or typographical errors. Changes are periodically added to the information herein; these changes will be incorporated in the new editions of the publication. ATG may make improvements and/or changes in the publication and/or product(s) described in the publication at any time without notice.
Limitation of Liability
In no event will ATG be liable for direct, indirect, special, incidental, economic, cover, or consequential damages arising out of the use of or inability to use this documentation even if advised of the possibility of such damages. Some states do not allow the exclusion or limitation of implied warranties or limitation of liability for incidental or consequential damages, so the above limitation or exclusion may not apply to you.
Contact
ATG 25 First Street Cambridge, MA 02141 617.386.1000 phone 617.386.1111 fax www.atg.com
Contents
1 2
1 5
6 6 7 9 9 9 10 11 11 13 14
Repository Queries
Repository Query API atg.repository.QueryBuilder atg.repository.QueryOptions Repository Query Examples Repository Queries in the ATG Control Center Repository Query Language RQL Overview Comparison Queries Text Comparison Queries Property of Property Queries Logical Operators Collection Queries Includes Item Queries Is Null Queries Count Expressions All Query Full Text Search Queries ID-based Queries Order By Directives Range Directives Parameters in Queries
17
17 17 18 19 22 22 23 24 24 24 25 25 26 26 26 26 26 27 28 28 29
iii
Contents
Parameterized Field Queries RQL Examples RQL Grammar
29 30 30
4 5
33
33
35
35 37 38 38 39
41
41 41 42 42 43 43 44 44 45 46 46 47 48 48 48 48 49 50 50 53 55 55 56 56 58 58 59 59 63
iv
Contents
65
65 66 67 68 68 68 69 69 70 71 72 73 74 75 75 76 77 77 78 79 79 80 80 82 82 83 83 85 86
89
89 90 91 91 92 92 93 93 93 95 95 96 96 97
v
Contents
SQL Named Queries Stored Procedures Using Stored Procedures with Oracle Databases Named Queries and Item Inheritance Named Query API Text Search Queries Simulating Text Search Queries Wildcards in Queries Not Queries and Null Values Outer Joins Table Ownership Issues Unsupported Queries in the SQL Repository
107
107 108 108
111
111 112 113 113 113 114 114 114 115 115 115 116 116 117 117 118 118 118 120 121 122 122 123 123 124 125 125
vi
Contents
Cache Statistics Cache Loading Automatic Reloading Loading with the load-items Tag Loading with the query-items Tag Loading with the dump-cache Tag Cache Flushing Cache Invalidation Service Caching and Item Descriptor Inheritance
147
147 148 149 149 150 156 157 160 162 162 164 164 164 164 165 165
vii
Contents
rql Tag sql-query Tag sql Tag input-parameter-types Tag returns Tag dependencies Tag Development Operation Tags transaction Tag rollback-transaction Tag import-items Tag add-item Tag update-item Tag query-items Tag print-item Tag set-property Tag remove-item Tag remove-all-items Tag export-items Tag load-items Tag dump-caches Tag print-ddl Tag Document Type Definition for SQL Repository Definition Files Sample SQL Repository Definition Files Repository Example A: Simple One-to-One Repository Example B: One-to-One with Auxiliary Table Repository Example C: One-to-Many with an Array Repository Example D: One-to-Many with a Set Repository Example E: One-to-Many with a Map Repository Example F: One-to-Many Mapping to Other Repository Items Repository Example G: Ordered One-to-Many Repository Example H: Many-to-Many Repository Example I: Multi-Column Repository IDs Configuring the SQL Repository Component Registering a Content Repository SQL Repository Component Properties
165 165 166 166 166 166 167 167 168 168 168 169 170 171 171 172 173 173 173 174 174 175 182 183 184 185 186 188 189 190 191 193 195 195 196
207
207 208 209 209 211 211 212 213 213 214
viii
Contents
Book Example Repository Definition File Book Example SQL Table Creation Statements Adding Content to the Content Repository Accessing Items in the Content Repository Configuring a SQL Content Repository
14 Repository Loader
Repository Loader Architecture FileSystemMonitorService FileSystemMonitorScheduler LoaderManager TypeMapper and TypeMappings Content Handlers and Back Ends User Interface Elements Using the Repository Loader Repository Loader Formhandlers and Administration Pages Error Policies Repository Loader RMI Client RLClient Hints Using a Repository Loader Manifest Repository Loader Manifest File Document Type Definition Repository Loader Manifest File Tags Configuring the Repository Loader Components Configuring the FileSystemMonitorService Configuring the LoaderManager Configuring the FileSystemMonitorScheduler Configuring the TypeMapper Configuring a TypeMapping Configuring the Xml2RepositoryContentHandler Choosing your Configuration Repository Loader Example User Example Item Type Example Item Pathnames Type Mappings and Content Handlers Example TypeMapper xml2repository Schemas Running the RL Example
219
219 221 221 221 221 222 222 222 222 223 223 224 225 225 226 226 227 228 229 230 230 233 233 234 236 237 237 238 239 239
241
241 243 245 247
16 Composite Repositories
Use Example
249
249
ix
Contents
Primary and Contributing Item Descriptors Item Inheritance and Composite Repositories Transient Properties and Composite Repositories Non-Serializable Items and Composite Repositories Property Derivation Configuring a Composite Repository Property Mappings Excluding Properties Link Methods Creating Composite and Contributing Items Missing Contributing Items Configuring the Composite Repository Component Composite Repository Queries Composite Repository Caching Composite Repository Definition File Tag Reference composite-repository-template Tag header Tag item-descriptor Tag primary-item-descriptor Tag contributing-item-descriptor Tag attribute Tag property Tag primary-item-descriptor-link Tag link-via-id Tag link-via-property Tag Composite Repository Document Type Definition Sample Composite Repository Definition File
249 250 250 250 250 251 251 252 252 252 253 253 254 255 255 255 255 256 258 259 259 260 261 262 262 263 265
17 Secured Repositories
Secured Repository Features Access Rights Creating a Secured Repository Modifying the Underlying Repository Configuring the Secured Repository Adapter Component Register the Secured Repository Adapter Writing the Secured Repository Definition File Group Membership ACLs and Personae ACL Syntax Standard Access Rights ACL Examples Secured Repository Definition File Tag Reference secured-repository-template Tag item-descriptor Tag property Tag default-acl Tag descriptor-acl Tag
271
271 272 273 274 275 276 276 277 278 278 279 279 280 280 280 281 281 282
x
Contents
owner-property Tag acl-property Tag creation-base-acl Tag creation-owner-acl-template Tag creation-group-acl-template Tag Secured Repository Definition File Document Type Definition Secured Repository Example Performance Considerations Exceptions Thrown by the Secured Repository
18 LDAP Repositories
Overview: Setting Up an LDAP Repository LDAP Directory Primer Hierarchical Tree Structure LDAP Data Representation Hierarchical Entry Types Directory Schema LDAP and JNDI LDAP Sources LDAP Repository Architecture LDAP Repository Items and Repository IDs Item Descriptors and LDAP Object Classes Item Descriptor Hierarchies and Inheritance Id and ObjectClasses Properties Additional Property Tag Attributes New Item Creation Repository Views in the LDAP Repository Repository View Definition LDAP Repository View Example LDAP Repository Queries ID Matching Queries Unsupported Queries in the LDAP Repository Configuring the LDAP Repository Components /atg/adapter/ldap/LDAPRepository /atg/adapter/ldap/InitialContextPool /atg/adapter/ldap/InitialContextEnvironment /atg/adapter/ldap/LDAPItemCache /atg/adapter/ldap/LDAPItemCacheAdapter /atg/adapter/ldap/LDAPQueryCache /atg/adapter/ldap/LDAPQueryCacheAdapter LDAP Password Encryption LDAP Repository Definition Tag Reference DOCTYPE Tag ldap-adapter-template Tag header Tag view Tag item-descriptor Tag
293
294 295 295 296 296 297 298 299 299 300 300 303 304 305 306 307 307 308 309 309 310 310 311 312 313 315 316 316 317 317 318 318 318 318 319 319
xi
Contents
id-property Tag object-classes-property Tag object-class Tag property Tag option Tag attribute Tag child-property Tag new-items Tag search-root Tag Sample LDAP Repository Definition File Document Type Definition for LDAP Repository Definition Files
320 320 321 322 324 324 325 326 326 327 328
Index
333
xii
Contents
1 Introduction
Data access is a large part of most Internet applications. The ATG Data Anywhere Architecture provides a unified view of content and data across a business for organizations and their customers. The core of the ATG Data Anywhere Architecture is the Repository API. Through the Repository API, you can employ a single approach to accessing disparate data types, including SQL databases, LDAP directories, content management systems, and file systems. The ATG Data Anywhere Architecture offers several advantages over the standard data access methods such as Java Data Objects (JDO), Enterprise JavaBeans (EJB), and Java Database Connectivity (JDBC). Among the differences: 1. Data source independence The ATG Data Anywhere Architecture provides access to relational database management systems, LDAP directories, and file systems using the same interfaces. This insulates application developers from schema changes and also storage mechanism. Data can even move from a relational database to an LDAP directory without requiring re-coding. Java Data Objects support data source independence, but it is up to vendors to provide an LDAP implementation. Fewer lines of Java code Less code leads to faster time-to-market and reduced maintenance cost. Persistent data types created using ATG Data Anywhere are described in an XML file, with no Java code required. Unified view of all customer interactions A unified view of customer data (gathered using web applications, call center applications, and ERP systems) can be provided without copying data into a central data source. This unified view of customer data leads to a coherent and consistent customer experience. Maximum performance - Our intelligent caching of data objects ensures excellent performance and timely, accurate results. The JDO and EJB standards rely on a vendor implementation of caching which may or may not be available. Simplified transactional control The key to overall system performance is minimizing the impact of transactions while maintaining the integrity of your data. In addition to full Java Transaction API (JTA) support, ATG Data Anywhere allows both page developers and software engineers to control the scope of transactions using the same transactional modes (required, supports, never, etc.) used by EJB deployment engineers. Fine-grained access control You can control who has access to which data at the data type, data object, even down to the individual property using Access Control Lists (ACLs). Integration with ATG product suites - Our personalization, scenarios, commerce, portal, and publishing applications all make use of repositories for data access. A
2.
3.
4.
5.
6.
7.
1
1 - Introduction
development team is free to use EJBs along side of ATG technology, but the easiest way to leverage investment in ATG technology is to follow the example set by our solution sets. The ATG solution sets satisfy all of their data access needs using repositories. With the ATG Data Anywhere, the application logic created by developers uses the same approach to interact with data regardless of the source of that data. One of the most powerful aspects of this architecture is that the source of the data is hidden behind the Dynamo Repository abstraction. It would be easy to change from a relational data source to an LDAP directory since none of the application logic would need to change. Once data is retrieved from a data source it is transformed into an object-oriented representation. Manipulation of the data can then be done using simple getPropertyValue and setPropertyValue methods. The Repository API ties in closely with ATGs targeting APIs, so you can retrieve items from the repository based on a variety of targeting rules, as well as retrieving specific identified items. The figure below provides a high-level overview of the ATG Data Anywhere Architecture.
This guide includes the following chapters: Repository API Repository Queries SQL Repository Overview SQL Repository Architecture SQL Repository Data Models SQL Repository Item Properties
2
1 - Introduction
SQL Repository Queries Localizing SQL Repository Definitions SQL Repository Caching Development, Testing and Debugging with the SQL Repository SQL Repository Reference SQL Content Repositories Repository Loader Repository Web Services Composite Repositories Secured Repositories LDAP Repositories
3
1 - Introduction
4
1 - Introduction
2 Repository API
The Dynamo Repository API (atg.repository.*) is the foundation of persistent object storage, user profiling, and content targeting in Dynamo. A repository is a data access layer that defines a generic representation of a data store. Application developers access data using this generic representation by using only interfaces such as Repository and RepositoryItem. Repositories access the underlying data storage device through a connector, which translates the request into whatever calls are needed to access that particular data store. Connectors for relational databases and LDAP directories are provided out-ofthe-box. Connectors use an open, published interface, so additional custom connectors can be added if necessary. Developers use repositories to create, query, modify, and remove repository items. A repository item is like a JavaBean, but its properties are determined dynamically at runtime. From the developers perspective, the available properties in a particular repository item depend on the type of item they are working with. One item might represent the user profile (name, address, phone number), while another may represent the meta-data associated with a news article (author, keywords, synopsis). The purpose of the Repository interface system is to provide a unified perspective for data access. For example, developers can use targeting rules with the same syntax to find people or content. Applications that use only the Repository interfaces to access data can interface to any number of backend data stores solely through configuration. Developers do not need to write a single interface or Java class to add a new persistent data type to an application Each repository connects to a single data store, but multiple repositories can coexist within Dynamo, with various applications and subsystems using different repositories or sharing the same repository. Because of this approach, applications that use only the Repository API to access data can interface to any number of back-end data stores solely through configuration. For example, the security system can be directed to maintain its list of usernames and passwords in a SQL database by pointing the security system at a SQL repository. Later, the security system can be changed to use an LDAP directory by reconfiguring it to point to an LDAP repository. Which repositories you use depends on the data access needs of your application, including the possible requirement to access data in a legacy data store. ATG 6 includes the following models for repositories: SQL Repositories, which use ATGs Generic SQL Adapter (GSA) connector to perform a mapping between ATG and data stored in a SQL database. You can use a SQL repository to access content, user profiles, application security information, and more. SQL Profile Repository, included in the ATG Personalization module, which uses the Generic SQL Adapter connector to perform a mapping for user data contained in a SQL database. See the ATG 6 Personalization Programming Guide.
5
2 - Repository API
LDAP Repository, which uses the Dynamo LDAP connector to access user data in an LDAP directory. See the LDAP Repositories chapter. Composite Repository, which provides a means for using more than one data store as the source for a single repository. See the Composite Repositories chapter. Versioned Repositories, an extension of the SQL Repository used in ATG Publishing. See the ATG 6 Publishing Administration and Development Guide.
When you store content in a repository, in addition to the meta-information about a document, you need access to the physical piece of content that is the document and path information that tells you where the document is stored. To handle this problem, we have developed the content-specific repository extensions located in the atg.repository.content package, which is described in the ATG 6 API Reference and in the SQL Content Repositories chapter in this ATG Repository Guide.
Repository Architecture
A data store may contain many diverse types of objects. The repository is not the data store itself; instead, it is a collection of JavaBeans whose properties can be found and stored in the data store. The mission of the repository is to provide a mechanism to retrieve the data elements and a run-time representation of the available meta information about each object. This goal is achieved through three main conceptual parts of the Repository API: Repository Items Item Descriptors Repository Queries
For example, a repository might track elements of an organization. Each employee would have a corresponding repository item, as would each department. An employee item descriptor would specify all of the properties that an employee repository item could possess; a department item descriptor would specify all the possible properties of a department. An application can build queries that return the appropriate employee or department repository items as they are needed by the application.
Repository Items
A repository is a collection of repository items. In general, a repository item (a JavaBean component implementing atg.repository.RepositoryItem or one of its sub-interfaces) corresponds to the smallest uniquely identifiable entity in the underlying data store. In the SQL repository, for example, a repository item often corresponds roughly to a row in a table. In the SQL profile repository, each user profile is a repository item. Each repository item is made of properties. These properties store the data that makes up a repository item. Each property has a name, such as id, firstName, or lastName. In the SQL repository, these properties correspond roughly to columns of a table. The properties available to a type of repository item are defined in the repositorys item descriptors.
6
2 - Repository API
Each repository item must have an identifier, which is called a repository ID. The repository ID must uniquely identify the repository item from all other repository items of the same type. The repository will typically be configured to find the repository ID from some elements of the underlying data. In the SQL repository, for instance, each item descriptor must specify the columns that act as the repository ID (which will usually be the same as the tables primary key). Depending on the repositorys configuration, the repository ID may or may not be exposed as a property of the repository item. Properties of repository items may be single-valued or multi-valued. In some repository implementations, such as the SQL repository, a propertys value may refer to one or more other repository items. This enables a repository item to use properties that are complex data structures. It also lets a repository items property refer to items in other repositories. The combination of item descriptors, properties, identifiers, and items allows a repository to read application data from the underlying data store, and to write application data back to the data source. Some repositories allow certain properties or even entire item descriptors or repositories to be exposed as read-only. Sometimes properties can even act as translators between the underlying data source and the Java application. For example, you might have a database column that stores first names of users. Your Dynamo application, however, wants to use first names that are exclusively uppercase. You can define a repository property named firstNameUpperCase that takes the first name value from the database and returns it as an uppercased version. The Repository API allows for this kind of flexibility without modifying any application code.
Item Descriptors
Each repository item belongs to one of several item types. An item type describes all the properties that are common to the repository items of that type. The Repository API allows you to perform a task similar to JavaBean introspection to find out information about the properties of a repository item. Each item type is described by a Repository item descriptor (also called a RepositoryView). An item descriptor implements the atg.repository.RepositoryItemDescriptor interface and may subclass atg.repository.ItemDescriptorImpl. The item descriptor gives a name to the type, and also describes the properties for that type. The name of each property is defined in the item descriptor, as well as the class of the Java object used to represent that type (Integer, String, etc.). The item descriptors exposed by a repository depend on a combination of the underlying data store and the configuration of the repository. In the SQL repository, for example, each database table might have its own repository item descriptor. Another alternative might join multiple tables into a single item descriptor. Repositories can support multiple item descriptors. For example, a SQL repository instance that supports a commerce application might have different item descriptors representing users, products, product categories, orders, etc. Note the use of three closely related terms: item type, item descriptor, and Repository View. An item type is a collection of repository items that share a common set of properties. For example, a bookstore application might use item types for customers, books, and authors. Each item type is defined by an item descriptor. There is a one-to-one correspondence between item descriptors and Repository Views. The figure below shows an example of two item descriptors used by a repository that stores customer information.
7
2 - Repository API
Inside each repository, there can be several types of items defined by item descriptors and for each item type there can be several repository items. The definition of each type of item is described in a repository definition file using XML. In this example, the Visitor Profile Repository defines two types of items (user and address). Developers can model relationships between types of items as shown in the following figure. In this example, the user item type has a property named address. The value of the address property is a repository item of another item type named address.
item-descriptor user
Repository Item Joe (id 101) Repository Item Sue (id 102)
A property in one repository item can be a linked to another type of repository item which allows developers to map relationships (one-to -one, one-to -many, etc.)
The ItemDescriptor mechanism is built upon Dynamos Dynamic Beans (atg.beans.*) system (described in the Nucleus: Organizing JavaBean Components chapter of the ATG 6 Dynamo Programming Guide), which allows you to describe properties for Java objects without defining the getX and setX methods for each property required by the JavaBean specification. This interface is used to describe a set of dynamic properties that occur together and have consistent behavior and semantics. An item descriptor essentially acts like a BeanInfo where one can get access to the PropertyDescriptors that compose the repository item. (For information about BeanInfos and PropertyDescriptors, see the JSDK 2 API documentation for java.beans.BeanInfo and java.beans.PropertyDescriptor.) Most repositories support simple property types such as Strings and Integers. Dynamo repositories can also use the Java Collections Framework to model complex relationships between items using familiar object-oriented concepts. You can store a list of addresses as a Set, List, Map, or array, whichever make sense for your applications needs. Some repositories support complex property types, in which the value of the property is itself another repository item or collection of repository items. For example, a repository might have item descriptors for types Person and Address. The Person item descriptor might have an addresses property that exposes the list of Addresses that a Person might have. This property might be of type RepositoryItem[], and the repository items in that array will use the Address item descriptor. This allows repositories to represent one-to-one, one-to-many, or many-to-many relationships.
8
2 - Repository API
The information stored in ItemDescriptor components is usually not needed for development of your Dynamo application. This property metadata is available for applications that need to expose a user interface that allows one to explore and navigate the state of a repository. The ATG Control Center uses the ItemDescriptors of the repository to automatically constrain the user interface with appropriate selections.
MutableRepository
The base interfaces of a repository define an immutable data store. It provides a read-only version of the elements contained in the repository. Extensions of the Repository interfaces provide facilities to create, update, and remove items from a repository. See atg.repository.MutableRepository. The design goal for updates was to allow transactional integrity across a set of changes in a high performance manner. Thus when an item needs to be updated, a clone of the object is returned and changes are made to the cloned object. Any modifications to this object are not observed by the repository until the object is submitted for an update action. Generally, implementations of repositories use caches to improve performance. Items retrieved out of a repository either through a query process or directly by ID from the repository are cached. Typically, cache policies are based on least recently used (LRU) design patterns or time-based expiration. See, for example, the SQL Repository Caching chapter.
atg.repository.Repository
The atg.repository.Repository interface is the base definition of any repository implementation. This interface provides methods to access RepositoryItems, RepositoryViews and ItemDescriptors, corresponding to the three main elements of the repository. Given a unique ID or set of IDs, you can retrieve items from the repository using the following methods:
9
2 - Repository API
RepositoryItem getItem(String pId, String pDescriptorName) RepositoryItem[] getItems(String[] pIds, String pDescriptorName)
Depending on the repository implementation, item IDs may take different forms. In SQL repositories, a repository item ID by default is numeric and is auto-generated through the IdGenerator service. See ID Generators in the Core Dynamo Services chapter of the ATG 6 Dynamo Programming Guide. The SQL repository also supports composite repository item IDs. In that case, you can retrieve items from the repository using these methods:
RepositoryItem getItem(CompositeKey pId, String pDescriptorName) RepositoryItem [] getItems(CompositeKey [] pIds, String pDescriptorName)
In other cases, an item ID might be the path of the document, as in some of the file-system based repositories. The Repository API includes the RepositoryItemDescriptor interface, a subinterface of atg.beans.DynamicBeanInfo (see Dynamic Beans: DynamicBeanInfo in the Nucleus: Organizing JavaBean Components chapter of the ATG 6 Dynamo Programming Guide). This lets you access the dynamic bean info of the available repository items, such as the property descriptors and property names, using this method:
RepositoryItemDescriptor getItemDescriptor(String pName)
You can get the list of all the ItemDescriptors that are available from the itemDescriptorNames property.
atg.repository.RepositoryView
If you do not have an exact repository ID, you can search for items in the repository through a RepositoryView. Item descriptors and RepositoryViews often have a one-to-one relationship and often have the same name. You can find out what views are available through the viewNames property of the Repository component. This is useful if you need to build a system to navigate and view the entire contents of a repository. The IDs for items in different item types may or may not overlap. There may be no view that can return all items in the repository, but if there is, it will be the default view. If you need to use the default view, you can use the view named by the defaultViewName property. Alternatively, you can create properties for your own services that allow you to explicitly name the view your code is interested in using. Once you have a name, you can retrieve that view through the RepositoryView getView(String pName) method. From this returned object you can then build and execute a query. The RepositoryView can also implement atg.repository.RepositoryViewContainer if the repository needs to express a hierarchy of RepositoryViews. For example, a document management system may have a root view for all documents. From that you may have sub-document types like white papers, promo blurbs, images, etc. Further, the sub-view images may also have a refinement for specific image types like JPEG and GIF. You can see what attributes are available for building queries by accessing the itemDescriptor property of the RepositoryView. This describes all the property information about all the items that can be returned by this view. Sets of repository items can be gathered by queries, which you can build using the atg.repository.QueryBuilder interface. This QueryBuilder object can be retrieved from the views queryBuilder property. Once a query is built, each RepositoryView implementation translates the
10
2 - Repository API
internalized data structure into its native query language, for example SQL, and then return an array of repository items that match the supplied query criteria. This is the mechanism by which the targeting engine understands how to translate the rule syntax into the appropriate method calls of the QueryBuilder. Once a query is built from the QueryBuilder, it is executed via the various executeQuery methods defined in the RepositoryView interface, such as:
RepositoryItem[] executeQuery(Query pQuery)
These overloaded executeQuery methods allow range queries and sorting criteria to be added to the execution of the query. The methods return either: an array of RepositoryItems, for elements that match the where clauses of the query, or null, if no elements could be found.
atg.repository.RepositoryItem
The atg.repository.RepositoryItem interface is the immutable interface that represents an element from a repository. Each RepositoryItem is uniquely identified through its repositoryId property. The ItemDescriptor that describes the dynamic bean info about the item is available through the itemDescriptor property. Given the repository item, you can also know what repository it came from with the repository property. To retrieve the attributes of the RepositoryItem, use the getPropertyValue(String pPropertyName) method. You can retrieve subproperties using the DynamicBeans.getSubPropertyValue method, which takes a hierarchy property name of the form propertyName1.subPropertyName2.subSubPropertyName3. A Dynamic Bean property mapper has been registered for the RepositoryItem interface. This allows you to reference the names of these properties as though they were JavaBean properties in the Dynamo Servlet Bean syntax. See the Dynamic Beans section of the Nucleus: Organizing JavaBean Components chapter of the ATG 6 Dynamo Programming Guide.
atg.repository.MutableRepository
Some repository services implement MutableRepository, a subclass of Repository. The SQL repository implements this interface. The MutableRepository interface defines functions for four operations: creating, adding, updating, and removing repository items.
createItem
There are two createItem methods:
createItem(String pDescriptorName) createItem(String pId, String pDescriptorName).
Each of these requires a DescriptorName parameter, which should be the name of the RepositoryView or ItemDescriptor that describes the repository item you wish to create. Each repository has a default ItemDescriptor, which may allow your code to use the defaultViewName property of the repository to
11
2 - Repository API
supply this value. One of the createItem methods takes a potential unique ID to use for the MutableRepositoryItem to create. If you do not supply an ID, one will be automatically generated and guaranteed to be unique. In the SQL profile repository, for example, the createItem methods return a transient instance of a MutableRepositoryItem. At this point, the profile does not exist persistently in a data store. The item exists only as the object reference you are returned. You may attempt to re-fetch the object (if the users session has not expired or the server has not been restarted) through the getItem(String pId, String pDescriptorName) method of the Repository (unless the GSARepository.storeTransientItems property is set to false). Maintaining profile RepositoryItems in RAM rather than in the profile database allows anonymous users to be represented in the same Repository API, but does not hamper performance for handling requests for large sites. It becomes untenable to attempt to create anonymous user database records for web sites that have a large volume of users.
addItem
Once an item is created, at some later point you might want to turn it into a persistent repository item. To do so, use the addItem method. This takes in the repository item that you want to add persistently:
RepositoryItem addItem(MutableRepositoryItem pItem)
removeItem
Removing an item from the repository is very easy. Pass the ID and ItemDescriptor name of the item you want to remove persistently to the removeItem method. The items property values will be deleted and will no longer be accessible from the repository:
removeItem(String pId, String pDescriptorName)
updateItem
The MutableRepository updates a repository item in a transactionally aware manner. It does not occur like standard JavaBeans because we want to make sure the update operation in the backend data store (such as a relational database) is efficient. Thus, updating an item takes three steps: 1. Fetch a mutable version of the repository item through the getItemForUpdate and getItemsForUpdate methods. These methods return instances of MutableRepositoryItem. This interface extends RepositoryItem and adds one method:
setPropertyValue(String pPropertyName, Object pPropertyValue)
2.
Use the setPropertyValue method of MutableRepositoryItem to change as many properties as you wish. These changes will not be reflected in the repository until the final updateItem operation is invoked. Save the changes with the updateItem method. This method extracts all the changes required for the item and updates the item in the data store. Depending on how you have configured transactional behavior, the update can be committed immediately, or the update may happen automatically when the associated transaction commits. See
3.
12
2 - Repository API
Repositories and Transactions in the SQL Repository Architecture chapter. If there was any type of error, a RepositoryException is thrown. For example:
try { RepositoryItem user = ... // get a reference to the user you want to update MutableRepository mutableRepository = (MutableRepository)user.getRepository(); MutableRepositoryItem mutableUser = mutableRepository.getItemForUpdate(user.getRepositoryId(), user.getItemDescriptor().getItemDescriptorName()); mutableUser.setPropertyValue("name", "bob"); mutableUser.setPropertyValue("age", new Integer(26)); mutableRepository.updateItem(mutableUser); } catch (RepositoryException exc) { // deal with exception }
This same methodology should be applied for RAM-based RepositoryItems that you have created through the createItem method. No database transaction will be performed, but the values will be updated in the repository. Dynamo includes three classes that provide useful methods for dealing with repository items: atg.repository.servlet.RepositoryFormHandler atg.userprofiling.ProfileForm atg.userprofiling.ProfileFormHandler See the Using Profiles and Profile Forms chapter in the ATG 6 Page Developers Guide and the source code for the ProfileForm and ProfileFormHandler classes, included in the ATG Personalization module distribution in the <ATG6dir>/Dps/src/Java/atg/userprofiling directory.
atg.repository.PropertiesChangedEvent
When a repository item is modified, its item descriptor broadcasts locally a PropertiesChangedEvent. This event can be one of the following types:
Description the item has been removed in this transaction properties of an item have been changed in this transaction the item was newly added to the database some application code called the removeItemFromCache method
13
2 - Repository API
Properties
item
Description The item that is changed. This is set to null if the item that was modified is not currently in the cache. In that case, look at the repositoryId property for the identity of the item that was changed. The repository ID of the item that is changed. The item descriptor of the item that is changed. A Map in which the keys are RepositoryPropertyDescriptors and the values are the new property values. If all properties have changed (or may have changed), a null value is returned for the properties map. Returned only for UPDATE events.
If you have a component that you want to be notified when repository item properties change, it can implement the atg.repository.PropertiesChangedListener interface. You can add your PropertiesChangedListener implementation to the atg.repository.ItemDescriptorImpl returned by the repository.getItemDescriptor() method, using the method ItemDescriptorImpl.addPropertiesChangedListener.
public static MutableRepositoryItem cloneItem(RepositoryItem pItem, boolean pDeepCopy, Map pPropExceptions, Map pExcludedProperties, MutableRepository pDestRepository, String pId) throws RepositoryException, DuplicateIdException
14
2 - Repository API
Description Item to copy. The mode of the copy. If true, the method creates a deep copy of the item and its properties. Otherwise, the method creates a shallow copy, only getting references of child RepositoryItems. Note that shallow copying will only work if the source and destination repositories are the same. Hierarchical map of property name exceptions to the above mode. Keys are property names, while values are either null or, if the property is another repository item, another Map. For example, if you clone a product item using pDeepCopy=true, you could add the key parentCategory with a null value into pPropExceptions. This would result in a shallow copy of the product.parentCategory. Alternatively, you could add the key parentCategory but set the value to another map of exceptions that included the key/value pair keywords=null. This would result in a deep copy of product.parentCategory but a shallow copy of the product.parentCategory.keywords. Optional. Properties to exclude from the clone. Keys are item descriptor names and the values are collections of property names to exclude. Optional. Repository to copy the new item into. If the source and destination repositories are the same, properties that are items will be cloned to the repository of the source item-property. Optional; if null, the new item is copied to source repository. Repository ID to use in the copy of the item. Optional; if null, a unique ID is automatically generated.
Parameter
pItem pDeepCopy
pPropExceptions
pExcludedProperties
pDestRepository
pId
15
2 - Repository API
16
2 - Repository API
3 Repository Queries
All repositories have the ability to execute queries. A query is a request to find all of the items of a particular item type that fits a certain set of criteria. Those criteria are specified in terms of the item types properties. For example: Find all Person items whose age property is greater than 30 The Repository API can express a wide variety of queries, including queries that match patterns in text, query through collections, or even query through complex values. Queries can also specify the order in which results should be returned, and can specify which results from a large result set should be returned. For example, a query can express something like: Find all Person items whose lastName starts with A, whose interests includes biking, and whose addresses property contains an Address with a zipCode of 02139. Order the results by lastName, and return only items 10-20. Queries can be built and executed using the Repository API. For complex queries, this can be a tedious task. In these cases, or in cases where the Repository API should not be used directly, queries can be represented in a textual form called Repository Query Language (RQL). See the Repository Query Language section of this chapter for details of this language and other information about repository queries. In most cases, however, repository queries can be constructed easily using targeting UI components in the ATG Control Center.
atg.repository.QueryBuilder
The atg.repository.QueryBuilder interface defines the available query operations that repositories should support. The QueryBuilder interface enables you to build Query objects that can be passed to the repository. A Query is constructed from QueryExpressions. Each Query relates one or more QueryExpressions and a query operation. There are standard logical query operations, such as AND, OR, NOT, EQUALS, GREATER THAN, LESS THAN OR EQUALS, plus more complicated query operations like collection inclusion, pattern matching, and others. The implementation of QueryBuilder does not have
17
3 - Repository Queries
Query Creation Example
1.
to support all query operations it depends on what query features the data store supports. For unsupported query operations, the method should throw a RepositoryException. You can use the atg.repository.QueryOptions class to limit the size of a querys result set or otherwise modify the query.
The following example creates a query that returns all repository items whose gender property is female: Given a RepositoryView, we initialize a QueryBuilder for it:
QueryBuilder b = view.getQueryBuilder();
2.
Next, we create a QueryExpression for the gender property and a QueryExpression for the constant female:
QueryExpression gender = b.createPropertyQueryExpression("gender"); QueryExpression female = b.createConstantQueryExpression("female");
3.
4.
atg.repository.QueryOptions
You can use the atg.repository.QueryOptions class to specify ways that a query may be modified. You can set the QueryOptions properties, and then pass the QueryOptions bean to the following executeQuery method:
RepositoryItem[] executeQuery(Query pQuery, QueryOptions pQueryOptions);
The QueryOptions properties let you limit the size of the result set, direct how the result set should be sorted, and pre-cache specified properties:
Property Name
startingIndex
Description The index of the first element of a query result set that should actually be returned. By setting startingIndex and endingIndex, you can limit the size of the querys result set. The items beginning with the endingIndex element of the query result set are not returned. In other words, the total number of items returned is endingIndex - startIndex. A value of -1 indicates that there is no limit to the number of items returned. Specifies the sort order of a querys result set.
endingIndex
sortDirectives
18
3 - Repository Queries
A list of properties that should be pre-cached for each item in a querys result set at the time the query is run.
precachedPropertyNames
In the following example, we: 1. 2. Create an unconstrained query against the Profile Repository. Create a SortDirectives that contains a SortDirective, sorting the result set in ascending order by the login property. A SortDirective may be ascending or descending and case-sensitive or case-insensitive (although not all data stores will support case-insensitive sorting). Create a QueryOptions with a startingIndex of 0 and an endingIndex of 5. This limits the number of profiles returned to at most 5. The QueryOptions incorporates the SortDirectives. Execute the query, using the QueryOptions. Output the results, displaying the login property.
3.
4. 5.
// Creates an unconstrained query against the profile repository. // It returns at most the first 5 profiles. // Hints are used to precache login and password properties. // Results are sorted by the login property. Repository repository = (Repository) request.resolveName("/atg/userprofiling/ProfileAdapterRepository"); RepositoryView view = repository.getView("user"); Query query = view.getQueryBuilder().createUnconstrainedQuery(); String [] precachedPropertyNames = {"login", "password"}; SortDirectives sortDirectives = new SortDirectives(); sortDirectives.addDirective(new SortDirective("login", SortDirective.DIR_ASCENDING)); RepositoryItem [] items = view.executeQuery(query, new QueryOptions(0, 5, sortDirectives, precachedPropertyNames)); for (int i = 0; i < items.length; i++) out.print("<li>" + items[i].getPropertyValue("login"));
19
3 - Repository Queries
import atg.repository.*; MutableRepository pRepository =
The following example supposes we have an item descriptor named user with an integer property named userType. This is how we might perform a simple query to find users whose userType property is 2.
(MutableRepository)ServletUtil.getCurrentRequest().resolveName ("/atg/userprofiling/ProfileAdapterRepository"); // Queries are created using QueryBuilders and executed by // RepositoryViews. A Query is defined in the context of a // specific item descriptor and thus must be built and executed with // the right QueryBuilder and RepositoryView. RepositoryItemDescriptor userDesc = pRepository.getItemDescriptor("user"); RepositoryView userView = userDesc.getRepositoryView(); QueryBuilder userBuilder = userView.getQueryBuilder(); // create a QueryExpression that represents the property userType QueryExpression userType = userBuilder.createPropertyQueryExpression("userType"); // create a QueryExpression that represents the constant 2 QueryExpression two = userBuilder.createConstantQueryExpression(new Integer(2)); // now we build our query: userType = 2 Query userTypeIsTwo = userBuilder.createComparisonQuery(userType, two, QueryBuilder.EQUALS); // finally, execute the query and get the results RepositoryItem[] answer = userView.executeQuery(userTypeIsTwo); System.out.println("running query: userType = 2"); if (answer == null) { System.out.println("no items were found"); } else { for (int i=0; i<answer.length; i++) System.out.println("id: " + answer[i].getRepositoryId()); }
Lets expand the preceding example with a slightly more complicated query. The next code fragment builds on the preceding fragment to create the query userType < 2 AND login STARTS WITH "j":
20
3 - Repository Queries
import atg.repository.*; MutableRepository pRepository = (MutableRepository)ServletUtil.getCurrentRequest().resolveName ("/atg/userprofiling/ProfileAdapterRepository"); // reuse the building blocks we have to create // the "userType < 2" query Query userTypeLTTwo = userBuilder.createComparisonQuery(userType, two, QueryBuilder.LESS_THAN); // create the "login STARTS WITH j" query QueryExpression login = userBuilder.createPropertyQueryExpression("login"); QueryExpression j = userBuilder.createConstantQueryExpression("j"); //Note that we could make this query case-insensitive by adding another //parameter to the createPatternMatchQuery, with a value of true Query startsWithJ = userBuilder.createPatternMatchQuery(login, j, QueryBuilder.STARTS_WITH);
// now AND the two pieces together. You can AND together as many // Query pieces as you like: we only have two in our example Query[] pieces = { userTypeLTTwo, startsWithJ }; Query andQuery = userBuilder.createAndQuery(pieces); // execute the query and get the results answer = userView.executeQuery(andQuery); System.out.println("running query: userType < 2 AND login STARTS WITH j"); if (answer == null) { System.out.println("no items were found"); } else { for (int i=0; i<answer.length; i++) { RepositoryItem item = answer[i]; String id = item.getRepositoryId(); String l = (String)item.getPropertyValue("login"); Integer a = (Integer)item.getPropertyValue("userType"); System.out.println("item: " + id + ", login=" + l + ", userType=" + a); } }
21
3 - Repository Queries
Note that the ATG Control Center limits the number of items that can be returned by such queries. This limit is configurable and is set in the maxQueryCount property of /atg/devtools/RepositoryAgent. The default value is 1000.
22
3 - Repository Queries
You can define an RQL filter that is implicitly applied to all queries performed by the repository. See Repository Filtering in the SQL Repository Queries chapter. You can include RQL queries in <query-items> tags in the XML repository definition file. This is useful principally for unit testing queries, but can also be used to pre-load repository caches. See Querying Items in the Development, Testing and Debugging with the SQL Repository chapter and Cache Loading in the SQL Repository Caching chapter. You can use RQL directly by creating an atg.repository.rql.RqlStatement object. You can get an RqlQuery object from the RqlStatement object, and in turn get an atg.repository.Query object from the RqlQuery object. This approach can be simpler than using the QueryBuilder class to create the Query object.
RQL Overview
RQL is a textual query syntax, similar to SQL. It describes the set of conditions that must be matched by items of a particular item descriptor. The following is a simple RQL query that matches all items whose age property is greater than 30.
age > 30
Notice that the name of the item descriptor is not included in the RQL query. The item descriptor is usually implied by the context of the querys use. All of the standard comparison operators can be used, as well as logical operators like AND, OR, and NOT:
age > 30 AND (lastName = "jones" OR paymentOverdue = true)
Constants such as 30, true, or jones can represent numbers, boolean, or String values. String values are represented using Java syntax. They must be enclosed by quotes, and escape sequences for special characters or UNICODE characters must use Java escape syntax. Note that properties such as age or lastName must be the property names used in the repository. For example, a database might have a column named phone, but that might be mapped to the repository property primaryPhoneNumber. An RQL query must use the repositorys property name primaryPhoneNumber, not the column name phone. Also note that all RQL keywords (AND, OR, NOT, etc.) may be specified in either all upper-case or all lowercase. For example:
age > 30 and (lastName = "jones" or paymentOverdue = true)
RQL statements specify the conditions that an item must meet in order to be included in the result set. In addition, an RQL statement may specify other directives to be applied to the result set, including ordering of the results, or returning just a portion of the result set.
23
3 - Repository Queries
Comparison Queries
age > 30
These are the simplest RQL queries, in which a propertys value is compared against another property value, or against a constant. For example:
All of the standard comparison operators can be used: =, !=, <, <=, >, >=. These operators can even be applied to String properties and arguments, in which case ordering is determined by lexical order of the Strings. In general, these operators can only be used on properties that are scalar values. They shouldnt be used on properties that are array or collection values.
When you use text comparison queries, be sure to enclose the comparison value in double quotes, as in the examples above; otherwise, the RQL parser assumes the comparison term refers to a property name rather than a property value. The optional IGNORECASE directive can be applied to any of these queries to perform a case-insensitive comparison:
firstName STARTS WITH IGNORECASE "h" lastName ENDS WITH IGNORECASE "son" phoneNumber CONTAINS IGNORECASE "33" state EQUALS "utah"
Be aware, though, that negated pattern match queries have the potential to cause performance problems. You should consider the queries you want to use and plan your database indexes accordingly to avoid table scans. STARTS WITH and EQUALS queries can be optimized easily using database indexes. The other pattern match queries generally cannot be. Likewise case-insensitive pattern matching may affect the ability of the database to optimize the query.
24
3 - Repository Queries
another (or the same) item descriptor. For example, the address property might point to another item descriptor which itself has properties like city, state, and zip. Queries may drill down through these properties by using a dot notation. For example:
address.zip = "48322"
This query means find all people whose address property points to an Address item whose zip code is 48322. RQL allows for multiple levels of property-of-property expressions. For example,
department.manager.address.state.
Logical Operators
Any query expressions may be combined using the AND, OR, and NOT operators. Parentheses may be used to affect grouping. NOT has the highest precedence, AND the next highest precedence, and OR has the lowest precedence. For example, this expression:
name = "joe" OR NOT phone ENDS WITH "7" AND age > 30
Collection Queries
The above comparison operators, as well as the MATCH and MATCHES operators described below in the Full Text Search Queries section, should only be applied to scalar properties. There is another set of queries that may be applied to arrays or collections of scalar values, for example, properties of type int[], or Set of Strings. Use the INCLUDES, INCLUDES ANY, or INCLUDES ALL operators only for querying multi-valued properties. The INCLUDES query matches items for whom the specified property includes the specified value. For example:
interests INCLUDES "biking"
The INCLUDES query can also match one of a set of items. To do this, the ANY or ALL keyword must be used, and the comma-separated set of items must be enclosed in braces. For example:
interests INCLUDES ANY { "biking", "swimming" }
While this:
25
3 - Repository Queries
is equivalent to:
This query means find all people whose list of addresses includes at least one address whose zip code is 48322 and whose state is MI.
Is Null Queries
Seeing whether an expression evaluates to null should be done with an IS NULL query. For example:
phoneNumber IS NULL
Count Expressions
The COUNT operator can be used to query on the size of a collection property. For example:
COUNT (addresses) > 3
This will find all people whose addresses property contains 4 or more elements.
All Query
An RQL query of ALL will return all of the items in a particular item descriptor. This should, of course, be used with care since the result set could be very large. Usually this is combined with an ORDER BY or RANGE directive (described below). The RQL query is simply:
ALL
26
3 - Repository Queries
MATCH "mars"
This will return those items whose content matches mars in a full text search. (Content repositories allow parts of the items data to be designated as content for the purposes of display and searching). Another form of the query allows the full text search to proceed over a particular property:
firstName MATCHES "abr"
Note that MATCH and MATCHES queries apply only to scalar properties. Both forms of the query allow a special USING directive to pass special instructions to the underlying search engine. The format of this directive depends on the repository and whatever search engine it is using. For example, to use the Sybase Full-Text Search Specialty Data Store, the query would look like this:
firstName MATCHES "abr" USING "SYBASE_SDS"
To use the Oracle ConText full text search engine, the query would look like this:
firstName MATCHES "abr" USING "ORACLE_CONTEXT"
ID-based Queries
RQL offers the ability to query items based on their repository IDs. This ability should be used with care, since repository IDs are not portable across repository implementations. The first query searches for items that match a set of IDs. For example:
ID IN { "0002421", "0002219", "0003244" }
The next ID-based query applies only to content repositories, in which items have been organized into folders. This query restricts the search to only those items in the specified folders. The folders must be specified by ID:
IN FOLDERS { "10224", "10923", "12332" }
Note that passing in an empty or null set of IDs results in an exception. Composite IDs can be specified in RQL queries using the following format for integers:
[value1, value2 ... valueN]
27
3 - Repository Queries
id = ["dept2", "emp345"]
Such a query would return an item with a composite repository ID of dept2:emp345. A query like this would return items with any of the IDs dept2:emp345, dept2:emp346, or dept2:emp347:
ID IN { ["dept2", "emp345"], ["dept2", "emp346"], ["dept2", "emp347"] }
Order By Directives
Once a query has been defined using the above query elements, the result is a set of items. Using the ORDER BY directive, the results may be ordered by the items properties. For example:
age > 30 ORDER BY firstName
This will return all people for whom age is greater than 30, with the results ordered by the firstName property in ascending order. The results may be ordered in descending order by adding SORT DESC to the end of the directive:
age > 30 ORDER BY firstName SORT DESC
The SORT ASC directive may also be used, but its unnecessary because its already the default. The results may be ordered by multiple properties, each in ascending or descending order. For example:
age > 30 ORDER BY lastName, firstName SORT DESC
This will order the results by lastName. If multiple results have the same lastName, then within their group they will be ordered by firstName in descending order. A further directive, CASE IGNORECASE, may be used for case-insensitive sorting:
age > 30 ORDER BY firstName SORT ASC CASE IGNORECASE
Note that you can omit the tokens SORT and CASE, unless you are using parameters for the ASC/DESC or USECASE/IGNORECASE tokens.
Range Directives
Many queries have the potential for returning large result sets. Most applications dont want to display the entire result set - they may want to display just the first 10 results. Or they may want to page through the results, showing results 0-9, then results 10-19, etc. The RANGE directive is used to specify this in the RQL query. The RANGE directive must come after the ORDER BY directive (if any). It has three forms. The first is the most common:
age > 30 RANGE +10
This causes only the first 10 results to be returned. If the result set is already less than 10, then all of the results are returned.
28
3 - Repository Queries
The next form of the RANGE directive allows the results to start at a specified index:
age > 30 RANGE 10+
This causes the first 10 results to be skipped, and the remaining results to be returned. The final form of the RANGE directive combines the above two forms, and is often used for paging:
age > 30 RANGE 40+10
This skips the first 40 results, then returns up to the next 10 results.
Parameters in Queries
In all of the above examples, the queries contain hard-coded constants, such as 30 or joe. Most of the time, the actual values used in the query will not be known at the time the RQL statement is written. In these cases, the values may be substituted with parameter expressions. For example:
age > ?0 AND firstName CONTAINS ?1 RANGE ?2+10
Every ?{number} represents a parameterized value that will be filled in when the query is executed. How those values are supplied depends on the application performing the query. In the case of entity EJBs, where RQL queries are used to represent finder methods, the parameters are filled in from the arguments of the finder methods. For example, ?0 is substituted with the value of the first argument, ?1 with the second, etc. Parameter expressions can generally be used wherever constant values are used, including in RANGE expressions. However, parameter expressions may not be used in array expressions, such as ID IN or IN FOLDERS queries. In addition, parameter expressions may not be used as substitutes for property names all property names must be hard-coded into the RQL query when it is written.
In this example, only one object is passed into the query at runtime. However, this object is expected to have two public member variables called name and age. The query will extract the values of these member variables from the object and substitute those values for the ?0.name and ?0.age parameters. Note that the fields must be public member variables of the object that is passed in, not JavaBean properties. For example, the following object could be passed in to the query:
public class QuerySpecifier { public String name;
29
3 - Repository Queries
public int age; }
Parameterized Field Queries are used most often for entity EJBs, which allow primary key classes to contain multiple fields. In this case, only one object will be passed to the query (the primary key), but if the primary key spans multiple database fields, then the primary key object will contain the values of those fields in its public member variables.
RQL Examples
The following example shows how you might use a parameter expression in Java code. It creates an RqlStatement and then uses it in executing a query to find person repository items where the value of the age property is greater than 23.
RepositoryView view = repository.getView("person"); RqlStatement statement = RqlStatement.parseRqlStatement("age > ?0"); Object params[] = new Object[1]; params[0] = new Integer(23); RepositoryItem [] items =statement.executeQuery (view, params);
RqlStatement statement = RqlStatement.parseRqlStatement("lastName STARTS WITH ?0"); Object params[] ={new String("m")}; items = statement.executeQuery (view, params);
Note how in the text comparison queries the comparison value "m" is enclosed in double quotes; otherwise, the RQL parser assumes the comparison term refers to a property name rather than a property value.
RQL Grammar
The following is a formal definition of the RQL grammar:
RQLStatement:: Query OrderByClause RangeClause Query:: OR | AND | NOT | Comparison | ID IN | IN FOLDERS | ALL | TextSearch | PropertyTextSearch | INCLUDES ITEM | IS NULL | (Query)
The precedence order of the queries from highest to lowest is as follows: (Query)
30
3 - Repository Queries
Comparison, ID IN, IN FOLDERS, ALL, TextSearch, PropertyTextSearch, INCLUDES ITEM, IS NULL NOT AND OR OR:: Query OR Query ... AND:: Query AND Query ... NOT:: NOT Query Comparison:: Expression ComparisonOperator Expression ComparisonOperator:: = | != | < | <= | > | >= | INCLUDES ANY | INCLUDES ALL | INCLUDES | STARTS WITH [IGNORECASE] | ENDS WITH [IGNORECASE] | CONTAINS [IGNORECASE] IdIn:: ID IN StringArray InFolders:: IN FOLDERS StringArray All:: ALL TextSearch:: MATCH StringLiteral [USING StringLiteral] PropertyTextSearch:: ObjectExpression MATCHES StringLiteral [USING StringLiteral] IncludesItem:: Expression INCLUDES ITEM ( Query ) Expression:: CountExpression | ObjectExpression | ParameterExpression | ConstantExpression CountExpression:: COUNT ( ObjectExpression | ParameterExpression | ConstantExpression ) ObjectExpression:: PropertyName | ObjectExpression.PropertyName | ObjectExpression[Expression] PropertyName:: <Java identifier> ParameterExpression:: ?<Parameter number>[.<Field name>] ConstantExpression:: StringLiteral | IntegerLiteral | DoubleLiteral | BooleanLiteral | ArrayLiteral StringLiteral:: <Java string literal> The string literal uses the Java format, including escape characters (including octal and Unicode), and must be enclosed in double quotes. IntegerLiteral:: <Java integer literal> DoubleLiteral:: <Java double literal> BooleanLiteral:: true | false ArrayLiteral:: { ConstantExpression , ... } StringArray:: { StringLiteral , ... }
31
3 - Repository Queries
RangeClause:: RANGE <Starting Index> + <Count>
OrderByClause:: ORDER BY PropertyName [[SORT] [ ASC | DESC]] [CASE [ IGNORECASE | USECASE]] The SORT ASC/DESC directives are optional and default to SORT ASC. The CASE IGNORECASE/USECASE directives are optional and default to CASE USECASE.
32
3 - Repository Queries
The ATG Dynamo SQL repository can be used to connect ATG applications to a SQL database. A SQL database provides fast, scalable storage and retrieval of persistent information. The SQL repository works with a SQL database to store objects and make those objects visible inside an ATG application as Dynamic Beans. The uses of a SQL repository can be as varied as the uses of a relational database. ATG 6 includes SQL repositories used to store: User profiles (the Personalization modules SQL Profile Repository). See the SQL Profile Repositories chapter in the ATG 6 Personalization Programming Guide. Content to be displayed on a Web site (the SQL content repository). See this chapter and the SQL Content Repositories chapter. Security profiles used by the Administrative Security system. See the Managing Access Control chapter of the ATG 6 Dynamo Programming Guide. Information used in J2EE container managed persistence. See the Using Enterprise JavaBeans chapter in the ATG 6 J2EE Development and Deployment Guide.
In addition to these, an ATG Commerce site uses repositories for: The store catalog. See the Using and Extending the Default Catalog chapter in the ATG 6 Commerce Administration and Development Guide. In process orders. See the Purchase Process Services chapter in the ATG 6 Commerce Administration and Development Guide. Inventory. See the Inventory Framework chapter in the ATG 6 Commerce Administration and Development Guide. Gift lists and wish lists. See the Merchandising Services chapter in the ATG 6 Commerce Administration and Development Guide. Pricing and promotions. See the Pricing Services chapter in the ATG 6 Commerce Administration and Development Guide.
ATG 6 includes a component at /atg/registry/ContentRepositories that lists all SQL content repositories that have been registered with it.
33
4 - SQL Repository Overview
1.
Create the repository definition file for the SQL repository to use. This template is an XML file that defines the item descriptors and repository item attributes contained in your SQL repository and describes the relationship of your SQL repository to the SQL database. While the SQL repository is flexible enough to represent a variety of different data models, it cannot easily represent any arbitrary data model. Therefore, it is usually a good idea to design the SQL repository schema before you design your SQL database schema. The SQL Repository Data Models and SQL Repository Item Properties chapters describe how to design your item descriptors and other SQL repository elements. See also the SQL Repository Definition Tag Reference for full details of the XML tags used to create the SQL repository definition file.
2.
Configure a SQL Repository component. The SQL Repository components class is atg.adapter.gsa.GSARepository, which implements atg.repository.MutableRepository and atg.repository.content.ContentRepository and which extends atg.repository.RepositoryImpl. The definitionFiles property of the SQL Repository component points to the repository definition file for the repository. See the Configuring the SQL Repository Component chapter. Create the SQL database schema on your SQL database server. You can use the
startSQLRepository script with the outputSQL option to generate a preliminary
3.
form of the SQL needed to create the database schema, and then edit the output to optimize the database schema. See startSQLRepository Script and the Template Parser in the Development, Testing and Debugging with the SQL Repository chapter.
34
4 - SQL Repository Overview
The SQL repository is a generalized and flexible implementation of the Dynamo Repository API that an application can use to access data stored in a SQL database. See the Repository API chapter for more information. The SQL repository is implemented through the atg.adapter.gsa package. (GSA stands for Generic SQL Adapter.) The main Dynamo component in the SQL repository is an instance of the atg.adapter.gsa.GSARepository class, which implements the interfaces atg.repository.MutableRepository and atg.repository.content.ContentRepository and which extends the class atg.repository.RepositoryImpl. Whenever you need to create a new SQL repository instance, you should instantiate the atg.adapter.gsa.GSARepository class. The ATG 6 API Reference does not include Javadoc for this class and it is not intended that you access this class directly from Java code. Normally, you should access all Repository functionality using the interfaces atg.repository.Repository and atg.repository.MutableRepository. This will enable your classes to work with any repository implementation for the greatest flexibility. Certain methods like those for cache invalidation are defined on the class atg.repository.RepositoryImpl. It is anticipated that future repositories will extend that class and so again you can make your code more reusable and maintainable by accessing those methods on this base class rather than the implementation class of atg.adapter.gsa.GSARepository. The SQL repository uses an XML file called a repository definition file to describe the item descriptors that make up a repository. The repository definition file also describes the relationships between the repositorys item descriptors, repository items, and repository item properties, on the one hand, and the SQL databases tables, rows, and columns. The XML tags that make up the repository definition file are described in detail in the SQL Repository Reference chapter. The XML tags are also introduced in the examples in this chapter and in the SQL Repository Item Properties chapter.
35
5 - SQL Repository Architecture
1. 2. 3. Begin JTA transaction. Call setPropertyValue. Using the updateItem method: 1. 2. 3. 4. Begin JTA transaction. Call setPropertyValue. Call updateItem. At this point, SQL is issued. Commit JTA transaction. Changes are committed.
Commit JTA transaction. At this point, SQL is issued and the changes are committed.
Generally, you will want to call updateItem explicitly. This ensures that if you perform any queries between the change made in the setPropertyValue call and the commitment of the transaction, those queries will have the new property value available to them. You can configure ATG 6 to send repository item cache invalidation messages to other remote ATG servers. If you set the cache mode to distributed mode for an item descriptor, then cache invalidation message will be set. Also, if you set the cache mode to locked mode for an item descriptor, then when a server gives up ownership of the lock, it also invalidates the cache. See the SQL Repository Caching chapter. The SQL repository implements transaction isolation. The first time the item is accessed in a particular transaction, either through the getItem() call, or the first attempt to call getPropertyValue() in an item which was retrieved in a different transaction, we guarantee that it is up to date at that time. If the item is changed by another transaction while this transaction is in progress, we dont see those changes until a new transaction is started. By default, a transaction will be created and committed for each method call. This is generally not the most efficient way to handle repository item updates. It is generally most efficient to ensure that all of the method calls in creating or updating a repository item are performed in a single transaction. ATG 6 offers several different techniques for transaction demarcation that you can use to group repository method calls into a single transaction. These are described in detail in the Transaction Management chapter in the ATG 6 Dynamo Programming Guide. One option is to use the Transaction Dynamo servlet bean to explicitly create a transaction in a Dynamo Server Page. This servlet bean is described in Appendix B: Servlet Beans in Dynamo Application Server in the ATG 6 Page Developers Guide. For example, the following uses the current transaction, if any exists. If there is no current transaction, then one is created before calling the output open parameter, then committed at the end of the droplet:
<droplet bean="/atg/dynamo/transaction/droplet/Transaction"> <param name="transAttribute" value="required"> <oparam name="output"> ... do repository item work ...
36
5 - SQL Repository Architecture
</oparam> </droplet>
You can also use the JTA (Java Transaction APIs) to explicitly manage the transaction. For example, you might explicitly create a transaction around a repository item creation or update like this:
TransactionManager tm = ... TransactionDemarcation td = new TransactionDemarcation (); try { try { td.begin (tm); ... do repository item work ... } finally { td.end (); } } catch (TransactionDemarcationException exc) { ... handle the exception ... }
If you are writing a FormHandler component, you can simply extend the class atg.droplet.TransactionalFormHandler. This FormHandler automatically wraps a transaction around all property get method calls called from a page and another transaction around all property set or handle method calls made from a page. See the Working with Forms and Form Handlers chapter of the ATG 6 Dynamo Programming Guide for more information.
37
5 - SQL Repository Architecture
<gsa-template> <option value="investor" code="1"> <option value="broker" code="2"> <option value="guest" code="3"> </property> </table> </item-descriptor> </gsa-template>
This example modifies the standard repository definition as follows: changes the data-type of the userType property from int to enumerated, adds a few options for the enumerated value of the userType property, sets the cache-mode of the user item-descriptor to locked mode, and decreases the default item-cache-size from 1000 (the default) to 500.
If your base SQL repository definition file sets the expert attribute of a property to true, and if supplemental SQL repository definition files modify that property, you must also explicitly set the expert attribute of a property to true in the supplemental SQL repository definition files; otherwise the attributes value will revert to the default specified in the DTD.
38
5 - SQL Repository Architecture
A repository organizes repository items into types that have the same set of properties. Each item type is defined by an item descriptor.
Dynamo lets you represent this in a single repository using two independent types named book and author. They are independent in that they do not share any properties in their item descriptors. They may happen to each have properties like name or weight, but their properties are independently defined. Another way to look at it is that they each have their own item descriptor. The SQL repository also supports a simplified form of inheritance for item descriptors. See Item Descriptor Inheritance in the SQL Repository Data Models chapter.
39
5 - SQL Repository Architecture
40
5 - SQL Repository Architecture
Repository items correspond to business objects, like customers, and elements of business objects, like a customers shipping address. An item descriptor in a SQL repository defines a single type of repository item. It specifies the properties of its repository items and the database tables and columns that store these properties. This chapter describes how to define item descriptors and how to represent the relationships between item descriptors in a SQL repository definition. Note that a SQL repository cant necessarily work with any arbitrary SQL database model. The basic data model patterns are described in the Sample SQL Repository Definition Files section of the SQL Repository Reference chapter.
properties...
</table>
id Property
In order to retrieve a specific repository item, you need its repository ID. The primary table defines an id property, which specifies the repository ID of repository items of this item type. You can specify this property using the id-column-names attribute, and you do not need to further define the id property in the item descriptor. If you do not define an id property in the item descriptor, then the id property must use the default data-type, which is string. However, if you do explicitly define the id property in the item descriptor, using a <property> tag, you can query repository items by their ID and you can set a different data-type for the id property. The columns specified by the id-column-names attribute do not have to use the same data-type as each other; you can have a composite repository ID the elements of which are strings, integers, and longs. After a repository item is created, but before it is added to the database, you can change its repository ID by changing the value of the id property. Once the item has become persistent, you can no longer change the ID.
41
6 - SQL Repository Data Models
Single Column and Multi-Column Repository IDs
A repository ID can be represented by a single column in the database, with a corresponding Java type of String, Integer, or Long. A repository ID may also be represented by more than one column in the database, each column of which can be either String, Integer, or Long. This type of repository ID is referred to as a multi-column ID or composite key ID. A single property can be used to represent a single column of a multi-column ID or might represent all of the ID columns. So, both of the following are valid: Here, the ID property represents the two ID columns:
<table name="doc" type="primary" id-column-names="folder_id,doc_id"> <property name="ID" column-names="folder_id,doc_id" data-types="string,int"/> </table>
In this case, the folder property represents one of the ID columns, while the document property represents the other:
<table name="doc" type="primary" id-column-names="folder_id,doc_id"> <property name="folder" column-names="folder_id" data-type="string"/> <property name="document" column-names="doc_id" data-type="int"/> </table>
Both single-column repository IDs and multi-column repository IDs are encoded as strings. By default, a multi-column ID is encoded by concatenating the IDs elements, in the order specified by the item descriptors id-column-names attribute, with each element separated by a separator character. By default, this separator character is the colon (:). You can specify a different separator character using the item descriptors id-separator attribute. For example, in an item descriptor defined like this:
<item-descriptor name="user" id-separator="*"> <table name="user" type="primary" id-column-names="dept_id,emp_id">
properties...
</table> </item-descriptor>
you might have repository IDs that are string-encoded like this:
sales*bbanzai
You should not use brackets or commas for the separator character, since these characters are used by RQL and the SQL repository when specifying lists of IDs.
42
6 - SQL Repository Data Models
primary-table-name.id-column-names
properties...
</table> </item-descriptor>
the default IdSpace would be named user. In an item descriptor with a composite repository ID defined like this:
<table name="user" type="primary" id-column-names="dept_id,emp_id">
properties...
</table>
the default IdSpaces would be named user.dept_id and user.emp_id. In any case, you can override the default IdSpace names using the id-space-names attribute in the item descriptor definition:
<table name="user" type="primary" id-column-names="dept_id,emp_id" id-space-names="DepartmentId,EmployeeId">
properties...
</table>
See the ID Generators section of the Core Dynamo Services chapter in the ATG 6 Dynamo Programming Guide for more information about ID space names and how they affect the IDs of newly generated items.
Auxiliary Tables
You can handle some data relationships using auxiliary attribute tables. For example, you could store users and their addresses in two related database tables, as described in the following piece of an XML repository definition:
43
6 - SQL Repository Data Models
<item-descriptor name="user"> </table> <property name="city"/> <property name="state"/> <property name="zip"/> </table> </item-descriptor>
<table name="dps_user" type="primary" id-column-names="id"> <property name="login" data-type="string"/> <table name="dps_address" type="auxiliary" id-column-names="id"> <property name="address1"/>
Each user has a single address. For the purposes of this example, the user information is stored in a separate table from the users address information. Note that if you use auxiliary tables, each table definition, whether primary or not, must define an idcolumn-names attribute. This attribute defines the column names in the table that represent the repository ID. This tells us how to join auxiliary tables to the primary table. The columns in the idcolumn-names attribute must be listed in the same order as they are in the id-column-names attribute of the primary table.
References Constraints
In general, auxiliary and multi tables should not have REFERENCES constraints that point to each other. Instead, each of these tables can have a REFERENCES constraint that points to the primary table for the repository item. This limitation exists because the SQL repository processes insert and delete statements for auxiliary and multi tables in the same order. As a result, if you specify REFERENCES constraints between an auxiliary and a multi table or vice versa, a constraint error will result on either the insert or the delete.
The column-names attribute of a property specifies the database column that stores the property. Only id properties can have more than one column in their column-names attribute. Each property must also define its data type, using the data-types attribute. In the case of a multi-column id property, the datatypes attribute is a comma-separated list of data types, each entry of which corresponds to an entry in the column-names attribute. Each column can have a different data-type. If no data type is specified, the string data type is used by default. The valid data type names and their Java and SQL representations are
44
6 - SQL Repository Data Models
listed in the Data Type Correspondences section of SQL Repository Definition Tag Reference in the SQL Repository Reference chapter.
2.
Note that the multi-column-name attribute ensures that the ordering of the multivalues are maintained. The column specified by the multi-column-name attribute is used for multi-valued properties of data-type array, map, and list and is not used for sets (which are unordered). For map type properties, the values in the column specifiedy by the multi-column-name attribute must be a string. For list or array type properties, these values should be an integer or numeric type, and must be sequential. 3. The multi-valued property in this table must have a data-type of array, set, map or list:
<property name="..." column-name="interest" data-type="array" ...
4.
If the property is a collection of primitive data types (string, int, double, etc.), specify the data type of the members of the collection by using the component-data-type attribute in the <property> tag for the multi item property:
<property name="interests" column-name="interest" data-type="array" component-data-type="string"/>
Note that the SQL repository does not support multi-valued collections of binary type members. 5. If the property is a collection of repository items defined by other item descriptors (for example, an array of users), specify the repository item type of the members of the collection by using the component-item-type attribute to the <property> tag for the multi item property:
<property name="..." column-name="designers" data-type="array" component-item-type="user"/>
45
6 - SQL Repository Data Models
6. 7.
The value of the component-item-type attribute is the name of the item descriptor that defines the item type of the members of the collection of repository items. As with auxiliary tables, the ordering of the ID column names is important. The columns in the id-column-names attribute must be listed in the same order as they are in the id-column-names attribute of the primary table. You cannot establish a default value for multi-valued attributes.
The following example shows how a multi-valued attribute named interests can be expressed in the XML repository definition:
<item-descriptor name="user"> <table name="dps_user" id-column-names="id" type="primary"> <property name="login" data-type="string"/> </table> <table name="dps_interest" type="multi" id-column-names="id" multi-column-name="idx"> <property name="interests" column-name="interest" data-type="array" component-data-type="string"/> </table> </item-descriptor>
See also the Sample SQL Repository Definition Files section in the SQL Repository Reference chapter for more examples of one-to-many relationships in repository definitions.
Now you can use toUseElsewhere either in a writeObject call or in another setPropertyValue call.
Many-to-Many Relationships
You can also represent many-to-many data relationships in a SQL repository. For example, an author may have written multiple books, and a book may have multiple authors. Representing this kind of relationship depends on the type="multi" attribute in a <table> tag. You can represent a many-tomany relationship using two one-to-many relationships that point to the same intermediate table. The
46
6 - SQL Repository Data Models
following example represents a many-to-many relationship between the authors of a book and the books written by an author. Author items use a primary database table named author, book items use a primary database table named book, and both author and book items use a multi table named author_book as an intermediate table that handles the relationship between authors and books.
<item-descriptor name="author"> <table type="primary" name="author"> ... </table <table type="multi" name="author_book" id-column-names="author_id"/> <property name="booksWritten" column-name="book_id" data-type="set" component-item-type="book"/> </table> </item-descriptor> <item-descriptor name="book"> <table type="primary" name="book"> ... </table <table type="multi" name="author_book" id-column-names="book_id"/> <property name="authors" column-name="author_id" data-type="set" component-item-type="author"/> </table> </item-descriptor>
This example uses three tables, author, book, and author_book. The data type of the properties in the intermediate multi table must be set, not array, map or list. Note also that tables can have columns other than the ones referenced in the repository definition file, so long as such columns allow null values and so long as there is no design requirement that the repository recognize the existence of such columns.
47
6 - SQL Repository Data Models
Cascading Data Relationships
The SQL repository uses the cascade attribute in a <property> tag to better handle hierarchical properties, which is to say properties with either the item-type or component-item-type attributes. The cascade attribute can have one or more of the values insert, update, or delete. For example:
<property name="scenarios" item-type="scenario" cascade="update,delete"/>
Cascade Insert
If a repository item has a property with the item-type attribute and the cascade="insert" attribute set, then when the item is created: a new item of the type declared by the item-type attribute is also created; and this property is set to point to the other item created.
The cascade="insert" attribute is typically used with cascade="update" and cascade="delete" so that management of this item is completely automatic. The item is created, added, updated, and deleted along with the parent item. The cascade="insert" attribute is ignored for properties that use component-item-type.
Cascade Update
If a repository item has a property that refers to other items and that has the cascade="update" attribute set, then: when you call addItem(), any new (transient) items referenced by this property are added automatically to the repository; and when you call updateItem, any referenced items that have been modified are automatically updated. Any referenced items that are new (transient) items are added.
Cascade Delete
If a repository item has a property with the cascade="delete" attribute set, then when you remove the repository item, any items that are referenced by the property will also be removed. Also, when you remove a reference to this item, the item is automatically removed. You must take special care in using cascade delete in one-to-many relationships. Do not use cascade="delete" in properties on the many side of the relationship that refer to items on the one side of the relationship. The item on the one side of the relationship cannot be deleted safely, since multiple items may be referring to it. For example, suppose you have an item descriptor named company with an employee property that references many repository items defined by an employee item descriptor. The employee item descriptor itself defines a company property. In this one-to-many relationship, the employee property in the company item descriptor could use cascade="delete". However, the company property in the employee item descriptor should not use cascade="delete", since deleting one employee item would then delete the company item that is referenced by the remaining employee items.
48
6 - SQL Repository Data Models
You can use one of the following three values in the cascadeDeleteOrder attribute tag:
Value
first
Description Cascade deletion is performed before any deletes on the tables of this item. Cascade deletion is performed after deleting auxiliary multitable rows, but before deleting the primary table row. This is the default behavior. Cascade deletion is performed after all deletes on the tables of this item.
afterAuxiliaryBeforePrimary
last
49
6 - SQL Repository Data Models
Cascade Example
For example, consider the following item descriptors, which define an author repository item type and an address repository item type. The address property of the author repository item type has the attribute item-type="address", indicating that the value of the address of an author is a repository item of type address. The address property has the attribute cascade="insert,update,delete"; as a result, whenever an author type repository item is created, added, updated, or deleted, the corresponding address repository item will be also.
<!-<!-- The "author" item type --> <item-descriptor name="author"> <table name="author" id-column-names="author_id" type="primary"> <property name="name"/> <property name="address" item-type="address" cascade="insert,update,delete"/> </table> </item-descriptor> <!-<!-- The "address" item type --> <item-descriptor name="address"> <table name="address" id-column-names="address_id" type="primary"> <property name="streetAddress"/> <property name="city"/> <property name="state"/> <property name="zip"/> </table> </item-descriptor>
<!-- The "coat" item type --> <item-descriptor name="coat"> <table name="coat" id-column-names="id" type="primary"> <property name="name"/> <property name="description"/> <property name="color"/>
50
6 - SQL Repository Data Models
<property name="shippingWeight"/> <property name="size"/> <property name="season"/> </table> </item-descriptor> <!-- The "shorts" item type --> <item-descriptor name="shorts"> <table name="shorts" id-column-names="id" type="primary"> <property name="name"/> <property name="description"/> <property name="color"/> <property name="shippingWeight"/> <property name="size"/> <property name="pleated" data-type="boolean"/> </table> </item-descriptor>
And the database data model would have two unrelated tables:
This would work, but has a few drawbacks: Coats and shorts have a lot of properties in common, which in the above model means duplicated database columns and probably duplicated code. This model does not allow you to easily perform a query like: find all the items of clothing (shorts and coats) that have Star Wars in their description
An object-oriented approach like that used by the SQL repository allows you to define a base item descriptor class called clothing to hold the attributes common to coats and shorts. You can use simple inheritance to make coats and shorts subclasses of clothing. You can then model the data in your clothing catalog like this:
and the corresponding XML repository definition (with changes in bold type) becomes:
51
6 - SQL Repository Data Models
<option value="coat"/> <option value="shorts"/> </property> <property name="name"/> <property name="description"/> <property name="color"/> <property name="size"/> <property name="shippingWeight"/> name="shippingWeight"/> </table> </item</item -descriptor>
<!-<!-- The "clothing" item type, a base type --> --> <item<item -descriptor name="clothing" subsub-typetype-property="type"> <!-<!-- This is the primary table that holds clothing data --> --> <table name="clothing" type="primary" idid-columncolumn-names="id"> <property <p roperty name="type" datadata-type="enumerated">
<!-<!-- The "coat" item type, now a subclass of "clothing" --> <item-descriptor name="coat" supersuper-type="clothing" subsub-typetype-value="coat"> value="coat" <table name="coat" type="auxiliary" id-column-names="id"> <property name="season"/> </table> </item-descriptor> <!-<!-- The "shorts" item type, now a subclass of "clothing" --> <item-descriptor name="shorts" supersuper-type="clothing" subsub-typetype-value="shorts"> <table name="shorts" type="auxiliary" id-column-names="id"> <property name="pleated" data-type="boolean"/> </table> </item-descriptor>
Weve done two things here: Created the parent item descriptor, clothing, that holds attributes common to the coat and shorts types. Made the coat and shorts types into direct subclasses of the clothing type.
The clothing parent item descriptor has a sub-type-property attribute. This attribute defines the property of the clothing type that specifies which sub-types can be instantiated. Each item descriptor that extends the clothing item descriptor must specify a sub-type-value attribute, which is the value for this property that triggers use of this particular type. The sub-type-value must not be null. This is the value of the sub-type-property, which automatically implies this item type. If there is no subtype-value, this item-descriptor itself will never be returned from a getItem or query on one of its super-type item descriptors. Such an item descriptor may have one or more item descriptors that have it as its super-type and thus can serve as kind of an abstract item-descriptor. It should be noted that instances of objects are associated with their superclasses by ID. So, in this example, the ID of a coat always has a matching clothing ID.
52
6 - SQL Repository Data Models
As noted above, the sub-type-value of a sub-type item should never be null. If you have clothing items that are neither coats nor shorts, you should create a sub-type-value in the clothing item descriptor with a value of clothing and add a clothing option to the sub-type-property definition. For example:
<!-<!-- The "clothing" item type, a base type --> <item-descriptor name="clothing" sub-type-property="type" subsub-typetype-value="clothing"> value="clothing <!-<!-- This is the primary table that holds clothing data --> <table name="clothing" type="primary" id-column-names="id"> <property name="type" data-type="enumerated"> <option value="clothing"/> <option value="coat"/> <option value="shorts"/> <property/> ...
From the Repository API point of view, each ItemDescriptor maps to a single RepositoryView. When a SQL repository uses item type inheritance, each parent item type results in a RepositoryViewContainer that contains its subtype views as children.
Most important, the items returned will be a combination of coat items and shorts items. Without SQL repository support for inheritance, this query would have to be run against each subtype in turn. At first blush this may not seem significant, but envision a store with perhaps ten subtypes like this. Running this query against all of the subtypes would be painful. By building the inheritance knowledge into the SQL repository, we enable it to optimize the actual SQL queries needed. Here is what the code for this query would look like:
// get hold of the repository Repository gsa = ...; // get the view to use for querying "clothing" type items RepositoryView clothingView = gsa.getView("clothing"); // get a query builder QueryBuilder qb = clothingView.getQueryBuilder(); // build the query QueryExpression weightLimit = qb.createConstantQueryExpression(new Integer(2)); QueryExpression itemWeight = qb.createPropertyQueryExpression("shippingWeight");
53
6 - SQL Repository Data Models
Query q = qb.createComparisonQuery(itemWeight,
weightLimit, QueryBuilder.GREATER_THAN); // run the query RepositoryItem[] items = clothingView.executeQuery(q); // separate the coats and shorts and do whatever with them for (int i=0; i<items.length; i++) { RepositoryItem item = items[i]; // all clothing items have a name and a description logDebug("clothing: " + item.getPropertyValue("name") + ' ' + item.getPropertyValue("description")); // the ItemDescriptor defines the "type" of an item RepositoryItemDescriptor desc = item.getItemDescriptor(); // now we do different things, depending on the // type of clothing item we have if (desc.getItemDescriptorName().equals("coat") { // coats have a property called "season" logDebug("\tcoat, season = " + item.getPropertyValue("season")); // do coat-related things myCoatProcessor(item); } else { // shorts have a property called "pleated" logDebug("\tcoat, season = " + item.getPropertyValue("pleated")); // do shorts-related things myShortsProcessor(item); } }
In this example, we used the name of the item descriptor to determine the item type. You can also look at the value of the type property declared in your template. In our example, wed define the enumerated properties with the useCodeForValue attribute set to true and then the query would go something like:
54
6 - SQL Repository Data Models
Which technique to use is up to you and may be largely a matter of style. The item descriptor approach uses the actual name like coat or shorts. The type attribute approach uses the type code stored in the clothing table: typically something like 0 or 1, as in this case.
You can then create queries like these, even though the products item descriptor does not include
waterproofRating, size, or channelCount properties: products whose waterproofRating is 'JIS-4' products whose channelCount = 7 products whose waterproofRating is 'JIS-4' OR whose size > 7
55
6 - SQL Repository Data Models
Limitations of SQL Repository Inheritance
However, since these item descriptors will share the same tables (and thus the same data), it may not be wise to use both the old and new item descriptors. Instead, it may be preferable to use the super-type attribute and sub-type-property attribute if you want to use both.
The SQL repositorys inheritance support is simplified. By simplified we mean the following: A type can only inherit from one parent. No multiple inheritance is allowed. A class hierarchy can only have a single sub-type-property value. You can define a second level of sub-classing -- for example, you might define an item descriptor named bermuda-shorts that has shorts as its super-type-- but you cannot have another different sub-type-property.
All parent item descriptors (item descriptors that are used in super-type or copy-from attributes) must be fully defined by the time they are referenced in the XML repository definition file. They can either be defined in front of the new XML file in the same file or specified in an XML file that is parsed before this XML file. You should avoid using too many levels of inheritance. Queries against items whose properties span multiple sub-types may require joins of all of the tables in the hierarchy. If you use these kinds of queries, keep in mind that performance decreases as the number of tables joined increases.
Derived Properties
In a SQL repository, you can use derived properties. This feature enables one repository item to derive property values from another repository item or from another property in the same repository item. To illustrate: some data models are organized in a tree structure in which certain property values are passed down from other properties. For example, an organization might have divisions, departments, and employees, organized in a tree structure. A repository represents this tree structure with division, department, and employee item descriptors. Each of these item descriptors might define a property called spendingLimit. A business rule might specify that an employees spending limit comes from their department if it isnt set for that employee. If the spending limit is not set for the department then it should be derived from the spending limit for the division. This derived property relationship would be represented in a repository definition file like this:
<item-descriptor name="employee"> <property name="department" item-type="department"/> <property name="empSpendingLimit" data-type="int"/> <property name="spendingLimit" writable="false"> <derivation> <expression>empSpendingLimit</expression> <expression>department.spendingLimit</expression> </derivation> </property>
56
6 - SQL Repository Data Models
</item-descriptor> <item-descriptor name="department"> <property name="division" item-type="division"/> <property name="deptSpendingLimit" data-type="int"/> <property name="spendingLimit" writable="false"> <derivation> <expression>deptSpendingLimit</expression> <expression>division.divSpendingLimit</expression> </derivation> </property> </item-descriptor> <item-descriptor name="division"> <property name="division" item-type="division"/> <property name="divSpendingLimit" data-type="int"/> </item-descriptor>
Derived properties can use multiple levels of subproperties. So in our spending limit example, we might derive the employees spending limit this way:
<item-descriptor name="employee"> <property name="department" item-type="department"/> <property name="spendingLimit" data-type="int" writable="false"> <derivation> <expression>department.employeeDefaultInfo.spendingLimit</expression> </derivation> </property> </item-descriptor> <item-descriptor name="department"> <property name="employeeDefaultInfo" item-type="employeeInfo"/> <property name="deptSpendingLimit" data-type="int"/> </item-descriptor> <item-descriptor name="employeeInfo"> <property name="spendingLimit" data-type="int" writable="false"/> <property name="officeLocation" data-type="string"/> </item-descriptor>
Bear in mind that using derived properties can affect performance, and that the more complex the derivation, the greater the impact is likely to be. The FirstNonNull derivation method must be non-writable, unless you set a writable override property for the derived property. You can set a property to be not writable like this:
<property name="spendingLimit" data-type="int" writable="false"/>
57
6 - SQL Repository Data Models
Override Properties
The FirstWithAttribute and FirstWithLocale derivation methods can be writable, even if the property does not define a writable override property. See Override Properties in this section for more information.
There are times when you might want to explicitly set a property value rather then have the property derivation logic supply one. This is done by specifying an override-property attribute in the <derivation> tag:
<item-descriptor name="employee"> <property name="department" item-type="department"/> <property name="empSpendingLimit" data-type="int"/> <property name="spendingLimit"> <derivation override-property="empSpendingLimit"> <expression>department.spendingLimit</expression> </derivation> </property> </item-descriptor>
The above change specifies that if the empSpendingLimit property is not null then it is used as the value of the spendingLimit property, otherwise the spendingLimit property is derived as before. Using the override-property attribute lets you edit in a Repository Editor a property that would otherwise be derived from another property.
<item-descriptor name="user"> <property name="shipToAddress" writable="false" item-type="address"> <derivation> <expression>shippingAddress</expression> <expression>homeAddress</expression> </derivation> </property> <table name="user" > <property name="shippingAddress" item-type="address"> <property name="homeAddress" item-type="address"> </table> </item-descriptor>
58
6 - SQL Repository Data Models
Complex Derivations
The examples given so far show a comparatively simple tree model, but actually a repository item property can be derived from any other properties, so long as the properties can be represented as vertices on a path of a directed acyclic graph. That is, the derivation of a property can have multiple paths that are traversed to derive the value. For example, a shipping address might derive from the users specified shipping address, the users billing address or home address, or the users companys address:
<item-descriptor name="user"> <property name="shipToAddress" writable="false" item-type="address"> <derivation> <expression>shippingAddress</expression> <expression>billingAddress</expression> <expression>homeAddress</expression> <expression>company.address</expression> </derivation> </property> <table name="user" > <property name="shippingAddress" item-type="address"/> <property name="billingAddress" item-type="address"/> <property name="homeAddress" item-type="address"/> <property name="company" item-type="company"/> </table> </item-descriptor>
Note that this example assumes youve also defined item descriptors named address and company. To determine the value of the shipToAddress for a user, the expressions specified in the derivation are searched in order. Any expression may also refer to properties that are themselves derived.
Derivation Methods
A derived property definition can specify one of several different derivation methods to determine the appropriate property value. The SQL repository traverses in order each of the expressions in the <derivation> tag, applying the specified derivation method. There are six derivation methods included in ATG 6: firstNotNull Derivation Method firstWithAttribute Derivation Method firstWithLocale Derivation Method alias Derivation Method union Derivation Method collectiveUnion Derivation Method
59
6 - SQL Repository Data Models
firstNotNull Derivation Method
By default, the SQL repository derives a property by traversing the expressions in order, starting with the property itself. The first non-null value found is used as the property value. This is the firstNonNull derivation method. The firstNonNull method is the default derivation method, and so it is not necessary to specify it in the XML. However, the derivation method can be specified in the method attribute of a <derivation> tag in the SQL repository definition file, as in this example:
<item-descriptor name="employee"> <property name="department" item-type="department"/> <property name="empSpendingLimit" data-type="int"/> <property name="spendingLimit" writable="false"> <derivation method="firstNonNull"> <expression>empSpendingLimit</expression> <expression>department.spendingLimit</expression> </derivation> </property> </item-descriptor>
<item-desciptor name="myItem"> <property name="name"> <derivation method="firstWithAttribute"> <expression>englishName</expression> <expression>icelandicName</expression> <expression>shonaName</expression> </derivation> <attribute name="derivationAttribute" value="language"/> <attribute name="defaultKey" value="en"/> </property> <property name="englishName"> <attribute name="language" value="en"/> </property> <property name="icelandicName"> <attribute name="language" value="is"/> </property> <property name="shonaName">
60
6 - SQL Repository Data Models
If getKey returns sn (the user is in Zimbabwe, for example) then myItem.name will return the same value as myItem.shonaName.
Compares this locale to each expressions locale value. Returns the first property whose attribute matches.
The locale is searched in a locale-specific way. For example, if locale=fr_FR_EURO, it first looks for a property where the locale attribute is fr_FR_EURO, then looks for fr_FR, and finally looks for fr. There is also a defaultKey, which the keyService uses if the value with the real key is null. In other words, if the real key is de_DE and we are looking for displayName, but displayName_de is null, then return displayName_en instead (assuming its locale is en and the defaultKey is en or en_US). Using a defaultKey may slow performance. If no default key is defined, it is not used. If the default key is the same as the current key, there are no performance implications. In all other cases, there is an extra clause on all search terms, which can result in a slower search. Here is an example of a derived property definition using the firstWithLocale derivation method:
<property name="displayName data-type="string"> <derivation method="firstWithLocale"> <expression>displayName_en</expression> <expression>displayName_de</expression> </derivation> <attribute name="derivationAttribute" value="locale"/> <attribute name="keyService" value="/atg/userprofiling/LocaleService"/> <attribute name="keySubProperty" value="locale"/> <attribute name="defaultKey" value="en"/> </property>
61
6 - SQL Repository Data Models
<item-descriptor name="user" ...> <table name="USER" ...> <property name="firstName" ...> .... </table> <property name="name1"> <derivation method="alias"> <expression>firstName</expression> </derivation> </property>
For example, suppose an item descriptor defines a property named firstName. You want some application code to be able to refer to this property as name1. You can use the Alias derivation method to define name1 to be the equivalent of firstName, as follows:
In this example, when the name1 property is accessed, the firstName property of the item will be returned.
The siblings property represents a union of values in the sets brothers and sisters. The data type of the values in the collections defined in all the expressions of the derived property must be the same. If two or more of the properties to be combined in the Union derived property include the same element, then the Union derived property will have duplicate elements if the property is of type list, but will have unique elements if the property is of type set.
62
6 - SQL Repository Data Models
<expression>parentProducts</expression>
a catalogs property that is derived from a union of the catalogs properties of the items that make up the parentProducts property.
<item-descriptor name="sku" ...> <property name="catalogs"> <derivation method="collectiveUnion"> </derivation> <attribute name="collectionProperties" value="catalogs"/> </property> <table name="sku_prod" ... > <property name="parentProducts" data-type="set" component-item-type="product"> ... </table> </item-descriptor> <item-descriptor name="product" ...> <table name="prod_com" ... > <property name="catalogs" ... /> ... </table> </item-descriptor>
In this example, the union of product.catalogs will be returned for each product in the parentProducts property of the sku item. The derived property will be accessible using the catalogs property of the sku item.
you can cause a particular property not to be serialized. If you serialize a persistent item, only the transient properties (those not in a table tag) are serialized unless you explicitly set the feature descriptor attribute like this:
63
6 - SQL Repository Data Models
<property ...> <attribute name="serialize" value="true"/> </property>
If the item is persistent, its persistent properties will have been updated in the database already, and can be retrieved from the database if the session needs to be restored. See the Session Management chapter in the ATG 6 Installation and Configuration Guide for Dynamo Application Server for more information.
64
6 - SQL Repository Data Models
An item descriptor in a SQL repository can define various useful special types of properties. The following section describe some of these special property types, as well as other useful repository property attributes: Enumerated Properties Required Properties Unique Properties Date and Timestamp Properties Null Properties Property Validation with a Property Editor Class Maintaining Item Concurrency with the Version Property Repository Items as Properties Transient Properties Assigning FeatureDescriptorValues with the <attribute> Tag Linking between Repositories SQL Types and Repository Data Types User-Defined Property Types Property Fetching Handling Large Database Columns
Enumerated Properties
Properties of repository items can have a data type of enumerated. Enumerated properties are string properties constrained to a predefined list of valid values and stored as integer codes in the database. A TaggedPropertyEditor is registered for enumerated properties so that components like user interfaces can access the list of valid values. For these reasons, the list of valid values should be small. For example, it is not appropriate to use an enumerated attribute for something that could take on more than a hundred values. Instead, you might make a reference to another item with a single int property. Here is an example of an item descriptor definition that creates an enumerated property named
transactionType:
65
7 - SQL Repository Item Properties
<!-- The "transaction" item type --> <property name="amount" data-type="int"/> <option value="debit"/> <option value="purchase"/> </property> </table> </item-descriptor>
<item-descriptor name="transaction"> <table name="transaction" id-column-names="xact_id"> <property name="transactionType" data-type="enumerated"> <option value="credit"/>
In this example the list of valid String values is specified explicitly and the corresponding integer codes are generated by the SQL repository when the template is initialized. It is also possible to specify the integer codes explicitly, using the code attribute in the <option> tag:
<property name="transactionType" data-type="enumerated"> <option value="credit" code="0"/> <option value="debit" code="1"/> <option value="purchase" code="2"/> </property>
By default, an enumerated property returns its value as an integer code. You can instead configure an enumerated property so that the repository converts the integer code into a string value. Do this by using the useCodeForValue attribute in the property definition, as in the following example:
<property name="gender" data-type="enumerated"> <attribute name="useCodeForValue" value="false"/> <option value="male" code="0"/> <option value="female" code="1"/> </property>
In the above property example, with useCodeForValue set to false, then if you get the gender property, the string male or female is returned. With useCodeForValue set to true, then the integer code 0 or 1 would be returned instead. If your enumerated property returns an integer code, you can get the property editor for your enumerated property and use that to create a property editor that can convert between the string value and the integer code value. See the JavaBeans specification for a description of PropertyEditors.
66
7 - SQL Repository Item Properties
Reserved Option Code Values 0 - 999 (Note that some of these values may have already been used for that property in ATG products in versions 5.0 and 5.1. Check the repository definition file for the property in those cases. It may be better to limit your option code values to 101 - 999.) 1000 - 1999 2000 - 2999 3000 - 3999 4000 - 5999 8000 - 8999 6000 - 7999, 9000 +
In order to avoid collisions between option codes used by your repository definition and any option codes used in subsequent ATG products, you should observe the following convention in assigning option codes to enumerated property values:
DAS DPS DSS ATG Commerce ATG Publishing future ATG use
In the unlikely event that you need more than 1000 separate values for a single enumerated property, you could also use negative integers as option code values.
Required Properties
You can specify that a repository item property is required, using the required attribute in a property tag:
<property name="lastName" data-type="string" required="true" />
Then, if you also set the enforceRequiredProperties property of the GSARepository component to true (the default value), the repository checks to make sure all required properties are present when adding repository items and forbids the setting of a required property to null. Make sure your repository definition corresponds to your database schema; if your database schema defines a property as NOT NULL, your repository definition should define the property with the required="true" attribute. Note: If a property references an item that is defined in the database as NOT NULL but you cannot mark the property as required, indicate this by adding the references attribute tag and set its value to true, as in the following example:
67
7 - SQL Repository Item Properties
</property>
Unique Properties
You may want a repository item property to require a unique value. For example, you probably want each user profile to contain a login property that is unique to that profile and not shared by any other. You can mark a property as unique using the <attribute> tag, as in the following example:
<property name="login" data-type="string" required="true"> <attribute name="unique" value="true"/> </property>
Repository editors in the ATG Control Center enforce the requirement that the value be unique.
For more information about this technique, see the Assigning FeatureDescriptorValues with the <attribute> Tag section in this chapter.
Last-Modified Properties
In some applications, it is useful to know when a repository item has been modified most recently. You can set up a special property to store the last-modified time of a repository item. To configure a lastmodified property: 1. Add a last-modified property to the item descriptor. This property must be a persistent, single-valued property. Its datatype must be either a date or a timestamp. For example:
<property name="lastActivity" data-type="timestamp"/>
2.
Add the last-modified-property attribute to the item descriptor and set its value to the name of your last-modified property. For example:
68
7 - SQL Repository Item Properties
3.
Include an updateLastModified <attribute> element in the item descriptor with its value set to true. For example:
<attribute name="updateLastActivity" value="true"/>
The item descriptor might then look, in relevant part, like this:
<item-descriptor name="article" last-modified-property="lastActivity"> <attribute name="updateLastActivity" value="true"/> <table name=ARTICLES type="primary" ...> <property name="lastActivity" data-type="timestamp"/> ... </table> </item-descriptor>
When items of this type are added or updated, the lastActivity property will be updated with the current time.
Null Properties
If a propertys definition does not set the default value of a property, then its default value is null. For example, the following <property> tag defines a property named favoriteColor whose default value is null:
<property name="favoriteColor" data-type="string"/>
If you want to affirmatively set a default value to null, use this form:
<property name="favoriteColor" data-type="string" default="__NULL__"/>
This technique is useful if, for example, you are combining two or more repository definition files into a single template and need to override a non-null value and restore the default to null.
Sorting Properties
You can use the category attribute in a <property> tag to group similar properties together in a Repository user interface. All properties with the same category attribute value are listed together. For example, Login name and Password are listed together under the heading Login. If you examine the <property> tags that define the login and password properties, you will see that each <property> tag uses the attribute category="Login".
69
7 - SQL Repository Item Properties
For example:
If you examine the Basics category in the ATG Control Center user profile display, you will see the properties First name, Middle name, and Last name listed in that order. The order these properties appear in the ATG Control Center depends on the propertySortPriority attribute. Properties with the same category attribute are listed in ascending order according to the integer value of their propertySortPriority attributes. Properties that do not have a propertySortPriority attribute are listed in alphabetical order.
<property category="Basics" name="firstName" data-type="string" display-name="First name"> <attribute name="propertySortPriority" value="-3"/> </property>
Category Ordering
Categories are listed in the ACC in the following order: 1. 2. 3. 4. All named categories that contain required properties (properties with the attribute required="true"). All named categories that do not contain required properties. A special anonymous category containing properties that are not assigned to any category. A special Groups category containing boolean properties that correspond to content or profile groups.
Within each of these four sets, categories are listed in alphabetical order.
Property Ordering
Properties are listed together with other properties that have the same category attribute. Within a category, properties are listed in ascending order, according to the value of their propertySortPriority attribute. The default value for propertySortPriority is 0. Properties that have the same propertySortPriority value are listed in alphabetical order, according to their displayName attribute. The propertySortPriority attribute is set in an <attribute> tag that is a child tag of the <property> tag, as in this example:
<property category="Basics" name="firstName" data-type="string" display-name="First name"> <attribute name="propertySortPriority" value="-3"/> </property>
70
7 - SQL Repository Item Properties
A property editor can be used to validate a property value. Note, however, that a property editor doesnt have access to the repository item, only the property. Therefore, you cant use a property editor to make comparisons with other properties. You can also limit a propertys values to a list of valid choices using an enumerated property. See the Enumerated Properties section.
The value of the version property is incremented every time the item is updated. Its value starts as 0 when the item is created, is set to 1 when the item is added, and then is incremented in each subsequent update. The version number for a particular item is read and associated with that transaction the first time that item is referenced in a transaction. If you attempt to update the item from a transaction whose version does not match the current version number in the database, a ConcurrentUpdateException is thrown to abort that update. This exception is a subclass of RepositoryException. Here is a sample scenario that shows how the SQL repository uses the version property to implement optimistic locking: 1. 2. Dynamo1 reads a repository item for update. It obtains the items version property, which has a value, lets say, of 2. Dynamo2 reads the same repository item for update. Since Dynamo1 has not yet committed any changes to the item, Dynamo2 gets the same value for the items version property, 2.
71
7 - SQL Repository Item Properties
3. 4.
Dynamo1 updates the repository item. In the course of the update, the value for the version property in the repository item is checked to see whether it is the same as what is found in the corresponding database column. In this case, both values are still 2. The update to the repository item is committed, with the version property incremented by 1, so that the value of the version property is now 3. Dynamo2 attempts to update the repository item. When the value for the version property in the repository item is checked to see whether it is the same as what is found in the corresponding database column, the values do not match. Dynamo2 is holding a value of 2, while the value of the version property in the database is now 3. Dynamo2 throws a ConcurrentUpdateException and does not apply the changes in the update.
This can be very useful for simple and distributed cache modes where there is a possibility of overwriting another Dynamos changes. You can take advantage of optimistic locking in pages that include forms. Often in a form, you read the data in one transaction and update the data in another transaction. There is a possibility that another process might try to update an item in an intermediate transaction. To handle this case, you can place the version property value as a hidden field in your form. Then, you can either check that it is still the same yourself after you start the transaction which updates the item, or you just set the version property (along with the other properties in the item) and deal with the ConcurrentUpdateException when it occurs. For example, you can include in a page a hidden input tag like this:
<input type="hidden" bean="FormHandler.value.version">
You can also use a RepositoryFormHandler, which can set a version property just like it sets any other property of a repository item.
Repository gsa = ...; String myBookId = ...; // get my book from the db RepositoryItem book = gsa.getItem(myBookId,descriptorName); // get the author of my book (it's a dynamic bean too!) RepositoryItem author = (RepositoryItem)book.getPropertyValue("author");
72
7 - SQL Repository Item Properties
Without support for objects as properties, the application would have to get an authorId from the book and then perform another lookup to get the actual author. You can specify that a repository item property is another repository item, rather than a primitive, using the item-type attribute, rather than the data-type attribute. The following example shows a portion of a template that defines two item descriptors, book and author. Repository items of the book item descriptor have an author property whose value is another repository item, an author. Repository items of the author item descriptor have a book property whose value is another repository item, a book.
<!-- The "book" item type --> <item-descriptor name="book" default="true"> <table name="book" type="primary" id-column-names="book_id"> <property name="title"/> <property name="author" column-name="author_id" item-type="author"/> </table> </item-descriptor> <!-- The "author" item type --> <item-descriptor name="author"> <table name="author" id-column-names="author_id" type="primary"> <property name="lastName"/> <property name="firstName"/> </table> <table name="book" id-column-names="author_id" type="auxiliary"> <property name="book" item-type="book" column-name="book_id"/> </table> </item-descriptor>
<!-- The "book" item type --> <item-descriptor name="book" default="true"> <table name="book" type="primary" id-column-names="book_id"> <property name="title"/>
73
7 - SQL Repository Item Properties
</table> </item-descriptor> <!-- The "author" item type --> <item-descriptor name="author"> <property name="lastName"/> <property name="city"/> <property name="state"/> <property name="zip"/> </table>
<table name="book" id-column-names="author_id" type="multi"> <property name="books_written" data-type="set" component-item-type="book" column-name="book_id"/> </table> </item-descriptor>
Note that in this example, the book table is defined twice in the XML, first in the book item descriptor and then in the author item descriptor. The second time, this table is a multi-table where each author item may have more than one row with the id column. In a multi-table, all of the attributes that we define are multi-valued types. To define Array, List and Map types, you also must specify a multi-column-name attribute on the table tag. This specifies which column is to be used as the sorting value to determine the order of the List and the key for the Map. Now the properties author and books_written are actually real beans (in this case RepositoryItems) instead of just simple Java primitives. In the author item descriptor, the books_written property is a Set of RepositoryItems that correspond to books. The other types supported are List, Map, and Array.
This setting makes it easier for you to keep the book repository items synchronized with the author repository items that refer to them. See the Cascading Data Relationships section. You can then add a book item to a set of items in the books_written property like this:
Repository gsa = ...; RepositoryItem newBook = getRepository().createItem("book");
74
7 - SQL Repository Item Properties
If the books_written property does not have cascade="update", you will have to add the item using the addItem() method (thus inserting the row in the database) before you add it to the list:
Repository gsa = ...; RepositoryItem newBook = getRepository().createItem("book"); getProfileRepository().addItem(newBook); Set books_written = (Set) author.getPropertyValue("books_written"); books_written.add(newBook);
Remember that in each of these cases, it would be most efficient to ensure that all of these method calls are performed in a single transaction. See Repositories and Transactions in the SQL Repository Architecture section.
Querying Subproperties
The SQL repository allows applications to query repository items based on attributes of their attributes, sometimes referred to as subproperties. Continuing our book example, subproperty querying means that you can write repository queries like:
Get me all the books that were written by authors living in NY
Transient Properties
The SQL repository allows you to define properties of a repository item that are transient. Transient properties are never stored or read from the persistent data store. They are readable and writable, but not queryable. Transient properties are a convenience that give applications a hook on which to hang their own custom objects which are not persisted by the repository. You can specify a transient property by defining a <property> tag that is not associated with any database table, but which is instead a direct child of an <item-descriptor> tag. For example, in the following example, the user item descriptor has a transient property that specifies whether the user is logged in at that time:
<item-descriptor name="user" sub-type-property="userType"> <property name="loggedIn" data-type="boolean"> <table name="user" type="primary" id-column-names="id"> <property name="userType" data-type="enumerated"
75
7 - SQL Repository Item Properties
column-name="user_type"> ...
You can also define an entire item descriptor to be transient. Such an item descriptor has no <table> tags and no properties that are direct children of a <table> tag. The properties of transient item descriptor are queryable by default, unlike a transient property of an item descriptor with other properties that are persistent properties. In the case of a transient item descriptor, no indexing is used, so queries against large repositories will be slow. Using transient repositories is sometimes a useful testing tool during application development.
<!-- The "book" item type --> <item-descriptor name="book"> <table name="book" id-column-names="book_id" type="primary"> <property name="title"/> <property name="author"> <attribute name="maxLength" value="30"/> </property> </table> </item-descriptor>
It is also useful to be able to refer to values of Nucleus components as attributes. You can do this with the bean attribute of the <attribute> tag. For example:
<attribute name="documentRootPath" bean="/atg/demo/QuincyFunds/repositories/ FeaturesDataStore.relativePathPrefix" />
If you use a relative Nucleus address for the bean attribute, it refers to a component relative to the Repository component.
76
7 - SQL Repository Item Properties
You can access property attributes programmatically using the RepositoryPropertyDescriptor.getValue method. For example:
RepositoryPropertyDescriptor.getValue("maxLength");
Attribute
uiqueryable
Description If this attribute is set to false, the property is not available to a targeting UI element or an ATG expression editor in the ACC. If this attribute is set to false, the property cannot be modified or set in the ACC. Unless the property tag also has the writable=false attribute, the property can still be modified or set programmatically or through a form.
uiwritable
The repository attribute can be used with either the item-type or the component-item-type attribute to indicate that this item is in a different repository, not the current repository. The value of the linked property in the database is the repository ID of the item in the other repository.
77
7 - SQL Repository Item Properties
The repository attribute specifies a Nucleus component name relative to the location of the current repository. This enables you to create a composite repository that combines the item descriptors from more than one repository. Note, however, that a single repository query cant span repositories with different data sources. When you use composite repositories, make sure that your affiliated repositories do not use the same item descriptor names. In this way, your application can go to the composite repository, get an item descriptor named products, and not have to know that it is dealing with a separate repository.
If you set the sql-type or sql-types attribute for every non-transient property in your repository template, then the SQL repository does not attempt to check the JDBC type of your properties. One drawback of this approach is that the sql-type can vary depending on your database software vendor. If you set the sql-type attribute explicitly, you will need to review and update the value if you move your repository implementation to a different vendors database software.
78
7 - SQL Repository Item Properties
with names like repositoryName.properties, one file for each repository in your application. Each file in the /data/schemaInfoCache directory specifies the SQL type of each column in that repositorys schema, in the following format:
tablename.colname=SQLtype
The SQL type in this case is the integer type code specified in the class java.sql.Types. For example, setting the updateSchemaInfoCache to true in the ProfileAdapterRepository component might generate a file that begins like this:
## SchemaInfoCache - auto generated #Tue May 13 17:32:25 EDT 2003 dps_user_scenario.scenarioInstances=12 paf_usr_pgfld.creationDate=93 dps_email_address.emailAddresses=12
These files can be generated against one database and copied to another as long as the schemas are compatible. If you change the schema and this property is enabled, you need to remove the /data/schemaInfoCache directory so it is regenerated. The /data/schemaInfoCache directory does get regenerated automatically if you add a property that refers to a new column or a new table. If those files exist, they are used even if the updateSchemaInfoCache property is set to false. This is so that on a live server, you can use the schema info cache generated by a different staging server. This will, for example, help you in the event that one server in your cluster has some problem that prevents it from generating the schema information.
79
7 - SQL Repository Item Properties
indirectly, by specifying a type name of your type and registering it using the method
atg.repository.RepositoryPropertyDescriptor. registerPropertyDescriptorClass(String typeName, Class pPropertyDescriptorClass)
Note that if you use the indirect method you must set the userPropertyDescriptors property of the GSARepository component to include your type (or, in the alternative, call the registerPropertyDescriptorClass() method at some point before your XML repository definition file gets loaded). Most users will find the direct Java class approach simpler. In either case, you use the property-type attribute in the <property> tag to cause the SQL repository to use a property of your type.
Generally, if you specify a type with the property-type attribute, you do not need to use the data-type or item-type attribute to specify the type. However, in some cases, you might create user-defined properties that can represent more than one data type. In such cases, you can use the data-type attribute to further constrain the property.
If this property refers to another items type, this method sets the item descriptor for that type.
RepositoryPropertyDescriptor.setComponentItemDescriptor()
depending on which attributes are set in the <property> tag (data-type, item-type, componentdata-type, or component-item-type). They define the propertys Java class, the components property class (if the property is a multi-valued property), the RepositoryItemDescriptor for either a scalar or a multi-valued property which refers to other item types.
80
7 - SQL Repository Item Properties
If your property type can accept any of these values, you do not need to override these methods. If your property is constrained in what data types it supports (which is generally the case), you should put error checking logic into these methods to throw errors if an invalid type is specified. Your property descriptor should throw the unchecked IllegalArgumentException to provide details about what type is required. If your property type is very restrictive, you can implement these methods to return the appropriate values: RepositoryPropertyDescriptor.getPropertyType() GSAPropertyDescriptor.getComponentPropertyType() RepositoryPropertyDescriptor.getPropertyItemDescriptor() RepositoryPropertyDescriptor.getComponentItemDescriptor() This will prevent developers of repository definitions from having to set the data-type, componentdata-type, item-type, and component-item-type attributes. You may still want to put error checking in these methods to signal errors if they do provide invalid values. When you attempt to get or set one of these property values, your getPropertyValue or setPropertyValue method is called. The SQL repository provides the RepositoryItemImpl class as an argument to these methods so that you can call the methods setPropertyValue and getPropertyValue of the underlying item. If you call these methods with the same name as your property (this is available in the getName() method on your RepositoryPropertyDescriptor class), it will result in an infinite loop, so be careful. The getPropertyValue method receives an extra Object pValue argument. This is set to any value weve found in the cache for this property name, if any, or null if no value is in the cache. In your setPropertyValue method, you can call setPropertyValueInCache(this, yourvalue) if youd like to cache this property value for subsequent method calls. If your property is not set, you may choose to return the value of the getDefault() method on the RepositoryPropertyDescriptor. This will allow the user to set the default value for this property using the default attribute in the XML tag. This method calls setDefaultValueString, which converts the default value based on the class returned by getPropertyType, which then calls setDefaultValue. You may choose to modify this behavior by overriding these methods though typically this functionality is sufficient. Note that user defined properties must be serializable. The getPropertyValue and setPropertyValue methods do not need to work on an unserialized version, but the getPropertyType, getComponentType, getPropertyItemDescriptor, and getComponentItemDescriptor methods in particular do need to work. This is important so that the ATG Control Center can understand the type of property it is editing. To make your user-defined property queryable, it should represent a database column. Unless your userdefined property extends GSAPropertyDescriptor, then the property is not queryable and you should implement the method isQueryable to return false. If you want your user-defined property to be queryable, make sure it extends GSAPropertyDescriptor. You may also override the methods isWritable and isReadable to turn off write access or read access to your property respectively. Other methods such as isHidden, isExpert can also be overridden if you want to set additional Bean attributes. The method setValue(String pName, Object pValue) is called if any feature descriptor attributes are supplied with this property.
81
7 - SQL Repository Item Properties
Property Conversion Methods
rawToReal
User-defined properties that correspond directly to a column in a SQL table need to extend the class atg.adapter.gsa.GSAPropertyDescriptor. These classes need to implement three additional methods:
This method converts from the database version of the property value to the property value as it would be returned by RepositoryItem.getPropertyValue. For example, for you might convert the ID of an item into the item itself. You do not need to implement this method if the default behavior of the SQL repository is what you want.
realToRaw
This does the opposite of rawToReal. It converts from the version of the value given to setPropertyValue into the value given to the setObject call in JDBC. For example, if you have a property that specifies a reference to another item, you would convert from the RepositoryItem to its ID. For examples of these methods, see the source code for the
atg.adapter.gsa.EnumPropertyDescriptor class at: <ATG6dir>/DAS/src/Java/atg/adapter/gsa/EnumPropertyDescriptor createDBPropertyEditor()
This method is used by some reporting UIs or other tools which need to get the database value directly from SQL, but then want to convert that value to or from a String representation. For example, you might perform a query against the Profile Repository, and receive the code value for an enumerated property. You can then use the createDBPropertyEditor to convert the code value to its String representation for display in a UI, for instance. This method is like the method createPropertyEditor, but the property editor returned from createDBPropertyEditor should operate on the raw value (the value returned from the JDBC getObject call), not the real value (as returned by the RepositoryItem.getPropertyValue call).
82
7 - SQL Repository Item Properties
<item-descriptor name="images"> <table name="book" id-column-names="book_id" type="primary"> <property name="title"/> <property name="author"/> <property name="lastModifiedTime"/> <property name="contentFileName" data-type="string"/> </table> <property name="contentFile" name="contentFile" propertyproperty -type="atg.repository.FilePropertyDescriptor"> <attribute name="pathNameProperty" value="contentFileName"/> </property> </item-descriptor>
property implementation used in this example. If you extend the GSAPropertyDescriptor class, you have two additional methods that you can override. These methods convert data between the type that is stored in the database and the type that is stored in the cache. These methods are called only when the data is loaded from or stored to the database. If a cached value is found, it will be returned without calling these methods. Thus it is slightly more efficient to do conversion here than in the getPropertyValue or setPropertyValue methods.
//------------------------------------/** * Translate a raw property value to a real value. The real value is what * applications use. The raw value is what is stored in the DB. * @param pRawValue for a property * @return real value to use in applications for the property **/ public Object rawToReal(Object pRawValue) //------------------------------------/** * Translate a property value to a raw value. The real value is what
83
7 - SQL Repository Item Properties
**/ public Object realToRaw(Object pRealValue)
* applications use. The raw value is what is stored in the DB. * @param pRealValue for a property * @return raw value for storing this property in the DB
The following example is taken from the productCatalog.xml file in ATG Commerce. We define two user defined property descriptors. The first property descriptor is named data. It returns a java.io.File object when you call getPropertyValue("data") on one of the media-external items. The path name for this File object is computed by concatenating the value of the url property of the mediaexternal item with the value of the pathPrefix attribute below, ./docs. Thus the path is of the form:
./docs/<value of the url property>
The second user defined property computes a MIME type from the url property. It returns a MIME type string like "text/html" from the value of the url property. It uses a MimeTyper component to convert the suffix to a MIME type.
<!-- Media, which is stored on the external file system --> <item-descriptor name="media-external" display-name="Media - External" super-type="media" sub-type-value="external" item-cache-size="1000" query-cache-size="1000" version-property="version" id-space-name="media" content-property="data"> <table name="dcs_media_ext" type="auxiliary" id-column-names="media_id"> <property name="url" data-type="string" column-name="url" required="true"/> </table> <property name="data" property-type="atg.repository.FilePropertyDescriptor" writable="false" queryable="false"> <attribute name="pathNameProperty" value="url"/> <attribute name="pathPrefix" value="./docs"/> </property> <property name="mimeType" property-type="atg.repository.MimeTyperPropertyDescriptor" data-type="String" writable="false" queryable="false"> <attribute name="identifier" value="url"/> </property> </item-descriptor>
A new property type is defined by implementing a sub-class of the atg.repository.RepositoryPropertyDescriptor class. In this class, you can define values for the readable, writable, and queryable properties. They will also have the following additional methods that will typically be overridden by a user-defined property type:
84
7 - SQL Repository Item Properties
//------------------------------------//------------------------------------/** * This method is called to retrieve a read-only value for this property. * * Once a repository has computed the value it would like to return for * this property, this property descriptor gets a chance to modify it * based on how the property is defined. For example, if null is to * be returned, we return the default value. */ public Object getPropertyValue(RepositoryItemImpl pItem, Object pValue); //------------------------------------/** * Sets the property of this type for the item descriptor provided. */ public void setPropertyValue(RepositoryItemImpl pItem, Object pValue);
You can register user defined property types in a static registry so that they can be defined with a simple name, like Dynamo Server Page tag converters. List your user defined properties in the userPropertyDescriptors property of the GSARepository component.
Property Fetching
Normally, when a repository item is loaded from the database, properties in each table are loaded at the same time. By default, all of the primary table properties of a repository item are loaded when getItem is called for the first time on the item. You may need to modify this default property fetching behavior. For some applications, the database activity required in loading all of the primary table properties may result in performance suffering unnecessarily. For example, an application may want the SQL repository to load a large GIF property only if it is specifically asked for. This is referred to as lazy evaluation. On the other end of the spectrum, you may need to load properties from different tables immediately. For example, an application may want to always load a users last name whenever a profile is read from the database. For lack of a better term, we will refer to this as prefetching. Finally, some applications want to group properties so that when one value is requested, all values in this group are loaded. An example of caching groups would be loading a zip code and state code whenever a street address is loaded. You can achieve a finer level of control over property loading using cache groups in your repository definition. By default, the cache group of a property is the same name as the table that the property is defined in. You can set the cache group of a property with the group attribute in the propertys definition in the repository definition file. All properties with the same group attribute are fetched whenever any member of the group is fetched. Only those properties that are in the same cache group as the repository ID (or, if there is no ID property, then all the properties in the primary table) are loaded when getItem is called for the first time on an item. While generally you would define a cache group using the group
85
7 - SQL Repository Item Properties
<property name="address1" group="address"/> <property name="city" group="address"/> <property name="state" group="address"/> <property name="zip" group="address"/>
attribute in the propertys definition in the repository definition file, you can also define a cache group using the setGroup method of atg.adapter.gsa.GSAPropertyDescriptor. For example, an address might be composed of several properties, like this:
Using the group="address" attribute assures that the whole address is loaded whenever one element of the address is accessed, even if the properties are stored on different database tables. So, if you call getPropertyValue for the city property, the address1, state, and zip properties would be loaded as well. If you want to assure that only the repository ID is returned and none of the repository items other properties, you can isolate the repository ID in its own cache group:
<item-descriptor name="user" default="true"> <table name="usr_tbl" type="primary" id-column-names="id"> <property name="id" data-type="string" group="id"/> group="id" <property name="name" data-type="string" group="info"/> <property name="age" data-type="int" group="info"/> </table> </item-descriptor>
useSetBinaryStream
If useSetBinaryStream is set to true, then the SQL repository always uses setBinaryStream() instead of setBytes() in prepared statements. The setBinaryStream() is required for large byte arrays in some JDBC drivers. If useSetUnicodeStream is set to true, then the SQL repository always uses setUnicodeStream() instead of setString() in prepared statements. The setUnicodeStream() method is required for large Strings in some JDBC drivers. Setting useSetUnicodeStream="true" is recommended if you are using Oracle with internationalized content, but is not recommended if you do not have internationalized content in your database. Note that if you are using MS SQL Server, you must set useSetUnicodeStream to false.
useSetUnicodeStream
86
7 - SQL Repository Item Properties
If useSetAsciiStream is set to true, then the SQL repository always uses setAsciiStream() instead of setString() in prepared statements. You can useSetAsciiStream instead of useSetUnicodeStream, but then you lose the ability to handle internationalized values in the database. If useSetObject is set to true, then the SQL repository always uses setObject() instead of setInt(), setFloat(), setDouble(), or setString() in prepared statements.
useSetAsciiStream
useSetObject
87
7 - SQL Repository Item Properties
88
7 - SQL Repository Item Properties
The SQL repository adds a number of features to the basic query architecture of the Repository API described in the Repository Queries chapter. This chapter discusses the following topics: Repository Filtering Overriding RQL-Generated SQL Parameterized Queries Named Queries Text Search Queries Wildcards in Queries Not Queries and Null Values Outer Joins Table Ownership Issues Unsupported Queries in the SQL Repository
Repository Filtering
The SQL repository allows you to filter database read operations. For example, you might want a database lookup always to return only items whose activeFlag property is true. You can filter the repository by defining a repository query that specifies the filter criteria you want and associating it with the appropriate item descriptor. The filter is then automatically applied to all of the following operations:
Repository.getItem() Repository.getItems() MutableRepository.getItemForUpdate() MutableRepository.getItemsForUpdate() RepositoryView.executeQuery() RepositoryView.executeCountQuery()
There are three ways of defining a repository filter: Use the <rql-filter> tag in the definition file for an item descriptor. Set the filterQuery property of the item descriptor to a Query object.
89
8 - SQL Repository Queries
Set the rqlFilterString property of the item descriptor to an RQL string, which will be compiled into the Query object that defines the filter.
In most cases, the first method, using the <rql-filter> tag, is easiest and preferable.
<item-descriptor name="article"> <rql-filter> <rql>name starts with "n"</rql> </rql-filter> <table name="article" id-column-names="article_id"> <property name="name"/> <property name="date"/> </table> </item-descriptor>
This setting causes queries and item lookups for this item descriptor to return only items whose name property starts with n. The SQL repository issues SQL in the form of an extra WHERE clause condition to implement filtering so any given query or item lookup should be no slower with a reasonable filter tacked on. You can also use RQL substitution parameters in your filter query. For example:
<item-descriptor name="article"> <rql-filter> <rql>name starts with ?0 or availabilityDate < ?1</rql> <param value="n"></param> <param bean="/myApp/IssueDate"></param> </rql-filter> <table name="article" id-column-names="article_id"> <property name="name"/> <property name="availabilityDate" data-type="timestamp"/> </table> </item-descriptor>
In this second example, the RQL parameters are substituted into the query. The first parameter is a simple constant value. Typically it is not necessarily to substitute constant values as they can be inlined in the RQL query string. The second parameter is a Nucleus component. If an object of type atg.service.util.CurrentDate is used as a parameter (as in this example), the filtering logic will call getTimeAsTimeStamp() on that object and use that as the value of the parameter. This allows you to have a CurrentDate service used in a filter. Also note, as in this example, that the less than symbol (<) is a markup character and must be escaped in your XML file as <. For information about using RQL, see the Repository Query Language section in the Repository Queries chapter.
90
8 - SQL Repository Queries
potentially costly joins will be required for every item access. This could dramatically degrade repository performance. It is suggested that the filter not be changed too often. Each query executed by the SQL repository is AND-ed to the filter query before being executed (or looked up in the cache). If the filter or the RQL filter parameters are changed too often, the effectiveness of the query cache will be diminished. Changing the filter once or twice a day should not cause a problem.
GSARepository repo = (GSARepository)request.resolveName("/examples/TestRepository"); RepositoryView view = repo.getView("canard"); Object params[] = new Object[4]; params[0] = new Integer (25); params[1] = new Integer (75); params[2] = "french"; params[3] = "greek"; Builder builder = (Builder)view.getQueryBuilder(); String str = "SELECT * FROM usr_tbl WHERE (age_col > ?0 AND age_col < ?1 AND EXISTS (SELECT * from subjects_tbl where id = usr_tbl.id AND subject IN (?2, ?3)))"; RepositoryItem[] items = view.executeQuery (builder.createSqlPassthroughQuery(str, params));
91
8 - SQL Repository Queries
if (items == null) out.println(" Is null."); else{ for (int i = 0; i < items.length; i++){ out.println(items[i].toString() + "<br>"); } }
Parameterized Queries
A parameterized query is a Repository Query that is incomplete (that is, missing some data) when it is created, and then is supplied with that data when the query is executed. This is very similar to a PreparedStatement in JDBC. Parameterized queries are supported only in the SQL repository. You can substitute a parameter only for constant values, and not column specifications in a Repository Query. The use of parameters in a Query enables developers to reuse a single instance of that Query over and over again, supplying different parameter values at execution time. For example, if your goal is to create a Query like this:
select id from dps_user where first_name = 'keith'
only the value 'keith' can be parameterized; the column name first_name cannot. Sorting and range information also cannot be parameterized, so only constraints can use parameters. Furthermore, parameterized queries are used only in queries against the database; you cannot use parameterized queries against transient properties or in cases where the SQL repository components useDatabaseQueries property is set to false.
Each element in the pParameterValues array corresponds to a parameter in the given Query. pParameterValues[0] corresponds to the first parameter in the Query, pParameterValues[1] corresponds to the second parameter in the Query, and so on. When you create a Query, you need to remember how many parameters the Query has, and what order they come in, especially when making compound queries such as AND and OR. You can use the Query.getQueryRepresentation() method to obtain a string of the query representation including all parameter locations.
92
8 - SQL Repository Queries
In addition, the atg.repository.QueryBuilder interface is extended by an interface named atg.repository.ParameterSupportQueryBuilder. This interface adds a single method to create a parameter QueryExpression that can be used in queries created by the ParameterSupportQueryBuilder:
public QueryExpression createParameterQueryExpression() throws RepositoryException
The atg.adapter.gsa.GSAView class used by the SQL repository implements the atg.repository.ParameterSupportView interface, and the atg.adapter.gsa.query.Builder class implements the atg.repository.ParameterSupportQueryBuilder interface. This makes parameterized queries available in the SQL repository.
Parameter Locations Either expression or both expressions can be a parameter. Only the pattern argument can be a parameter. Only the searchstring argument can be a parameter. Only the collection argument (the first argument) can be a parameter, and it must be multi-valued. Either expression or both expressions can be a parameter (and they must be multi-valued in all cases).
createIncludesAnyQuery
93
8 - SQL Repository Queries
Then, you want to change that to
firstName = 'Phil'
// Get the repository through our made up getRepository( ) call Repository rep = getRepository(); RepositoryItemDescriptor desc = rep.getItemDescriptor("user"); RepositoryView view = desc.getRepositoryView(); QueryBuilder qb = view.getQueryBuilder(); // Build our first Query // firstName = 'Jerry' QueryExpression firstNameProp = qb.createPropertyQueryExpression("firstName"); QueryExpression jerryValue = qb.createConstantQueryExpression (new String("Jerry")); Query firstNameQuery = qb.createComparisonQuery(firstNameProp, jerryValue, QueryBuilder.EQUALS); // Execute our first Query RepositoryItem[] jerryItems = view.executeQuery(firstNameQuery); // Set up our second Query now QueryExpression philValue = qb.createConstantQueryExpression(new String("Phil")); firstNameQuery = qb.createComparisonQuery(firstNameProp, philValue, QueryBuilder.EQUALS); // Execute our second Query RepositoryItem[] philItems = view.executeQuery(firstNameQuery);
// Get the repository through our made up getRepository( ) call Repository rep = getRepository(); RepositoryItemDescriptor desc = rep.getItemDescriptor("user"); // Our RepositoryView is a ParameterSupportView this time, so we know it supports // parameters in Queries // Note this assumes we have advanced knowledge that this view is an instance of // a ParameterSupportView ParameterSupportView view = (ParameterSupportView)desc.getRepositoryView(); QueryBuilder qb = view.getQueryBuilder();
94
8 - SQL Repository Queries
// Builder our first Query up // firstName = 'Jerry' QueryExpression firstNameProp = qb.createPropertyQueryExpression("firstName"); QueryExpression parameterValue = qb.createParameterQueryExpression(); Query firstNameQuery = qb.createComparisonQuery (firstNameProp, parameterValue, QueryBuilder.EQUALS); // Execute our first Query Object[] args = new Object[1]; args[0] = new String("Jerry"); RepositoryItem[] jerryItems = view.executeQuery(firstNameQuery, args); // Set up our second Query now args[0] = new String("Phil"); RepositoryItem[] philItems = view.executeQuery(firstNameQuery, args);
In the first example, we have to create a new constant QueryExpression if we want to change the name from Jerry to Phil. This means we must also create a new instance of a comparison Query to use the new QueryExpression. In the second example, we can increase efficiency by creating just one Query object, and changing the value of the name we want in an Object array that is passed to the executeQuery method. This also allows you to cache a Query in your internal application (above the query cache layer), and pass in varying parameter values at execution time.
Named Queries
Named queries are a useful way to create reusable queries in the Repository Query API, so you dont have to build a new query object each time. In addition, named queries allow you to specify precisely the SQL statements to be used by a query, rather than using the generalized SQL generated by the SQL repository. A named query is a Repository Query or SQL statement that can be defined by the developer and be invoked by some String name. The basic Query API requires you to create a Query by calling several methods to create QueryExpressions, and then combine those QueryExpressions into Query objects. When you use named queries, once you have defined the query, you need to call only one method to execute the desired Query. This can reduce object creation and simplify custom code. Currently, named queries are supported in the SQL repository and the Integration Repository.
95
8 - SQL Repository Queries
Defining and Accessing Named Queries in Java Code
lastName ENDS WITH "son"
You can also create a named query using Java code, as described in the next section.
import atg.repository.*; ... public void sampleQueryTest() { // Get a hold of your repository somehow Repository rep = getRepository(); RepositoryItemDescriptor desc = rep.getItemDescriptor("user"); RepositoryView view = desc.getRepositoryView(); QueryBuilder builder = view.getQueryBuilder(); try { // Create your query QueryExpression prop = builder.createPropertyQueryExpression("firstName"); QueryExpression constant = builder.createConstantQueryExpression("Adam"); Query query = builder.createComparisonQuery(prop, constant, QueryBuilder.EQUALS); //See if we have a NamedQueryView to work with... if(view instanceof NamedQueryView) { NamedQueryView nameView = (NamedQueryView)view; String queryName = "firstNameQuery";
96
8 - SQL Repository Queries
// Create our named query nameView.createNamedQuery(queryName, query); // And execute it (normally you wouldn't do this all in the same method // since the purpose is really to have access to the Query for later) Query namedQuery = nameView.getNamedQuery(queryName); RepositoryItem[] results = nameView.executeQuery(namedQuery); } } catch(RepositoryException exc) { System.out.println(exc.toString()) } }
<item-descriptor name=...> ... <named-query> <sql-query> <query-name>myQuery</query-name> <sql> select id,first_name,last_name from dps_user WHERE login=? </sql> <returns>id,firstName,lastName</returns> <input-parameter-types>java.lang.String</input-parameter-types> <dependencies>login</dependencies>
97
8 - SQL Repository Queries
</sql-query> </named-query> ... </item-descriptor>
There are several things to note here. First, the SQL string contained in the <sql> element must contain the actual SQL column and table names, not the property names defined by the repository. Second, the SQL string must include the ID from the primary table in the item-descriptor. If the ID for an item descriptor is defined as a composite ID (using two or more columns), then all of the columns that make up the composite ID must be selected. The <returns> element is a comma-separated list of Repository property names that are returned by this query. These are property names so we can know the type of the column when were reading values from the returned ResultSet. The <dependencies> element indicates which properties this query depends on. If any of the properties in the <dependencies> element are changed, then this query should be removed from the query cache. This is typically, if not always, the properties referenced in the WHERE clause of the SQL statement, if any. The <input-parameter-types> element is a comma-separated list of class names that any parameters in the Query must be an instance of. There should be one value for each parameter, even if the classes used are similar; thus, if there are three String parameters in your Query, the <input-parameter-types> element must contain "java.lang.String, java.lang.String, java.lang.String". This tag serves two purposes. It is used primarily for type checking when a Query is executed. It is also used to indicate how many parameters there are in the Query. The properties that are used in the <returns> element must be: defined as readable in the repository, persistent properties defined in a table tag and not transient properties, single-valued and not multi-valued properties (unless they are composite IDs).
The property columns are returned in the order they appear in the <returns> tag. Since users do not have to define an explicit RepositoryPropertyDescriptor for the ID property, the ID property can be omitted from the <returns> element, but it still must exist in the SQL statement as the first column(s) selected. The <returns> element is optional. Select statements do not have to return any other properties besides the ID. Again, the ID property should only appear in the <returns> element if that item descriptor explicitly defines it with the <property> element. Otherwise, the value in the id-column attribute of the <item-descriptor> tag is used as the name of the ID column.
Stored Procedures
Stored procedures can also be used as a mechanism to return complex query results. Using a named query to invoke a stored procedure is very similar to a direct SQL named query, with one minor addition. The <sql> tag for a stored procedure uses an attribute named stored-procedure. When using a storedprocedure, this attribute must be set to true. Here is an example of how you might define a stored
98
8 - SQL Repository Queries
procedure in a repository definition file. Note that the values used in the <returns> and <dependencies> tags in this example are dependent upon what the stored procedure returns and defines constraints for.
<item-descriptor name=...> ... <named-query> <sql-query query-name="myQuery"> <sql stored-procedure="true"> { call myStoredProcedure (?, ?) } </sql> <returns>id,firstName,lastName<returns> <input-parameter-types>java.lang.String,java.lang.String </input-parameter-types> <dependencies>login</dependencies> </sql-query> </named-query> </item-descriptor>
The syntax for calling a stored procedure is different from executing a SQL statement. The body of the <sql> tag for a stored procedure must use the syntax required by java.sql.CallableStatement. There are two formats typically used in a CallableStatement, one for procedures, which dont have an explicit return value, and one for functions, which do. Unless you are using Oracle, stored procedures in the SQL repository must use the procedure form, as in the above example. The enclosing curly braces are required. The question marks indicate parameters. In stored procedures, parameters can be IN, OUT and INOUT, indicating that the values go in the database, come out, or both go in and come out. The SQL repository supports only IN parameters. Any OUT parameters or INOUT parameters will not function correctly, except in the case of Oracle stored procedures. Any stored procedure used must return a java.sql.ResultSet. Most JDBC drivers will do so by default, but there are exceptions. Oracle, for example, requires some special tuning, as described in the next section.
Note the leading ? =, which indicates that a value is returned. You might consider wrapping existing stored procedures with procedures that can format the results in a manner that the ATG SQL repository requires.
99
8 - SQL Repository Queries
Named Queries and Item Inheritance Named Query API
NamedQueryView Interface
Item descriptor subtypes inherit the named queries defined for their parent item types. For example, suppose, as in the Quincy Funds demo, you have defined a user item type with investor and broker subtypes. If the user item descriptor defines a named query called getUsersByLogin, then the investor and broker subtypes will also have access to that same named query.
An interface named atg.repository.NamedQueryView provides methods to create and access named queries. This interface extends atg.repository.RepositoryView. It includes the following methods:
public void createNamedQuery(String pQueryName, Query pQuery) Creates an association in the RepositoryView between a name and a Query object. After this
association is created, you can call getNamedQuery to get the Query object to be used for execution. If this method is called with a pQueryName that has already been assigned a Query, then the existing Query will be overwritten with the new Query.
public Query getNamedQuery(String pQueryName) Gets the Query object associated with the given name. If no such entry has been created for the given
Returns the names of all Named Queries that this RepositoryView knows about, or null if there are none.
public QueryDescriptor getQueryDescriptor(String pQueryName)
Returns a QueryDescriptor object that describes aspects of the requested NamedQuery. If there is no named query by the given name, null is returned.
public String getQueryName(Query pQuery)
Returns the name of the given Query, if any exists. Otherwise, null is returned.
QueryDescriptor
The atg.repository.QueryDescriptor interface defines methods for an object that describes a Repository Query and associates a Repository Query object with a user-defined String name. It defines the following methods:
public Query getQuery() public String getQueryName()
The atg.repository.query.QueryDescriptorImpl class is the base implementation of the QueryDescriptor interface and the atg.adapter.gsa.query.GSAQueryDescriptor class is the SQL repositorys subclass of QueryDescriptorImpl.
RepositoryViewImpl
The atg.repository.RepositoryViewImpl class implements NamedQueryView and supports the use of named queries.
100
8 - SQL Repository Queries
directory.
package atg.adapter.gsa.sample; import atg.repository.*; public class FullTextQuery extends Harness { //------------------------------------/** Class version string */ public static final String CLASS_VERSION = "$Id: FullTextQuery.java,v 1.1 2000/03/20 19:39:31 mstewart Exp $"; //------------------------------------/** * Run our sample. * @param pRepository repository to use * @exception RepositoryException if there is repository trouble **/ public void go(Repository pRepository) throws RepositoryException { // print header pln("### Running Sample Full-Text Query ###"); pln(CLASS_VERSION); pln(""); /* ** This example demonstrates how do perform some simple full-text repository ** queries. In the repository API all queries are performed using Query ** or QueryExpression objects. A QueryExpression is a building block you ** can use to create simple or complex queries. A Query is a repository ** query that can be executed. A Query can also be used as a building ** block to create more complicated queries. Here we perform a simple ** query to find user repository items whose story property ** includes text in which the word 'dog' appears within 10 words of the
101
8 - SQL Repository Queries
** word 'cat'. */ // QueryBuilder and RepositoryView.
// queries are created using QueryBuilders and executed by // RepositoryViews. A Query is defined in the context of a specific item // descriptor and thus must be built and executed with the right RepositoryItemDescriptor userDesc = pRepository.getItemDescriptor("user"); RepositoryView userView = userDesc.getRepositoryView(); QueryBuilder userBuilder = userView.getQueryBuilder(); // create a QueryExpression that represents the property, story QueryExpression comment = userBuilder.createPropertyQueryExpression("story"); // create a QueryExpression that represents a search expression // using the NEAR operator. QueryExpression dogNearCat = userBuilder.createConstantQueryExpression("NEAR((dog, cat), 10)"); // define the format being used by the search expression // appropriate to the database being used. This assumes an Oracle // database with the interMedia/Context full-text search option // installed. QueryExpression format = userBuilder.createConstantQueryExpression("ORACLE_CONTEXT"); // pick a minimum required score that the results must meet or exceed // in order to be returned by the full-text search engine. // See your search engine vendor's docs for more information on the meaning // and use of the score value. QueryExpression minScore = userBuilder.createConstantQueryExpression(new Integer(1));
// now we build our query: comment contains 'dog' within 10 words of 'cat' Query dogTenWordsNearCat = userBuilder.createTextSearchQuery(comment, dogNearCat, format, minScore); // finally, execute the query and get the results RepositoryItem[] answer = userView.executeQuery(dogTenWordsNearCat); pln("running query: story contains 'dog' within 10 words of 'cat' "); if (answer == null) { pln("no items were found"); } else { for (int i=0; i<answer.length; i++)
102
8 - SQL Repository Queries
pln("id: " + answer[i].getRepositoryId()); } } //------------------------------------/** * Main routine. This example uses no command line arguments **/ public static void main(String[] pArgs) throws Exception { runParser(FullTextQuery.class.getName(), pArgs); } } // end of class FullTextQuery
You can specify what properties a text search query should search, using the text-search-properties attribute in the <item-descriptor> tag that defines an item type. For example, the following value indicates that a text search should examine the keywords and content properties for matches:
<item-descriptor name="newsItems text-search-properties="keywords,content"> ...
The SQL repository will then convert text search queries into CONTAINS pattern match queries, which are implemented using the SQL LIKE operator. Simulated text search queries are useful for demos and standalone development when one wants to put in place the createTextSearchQuery() API calls without having to set up a text search engine. However, simulated text queries are extremely inefficient and are not supported for production systems. A simulated text search query using LIKE will typically cause a table scan, so you wouldnt want to use simulated queries in production.
Wildcards in Queries
Databases often treat % and _ as wildcard characters. In pattern-match queries in the SQL repository (such as CONTAINS, STARTS WITH, or ENDS WITH), we assume that a query that includes % or _ was intended as
103
8 - SQL Repository Queries
a literal search including those characters, and was not intended to include wildcard characters. The query generated therefore uses an escape character in front of the characters % and _ in pattern-match queries. An exception is where we use a pattern-match query to simulate a text search query, since in that case we want to allow wildcards to be passed through. You can disable this behavior by setting the escapeWildcards property of the SQL repository component to false. The escape character is \ (backslash) by default. You can set a different escape character using the wildcardEscapeCharacter property of the SQL repository component.
For example, if your query is balance = 101 or balance < 101, the query will (unsurprisingly) not return an item whose balance property is null. However, if your query is balance != 101, the query will still not return an item whose balance property is null. If you wish your query to return items whose queried property is null, you may use an IS NULL query, or an IS NULL clause as part of an OR query, for example:
(balance != 101) OR (balance IS NULL)
Outer Joins
By default, the SQL repository uses outer joins in queries that involve auxiliary tables. Different database vendors use different syntax to create outer joins. ATG 6 automatically sets the outerJoinSupport property of the GSARepository component to specify the appropriate type of outer join to be used by the SQL repository. You can also configure this property manually, using the following values:
Value
Database Vendor
Description
sql92 jdbc
Use FROM tablex x LEFT OUTER JOIN tabley y ON x.id = y.id Similar to sql92 but uses JDBC escape syntax {oj ... } to tell the JDBC driver to convert to native join syntax. Oracle
x += y
plus-equals
104
8 - SQL Repository Queries
star-equals
informix none
The metaDataSchemaPattern attribute should be set to the name of the database account that was used to create the tables that underlie the repository. The metaDataCatalogName attribute represents a catalog name. If the user does not own the table to be used by the repository, this attribute can be used once during initialization of the repository in a call to determine the column types. If the user does not own the table used by the repository, you can use the tablePrefix attribute to construct a qualified table name. This attribute is not used during the initial metadata query, but if present will be prepended to the table name when inserts or updates are made. For example:
<attribute name="tablePrefix" value="myPrefix."/>
For instance, the following snippet sets dps_user to use testing2 as the schema name for the metadata call. The string testing2. will be prepended to the table name for all other queries.
<gsa-template> ... <table name="dps_user" type="primary" id-column-name="id"> <attribute name="tablePrefix" value="testing2."/> <attribute name="metaDataSchemaPattern" value="testing2"/> ...
105
8 - SQL Repository Queries
In using these attributes, make sure you use the exact case (upper, lower, mixed) that your database uses to store object identifiers. For example, Oracle stores its identifiers in uppercase. So, you would set metaDataSchemaPattern to DYNAMO instead of Dynamo. See the Javadoc for java.sql.DatabaseMetaData.getColumns() for more information. See also Database Loggers in the Logging and Data Collection chapter of the ATG 6 Dynamo Programming Guide for more information about these attributes. In addition, you can set the metaDataSchemaPattern and metaDataCatalogName as properties in the SQL repository component. If they are set as a property, then that setting will be used as a default for every table in the repository that does not have the corresponding attribute explicitly set in the repository definition file. The attribute in the table tag takes precedence, in other words.
106
8 - SQL Repository Queries
You can use Java resource bundles to make it easier to localize a SQL repository definition. By using resources for repository item properties used in the ATG Control Centers Repository Editors, you can display labels and values suitable for the Repository Editor users locale. The SQL repository localization feature enables you to use resource files to localize: the values of the display-name and description of both item descriptors and properties the category of properties, and the strings used for representing values of enumerated property types.
You can use the <attribute> tag to set the resource bundle at either the property, table, or item descriptor level. A property will use its own resourceBundle attribute if it is set. If not, it looks for a resourceBundle attribute set in its <table> tag, and then for a resourceBundle attribute set in its <item-descriptor> tag. For example:
<item-descriptor name="user" ....> <attribute name="resourceBundle" value="atg.userprofiling.ProfileResources"/> ...
If you are using xml-combine="append" to add properties to an item descriptor defined in a different configuration layer, do not set the resourceBundle attribute in the item descriptor , as it will overwrite the setting of resourceBundle in the other configuration level. Set the resourceBundle at either the table or property level instead.
107
9 - Localizing SQL Repository Definitions
Localizing Properties
Standard Label Attribute display-name description category Localizable Label Attribute display-name-resource description-resource category-resource
To localize labels used in a Repository Editor, use the localizable attributes, as follows:
For example, to localize the display-name, use the display-name-resource attribute instead of the
display-name attribute: <item-descriptor name="user" ... display-name-resource="itemDescriptorUser"> <attribute name="resourceBundle" value="atg.userprofiling.UserProfileTemplateResources"/>
Then, for each locale you want to support, create resource bundle properties files for each repository definition. Each resource bundle consists of a list of keys defined in the resource label attributes, with the localized value. The UserProfileTemplateResources.properties resource bundle referred to in the preceding example would contain this entry:
itemDescriptorUser=User
When you specify a default, use the resource name as the value, such as:
<property name="emailStatus" ... data-type="enumerated" default="emailStatusUnknown" ... > <attribute name="useCodeForValue" value="false"/> <option resource="emailStatusUnknown" code="0"/>
108
9 - Localizing SQL Repository Definitions
Use caution when localizing the strings used for enumerated types. Remember that if you have useCodeForValue set to false, calling getPropertyValue will not return the localized property value. If you want to display the localized value on a page, you need to include the localized string in your page, using a Switch servlet bean to choose the proper value. For more information about resource bundles and localization, see the Internationalizing a Dynamo Web Site chapter in the ATG 6 Dynamo Programming Guide.
109
9 - Localizing SQL Repository Definitions
110
9 - Localizing SQL Repository Definitions
Thoughtful design of database access is a key to achieving acceptable performance in many Dynamo applications. You want to minimize how often your application needs to access the database, while still maintaining data integrity. An intelligent caching strategy is central to achieving these goals. This chapter includes the following topics that describe how to use caches with a SQL repository: Item Caches and Query Caches Cache Modes No Caching Locked Caching Distributed Caching Cache Configuration Cache Timeout Cache Statistics Cache Loading Cache Flushing Caching and Item Descriptor Inheritance
If query caching is enabled, the first time this query is issued, the result set is retrieved from the database and stored in the query cache. Then, the next time this same query is issued, the SQL repository can retrieve the result set from the cache, rather than needing to access the database.
111
10 - SQL Repository Caching
The item caches hold the values of repository items. Repository queries are performed in two passes, using two separate SELECT statements. The first statement gathers the IDs of the repository items that match that query. The SQL repository then examines the result set from the first SELECT statement and finds any items that already exist in the item cache. A second SELECT statement retrieves from the database any items that are not in the item cache. Query caching is turned off by default. If items in your repository are updated frequently, or if repeated queries are not common, you may not receive enough benefit from a query cache to justify the minor amount of overhead in maintaining the cache. See Cache Configuration for more information about enabling the query cache and setting the sizes of the item and query caches.
Cache Invalidation
The SQL repository computes which properties are used in each query. When any of these properties are modified, the affected queries are invalidated. In addition, whenever an item of a given item type is added to or removed from the repository, all queries are invalidated in the query cache for that item descriptor. Item cache entries are invalidated when that item is updated. When a set of properties of a repository item belonging to an item descriptor have been modified, the item descriptor fires a PropertiesChangedEvent (atg.repository.PropertiesChangedEvent). If you need specific notification when an items cache entry is invalidated, you can listen for these events. The PropertyChangedEvent class has a type property that indicates the cause of the event. The value of
type can be one of the following:
int Code 0 1 2 3
Description Properties of an item have been changed in this transaction. The item has been removed in this transaction. The item was newly added to the database. Some application code called the removeItemFromCache method on this item.
Property Name
item
Property Type
RepositoryItem
Description The item that was changed. This is set to null if the item that was modified is not currently in the cache. In this case, look at the repositoryId property for the identity of the item that was changed.
112
10 - SQL Repository Caching
String
repositoryId
This property can be set in place of the item property when the item itself is not in the cache and the origin of the event was in another server. The item descriptor name of the item modified in this transaction. A map that has a
RepositoryPropertyDescriptor as its key and
itemDescriptorName
String
properties
Map
the new property value as the value. If this value is null, specific detail about which properties were changed in this transaction is either not applicable or not available. For UPDATE events, a null properties map indicates that all properties in the item may have changed.
How and when cache invalidation is communicated across multiple Dynamo servers depends on the cache mode you select, as described in the Cache Modes section. You can also flush caches yourself, as described in the Cache Flushing section.
Cache Expiration
In addition to invalidating cache entries actively when repository items change, you can assure data integrity by configuring your caches to expire after the lapse of a specified time. See the Cache Configuration: Cache Timeout section.
Cache Modes
The SQL repository implements several different modes of caching. Which mode you choose depends on the nature of your Dynamo application. You can set the default caching mode on each item descriptor in a repository. Each propertys definition can override the cache mode of its item-descriptor. The caching modes implemented in the SQL repository are:
113
10 - SQL Repository Caching
No Caching (disabled)
Simple Caching (caching is handled separately in each VM, with no invalidation events to synchronize updates from different server instances) Locked Caching (read and write locks are used to synchronize the caches) Distributed Caching (caching with cache invalidation events)
By default, the SQL repository uses simple caching. To enable a different cache mode for an item descriptor, set the cache-mode attribute in the <item-descriptor> tag of the repository definition file:
<item-descriptor name="profile" cache-mode="locked">
<item-descriptor name="profile" cache-mode="locked"> <table name="dps_user"> <property name="password" cache-mode="disabled"> ... </table> ...
You can also set the property cache mode to inherit. This specifies that the default cache mode should be used. This value might be useful if a propertys cache mode has been set to disabled at one point in the CONFIGPATH and you want to enable the default cache mode for the property at a later point in the CONFIGPATH.
114
10 - SQL Repository Caching
The SQL Repository then will not store the repository items referred to, but will instead store only the repository IDs of the items referred to in the item cache.
No Caching
Disabled mode disables caching of items across transactions. Values for this item will be cached within the current transaction so that you see a consistent view of the item in any given transaction. When properties are not cached, they are loaded directly into a store that is visible for each transaction. If you call getPropertyValue() on the same property twice in the same transaction, the property value will not be re-queried. You can also disable caching for an individual property within an item-descriptor by setting the cachemode="disabled" attribute on the property itself. All other cache modes must be set at the item descriptor level, however, and cannot be set on a per-property basis. Disabled caching should be used with great caution, because it will result in database access for every page that accesses an item of this type. This potentially has a severe impact on performance. Caching should generally be disabled when there is a possibility that the underlying data will be changed by a non-Dynamo repository application. For instance, if you have an on-line banking application, and the same data is accessed by other applications in addition to Dynamo, you may want to turn off caching for displaying the users account balances. You could, in some circumstances, configure your integration so that when the data is modified by an external application, the repository cache is invalidated.
Simple Caching
In the simple caching mode, each VM maintains its own cache of the information. Any changes made by other servers will not be recognized in this server until the cache entry is manually or automatically invalidated. This mode is suitable for read-only repositories (which includes repositories like product catalogs where all changes are made on a staging server and not a live server) or architectures where you can guarantee that only one server will handle a particular repository item. You can use the item-cache-timeout attribute together with simple cache mode for caching user data associated with a particular session. If only one user at a time is accessing an item type, this approach can generally prevent access to stale data from a cache without the overhead imposed by locked mode caching. See Cache Timeout in this section for more information.
115
10 - SQL Repository Caching
Class
atg.repository.RepositoryImpl atg.repository.ItemDescriptorImpl
You can also use simple cache mode in cases where it is practical to invalidate caches manually by calling these methods:
Method
invalidateCaches removeItemFromCache invalidateItemCache invalidateCaches
atg.repository.RepositoryView
invalidateQueryCache
See the Cache Flushing section of this chapter, which also describes the Cache Invalidator service, which can be used to control cache invalidation manually.
Locked Caching
For some item descriptors, particularly those that represent user data, it may be most efficient to use read and write locks to manage access to your data. At any given time, you can have one server that is caching or modifying an item (or no servers modifying the item and zero or more servers caching the item, for read-only access). This avoids the need for you to broadcast cache invalidation events to all servers each time you make a change to the item. Locked mode may be the best choice for item types that are modified frequently during runtime. It also does the best job of guaranteeing that only up-to-date data is seen by your application. If an item uses locked mode caching, then the repository consults a central lock server to obtain read or write access for the item. The lock server allows multiple simultaneous readers, but only a single writer. Lock ownership (either read or write) is cached on the server that last requested that type of access to that item. As soon as another server requests a write lock for that item, all servers which currently have ownership are asked to give up lock ownership as soon as all transactions which own that lock release it. When you set the cache-mode attribute to locked in a repository, Dynamo uses read and write locks to control which threads can access an item. (Read locks are also called shared locks and write locks are called exclusive locks.) The exact behavior depends on how you set the isolation level for the item descriptor.
Isolation Levels
To minimize deadlocks when you use the locked cache mode, you can configure the isolation level of an item descriptor. You can use one of three isolation levels:
116
10 - SQL Repository Caching
Description This is the default value. Dynamo gets a read lock on the item when you first call getItem or getPropertyValue for an item in a transaction. If you then try to change the item, you release the read lock and try to acquire a write lock. New property values are visible only once they have been committed. You are not guaranteed that subsequent getPropertyValue calls made in the same transaction return the same value. Dynamo acquires a read lock the first time you call getItem or getPropertyValue for an item in a transaction. When you call getItemForUpdate or setPropertyValue in a transaction, Dynamo attempts to convert the read lock into a write lock. Prevents different transactions from reading an item at the same time either from the same server, or from different servers.
Isolation Level
readCommitted
repeatableRead
serializable
Set the isolationLevel attribute in an <attribute> tag that is a sub-tag of the <item-descriptor> tag. For example:
<item-descriptor name="user" cache-mode="locked"> <attribute name="isolationLevel" value="readCommitted"/> ... </item-descriptor>
Locking Exceptions
When you attempt to get a write lock on an item, the following exceptions may arise: If you get a deadlock when you call getItem or getItemForUpdate, a RepositoryException is thrown. If you are using isolationLevel="repeatableRead" and you are unable to convert a read lock to a write lock (presumably because another transaction was trying to do the same thing at the same time), a ConcurrentUpdateException is thrown. If you are calling a method, such as setPropertyValue, which does not throw a RepositoryException, you get an IllegalArgumentException instead.
117
10 - SQL Repository Caching
Configuring Lock Managers
If you use locked cache mode, you should disable the query cache, since query cache invalidation messages are not distributed between server instances. If query caching is important for the performance of your application, use distributed cache mode. Remember also that you can invalidate caches using a time expiration setting (see the Cache Configuration: Cache Timeout section) or by sending cache invalidation events programmatically (see Cache Flushing).
When you set the cache mode for any item descriptors to locked mode, you must also configure one or more lock servers for that repository. A lock server synchronizes caches among different Dynamo servers so that data integrity is maintained even if an item is modified at the same time by different servers. If you do not configure a lock server and you have configured locked cache mode, caching is disabled.
ClientLockManager
For each SQL repository that contains any item descriptors with cache-mode="locked", you must set the lockManager property of the Repository component to refer to a ClientLockManager. Dynamo comes configured with a default client lock manager, which you can use for most purposes:
lockManager=/atg/dynamo/service/ClientLockManager
When you first install ATG 6, the ClientLockManager component has its useLockServer property set to false, which disables use of the lock server. In order to use locked mode repository caching, you must set this property to true. This setting is included in the ATG 6 liveconfig configuration layer, so you can set the useLockServer property by adding the liveconfig configuration layer to the environment for all your Dynamo servers. See the Common Configuration Changes section in the Configuring ATG 6 chapter of the ATG 6 Installation and Configuration Guide for Dynamo Application Server for information about the liveconfig configuration layer. You must also set the lockServerPort and lockServerAddress properties to match the port and host of your ServerLockManagers components. For example, suppose you have two ServerLockManagers, one running on host tartini and port 9010 and the other running on host corelli and port 9010. You would configure the ClientLockManager like this:
$class=atg.service.lockmanager.ClientLockManager lockServerAddress=tartini,corelli lockServerPort=9010,9010 useLockServer=true
ServerLockManager
You should also configure one or more Dynamo servers to start the /atg/dynamo/service/ServerLockManager on startup. To do this, add the ServerLockManager to the initialServices property of /atg/dynamo/service/Initial in the server-specific configuration layer for the server in which youve chosen to run a ServerLockManager. For example, if you wanted to run the ServerLockManager in a Dynamo server instance named derrida, you could add this properties file at <ATG6dir>/home/servers/derrida/localconfig/atg/dynamo/service/Initial.properties:
#server/derrida #/localconfig/atg/dynamo/service/Initial.properties:
118
10 - SQL Repository Caching
initialServices+=ServerLockManager
ServerLockManager Failover
You can configure more than one ServerLockManager. One ServerLockManager acts as the primary lock server while the other acts as backup. If the primary ServerLockManager fails, then the backup ServerLockManager takes over and clients will begin to send lock requests to the backup. If both ServerLockManagers fail, caching is simply disabled. Under that condition, the site still functions, but just slower since it must access the database more frequently rather than using the cache. The cache mode also switches into disabled mode for all transactions that are unable to obtain the lock. Once a ServerLockManager is restored, caching resumes. A ServerLockManager component is configured with the following properties:
Property port
Description The number of the port this server runs on. This should match the lockServerPort property of the ClientLockManager. The hostname of the other ServerLockManager. This should match the lockServerAddress property of the ClientLockManager. The number of the port the other ServerLockManager runs on. The time in milliseconds to wait before trying to connect to another server. If this server detects that the other lock server has failed, it waits the time in milliseconds specified by this property before becoming the primary lock server.
otherLockServerAddress
otherLockServerPort
otherServerPollInterval
waitTimeBeforeSwitchingFromBackup
For example, if you have two ServerLockManager components named tartini and corelli, each running on port 9010, they could be configured like this:
119
10 - SQL Repository Caching
# corelli:9010 $class=atg.service.lockmanager.ServerLockManager handlerCount=0 port=9010 otherLockServerAddress=tartini otherLockServerPort=9010 otherServerPollInterval=2000 waitTimeBeforeSwitchingFromBackup=10000
It is best if the primary ServerLockManager runs in a Dynamo instance that does not also handle user sessions by running a DrpServer. Not only does this prevent the load on the ServerLockManager from affecting user sessions, but it also lets you stop and restart the DrpServer without restarting the ServerLockManager. If you find that there is enough lock contention on your site that the lock server itself becomes a bottleneck, then you might choose to create separate lock servers for different repositories to distribute the load. Note that in this situation, the lock server will be unable to detect deadlocks that span lock servers. In this situation, you will need a separate ClientLockManager instance in each Dynamo to refer to each ServerLockManager.
The identity of the lock Does this process have a read lock on that key? Note that multiple processes can simultaneously have the read lock but when one process has a read lock, no other process will have the write lock. Does this process have the write lock on that key? Has another process requested this read/write lock? Each process first goes to the ServerLockManager. The ServerLockManager sends out the read/write request to each client that owns the lock for that key when necessary. This is only set when read or write owned are set to true and there are active lock owners that conflict with the lock requested (i.e. a write lock is requested when this process has a thread with active read ownership). This is true if there is an active read or write lock and if the lock was acquired via the ServerLockManager. If the ServerLockManager becomes unavailable, each process will still give out locks but will return the Globally owned flag as false, indicating that this lock is only good for this process. The SQL repository uses this flag to disable caching.
Globally owned
120
10 - SQL Repository Caching
Set to the thread that owns this lock in this process. There can be only one. Set to the threads that own this lock in this process. There can be more than one. The list of threads that are waiting for this lock.
Read/write waiters
The lock table for a ServerLockManager is similar but in that case the lock owners are actually the processes, not the threads within a process.
Distributed Caching
If an item descriptor uses distributed cache mode, then when an item is changed in one JVM, events are sent to all other JVMs which might be caching items of this type. These JVMs are given the type and ID of the item that was modified and then invalidate the cache entry for that item and any queries which might be affected (all queries which depend upon items of that type). Distributed cache mode is suitable for situations where items will be read and cached frequently and modified infrequently. It is not suitable for profile data, which is cached frequently and modified frequently. It is typically used for content that is cached by all JVMs participating in the cluster, but is not modified frequently. An item that is modified more than 50 or 100 times per second on a site is not suitable for distributed cache mode, because the additional network activity created by all the cache invalidation messages may outweigh the benefits of caching. In distributed cache mode, servers that are caching items of a particular item type need to find each other. In the default configuration, this is done automatically. Each Repository component references a GSAEventServer. Each Dynamo instance has one GSAEventServer and each repository in a Dynamo instance references the same GSAEventServer. The GSAEventServer class is an extension of RequestServer and uses TCP communication (ip:port) to send and receive events. Dynamo maintains a table in the same database as the data you are caching. This database table stores the IP address and port number of each server that is running with distributed cache mode for each particular item descriptor type. When a Dynamo server starts up, it adds itself to this table and when it is shutdown cleanly, it removes itself from this table. This table is named das_gsa_subscriber. When a Dynamo server modifies an item with cache-mode="distributed" set on its item descriptor, it sends an event to all of the other servers registered in this table containing the ID of the item and the item descriptor name. If a Dynamo server fails to connect to another server in this table, it prints a warning message and removes the row for that item type for that server. When the Dynamo server sends an event, it also checks to make sure the current EventServer instance is still in the das_gsa_subscriber table. If not, it prints a warning and adds itself back in. In this default configuration, you dont need to do anything to use this particular mode except to ensure that the das_gsa_subscriber table has been properly created in your database. Each Dynamo server listens on a randomly assigned port chosen by the operating system (guaranteed to be available). As
121
10 - SQL Repository Caching
EventServer
described above, the database is used to communicate this information from one server to the next so that each Dynamo server can find all of the other Dynamo servers that cache a particular item type.
Distributed cache mode uses EventServer components (of class atg.adapter.gsa.event.GSAEventServer) to communicate distributed cache mode events with each other. All SQL repository instances in a given JVM use the same EventServer instance. The Nucleus address of the EventServer component is specified by the eventServer property of the GSARepository component. The default EventServer component is /atg/dynamo/server/SQLRepositoryEventServer. When Dynamo starts, the first repository that uses distributed cache mode causes the EventServer instance to be created. At this time, the EventServer opens up a server socket on either a specific port or a randomly assigned port (a randomly assigned port is used if the EventServer uses port=0, which is the default setting). When an item is modified by one Dynamo server, a cache invalidation event is sent to all other servers that might cache that item type. At this time, a persistent connection is created to all other EventServer components in the cluster as listed in the das_gsa_subscriber table. Events are written into this connection. When the EventServer receives a GSAEvent, which contains an item descriptor name, it looks up the
EventListener for that item descriptor name and sends the event to the matching repository. GSARepository implements the GSAEventListener interface. When a GSARepository component
receives a GSAEvent, it removes the item identified by the GSAEvent from its cache. If a connection with a particular server is lost (typically because the server process goes down), an attempt is made to reconnect to that server every 20 seconds until that server returns. You can set this timeout value (specified in milliseconds) with the connectTimeout property of the EventServer component:
connectTimeout=20000
When Dynamo starts up, the EventServer broadcasts a dummy invalidation event. This dummy event is used to form all of the connections that may be required and to ensure that the das_gsa_subscriber table is valid and properly updated for all item types. Also note that distributed cache mode is used by the internally defined item descriptor used to access the das_gsa_subscriber table. This is necessary so that other servers can be made aware dynamically of changes made to this table while the server is running.
122
10 - SQL Repository Caching
tries to send an invalidation event. A server also prints a warning if it is unable to send an event to one of the servers listed in the table. If you are able to modify items with cache-mode="distributed" on each server without seeing any warnings, your das_gsa_subscriber table is configured correctly. If you want to disable the autoUpdateSubscribers mode, you can still use it to populate the default contents of the das_gsa_subscriber table. To do this, you will need to assign available ports for the SQLRepositoryEventServer component on each server in your cluster. Initially, leave the autoUpdateSubscribers property of the GSARepository components set to its default value (true) and start all servers in your cluster. Once all servers have started up, the contents of this table should be populated with unique available port numbers. Now stop all servers, using <ctrl-C> (to prevent them from removing themselves on exit) and then set autoUpdateSubscribers=false for each GSARepository component that uses distributed cache mode. When you start the servers in the cluster this time, they will neither add nor remove themselves from the table.
Cache Configuration
The SQL repository maintains separate item caches and query caches for each item descriptor. This lets you size and flush caches separately for separate item types. You can configure these caches using the following attributes in an <item-descriptor> tag:
123
10 - SQL Repository Caching
item-cache-size query-cache-size
The maximum number of items belonging to this item descriptor to store in the item cache. When the number of items requested exceeds this number, the least recently accessed item is removed from the cache. The maximum number of queries of this item descriptor to store in the query cache. When the number of queries issued against this item descriptor exceeds this number, the least recently used query is removed from the cache. The maximum time in milliseconds that an entry can remain in the item cache before it is refreshed. See Cache Timeout for more information. The maximum time in milliseconds that an entry can remain in the query cache before it is refreshed. See Cache Timeout for more information. The time in milliseconds that an item cache entry can remain unused before it is removed from the cache. See Cache Timeout for more information.
item-expire-timeout
Integer. Default is -1, which means items remain in the cache indefinitely, until otherwise invalidated. Integer. Default is -1, which means items remain in the cache indefinitely, until otherwise invalidated. Integer. Default is -1, which means items remain in the cache indefinitely, until otherwise invalidated.
query-expire-timeout
item-cache-timeout
Note that the query cache and item cache sizes are set by default is 0, effectively disabling the caches. This is an appropriate setting for evaluation and development; for testing and deployment, you need to set these caches to more appropriate values. Assuming you have realistic testing scripts, you may be able to estimate reasonable cache sizes during testing, using the feature described in the Cache Statistics section of this chapter.
124
10 - SQL Repository Caching
Cache Timeout
The SQL repository offers two different approaches to setting a maximum lifetime for items in the cache. Each of these approaches uses a timeout attribute in the item descriptor definition. The first approach uses the item-expire-timeout and query-expire-timeout attributes. Each of these attributes specifies the number of milliseconds that an entry can stay in the item cache or query cache before it needs to be refreshed. For example, if you set item-expire-timeout to 1000 milliseconds, the SQL repository will reload those properties after they have been in the cache for 1 second. If you set queryexpire-timeout to 1000 milliseconds, the SQL repository re-executes the query after that entry has been in the query cache for 1 second. To use the item-expire-timeout and query-expire-timeout attributes, set the value (in milliseconds) in the item-descriptor tag. For example:
<item-descriptor name="order" cache-mode="simple" item-expire-timeout="180000" query-expire-timeout="180000"> ... </item-descriptor>
Under the second approach, the SQL repository expires entries in the item cache when they have been unused for the specified amount of time. In this case, an item remains in the item cache as long as it is being accessed in this particular Dynamo instance before the time specified by the item-cachetimeout attribute for that item descriptor has elapsed. If an item has not been accessed for the duration of the item-cache-timeout value, then its entry in the item cache is invalidated. The default, 0, lets the item remain in the cache indefinitely until otherwise invalidated. To use the item-cache-timeout attribute, set its value (in milliseconds) in the item-descriptor tag for item types that are associated with a particular user session. For example:
<item-descriptor name="order" cache-mode="simple" item-cache-timeout="180000"> ... </item-descriptor>
125
10 - SQL Repository Caching
In most cases, using the item-expire-timeout attribute will be preferable to using the item-cachetimeout attribute, since the item-cache-timeout approach is not suitable for certain types of data access. You can use either of these cache timeout approaches as an alternative to using the locked cache mode. You can use cache timeout attributes together with simple cache mode to control the behavior of the repository caches. The cache timeout attributes are useful as an alternative means to using locked cache mode for caching user data associated with a particular session. A users session is typically handled by a single Dynamo server for as long as the session lasts. If a users session expires, and then the user moves to another Dynamo server, the cached data will expire before the user has a chance to log back in to a server that might have previously cached stale data for that user. This attribute is also a means to reduce a systems memory requirements by removing cached items that are no longer being accessed.
Cache Statistics
You can view statistics on caches in a repositorys page in the Administrative Interface Component Browser, under the heading Cache usage statistics. For example, the Profile Repositorys page in the Component Browser can be found at:
http://localhost:8830/nucleus/atg/userprofiling/ProfileAdapterRepository/
Monitoring cache statistics during testing and after deployment can help you improve performance by setting cache sizes to appropriate levels. If you have a high quantity of misses and no hits, you are gaining no benefit from caching, and you can probably just turn it off, by setting the cache size to 0. If you have a mix of hits and misses, you might want to increase the cache size. If you have all hits and no misses, your cache size is big enough and perhaps too big. There is no harm in setting a cache to be too big unless it will fill up eventually and consume more memory than is necessary. The cache usage statistics table presents the following statistics:
Property
entryCount cacheSize usedRatio
Description The number of entries currently in the cache. The maximum size of the cache. The percent of the maximum cache size now in use. If this ratio is close to 100, you probably want to increase the cacheSize. The total number of attempts to retrieve items or query results from the cache. The total number of times an access attempt successfully retrieved an item or query result from the cache. The total number of times an access attempt did not find an item or query result from the cache.
accessCount
hitCount
missCount
126
10 - SQL Repository Caching
The ratio of hitRatio to accessCount. The number of times the entire cache has been invalidated. The number of individual entries that have been invalidated.
You can also examine the contents of a cache using the <dump-caches> tag. See dump-caches Tag in the SQL Repository Definition Tag Reference section of this chapter.
Cache Loading
You can achieve better performance in a SQL repository-based application by pre-loading the caches. Note that the improvement in performance from pre-loading caches comes at the expense of slower startups. You can configure the SQL repository to automatically save and restore its caches, or you can load the caches manually by using the startSQLRepository script with an appropriate XML file. It is probably a good practice to put cache-loading tags in a separate XML file with the same name as your repository definition file and rely on XML file combination to invoke the queries.
Automatic Reloading
You can configure a GSARepository component to automatically save the state of its item caches when it is stopped and reload the same items into its caches when it is restarted. Set the following property:
restoreCacheOnRestart=true
With this setting, the repository saves an XML file at the location specified by the repositorys cacheRestoreFile property. This XML file contains a set of tags that reload each of the items that existed in the cache when the repository component was stopped.
The PCDATA for this element is a comma-delimited list of repository IDs to pre-cache during start-up of the repository. The properties attribute, which is optional, specifies a list of property names to cache. For example, the following caches the displayName and description properties of four product items:
<load-items item-descriptor="product"> properties="displayName,description"> prod10001,prod100001,prod100002,prod100003 </load-items>
127
10 - SQL Repository Caching
Loading with the query-items Tag
You can also use the <query-items> tag in a repository definition file to load all or some portion of your repository items on startup. For example, you could load all items of the users type with this tag:
<query-items item-descriptor="users">ALL</query-items>
The <query-items> tag produces a large number of log messages when you use it to load many items. You can use the quiet="true" attribute to eliminate these messages:
<query-items item-descriptor="product" quiet="true">ALL</query-items>
You can use the id-only="true" attribute to output just the repository IDs of the items returned by the query:
<query-items item-descriptor="product" id-only="true">ALL</query-items>
For more information, see the Querying Items topic in the Development, Testing and Debugging with the SQL Repository section of this chapter.
might cause the following to be logged, if there were four product items in the cache:
============== START BUFFER PRECACHE ============== <load-items item-descriptor="product"> prod100003,prod100002,prod100001,prod10001 </load-items> =============== END BUFFER PRECACHE===============
Cache Flushing
You can flush (invalidate) the caches for an item descriptor or an entire SQL repository, using the following methods. Note that first, you must cast your atg.repository.RepositoryItemDescriptor to an atg.repository.ItemDescriptorImpl. If you are using distributed cache mode, use the Cache Invalidator, as described in the Cache Invalidation Service section below. The methods in atg.repository.ItemDescriptorImpl are:
128
10 - SQL Repository Caching
Invalidates the item caches for this item descriptor. Invalidates both the item and query caches for this item descriptor. Removes the specified item from any repository caches
invalidateItemCache()
invalidateCaches()
removeItemFromCache(String pId)
These methods also have versions that accept a boolean parameter that indicates whether the cache should be changed globally, or just for the local cache. These methods are:
removeItemFromCache(id, boolean pGlobal) invalidateCaches(boolean pGlobal) invalidateItemCache(boolean pGlobal)
If this global parameter is true, the invalidation occurs across the cluster. Otherwise, the invalidation occurs only in the local Dynamo instance. The removeItemFromCache method, when given a true value, will use one of two mechanisms to distribute the invalidation event: If the item descriptor uses distributed cache mode, it uses the event server to send the invalidation event. Otherwise, it uses the GSAInvalidatorService to send the event.
The invalidateCaches and invalidateItemCache methods, when given true for the global parameter, will always use the GSAInvalidatorService. If this service is not enabled, a warning is logged and the cache is only invalidated locally. This method in atg.repository.RepositoryImpl affects all caches in the repository:
invalidateCaches()
You can cast your repository to these classes and call these methods from there. You can both flush items of a specific kind, items and queries of a specific kind or a specific item with these methods. For example, here is how you might use the invalidateItemCache() method to invalidate the item caches for every item descriptor in a repository:
RepositoryImpl rep = getRepository(); String[] descriptorNames = getItemDescriptorNames(); // iterate over all the descriptors for (int i=0; i<descriptorNames.length; i++) {
129
10 - SQL Repository Caching
String name = descriptorNames[i]; }
You need to make sure that all Dynamo instances participating in the Cache Invalidator scheme are configured to access the same SQL-JMS database with the appropriate JDBC Datasource configurations. The SQL repository or repositories that will be invalidated may or may not be part of the same datasource.
The {repositoryPath} required parameter is the Nucleus address of the GSARepository component. You can optionally specify the [itemDescriptorName] parameter to invalidate only cached items of that item type. You can optionally specify the [repositoryItemId] parameter to invalidate only the specified repository item. The Cache Invalidator action is initiated by performing a RMI call to the
GSAInvalidatorService.invalidate() method. The RMI call is made by executing the GSA
130
10 - SQL Repository Caching
the Nucleus path of the SQL repository (required) If this is the only argument, then the cache is invalidated for the entire repository. Invalidates all items of this item type. Invalidates a specific item from the repository. item descriptor name (optional)
repositoryPath
repositoryName
repositoryId
item ID (optional)
You can enable additional debugging messages by setting the property loggingDebug=true in the following components:
/atg/dynamo/service/GSAInvalidatorService (for producer or message source
debugging)
/atg/dynamo/service/GSAInvalidationReceiver (for consumer or message sink
debugging)
Second, this topic needs to be configured in Patch Bay. A new file needs to be created in the CONFIGPATH at /atg/dynamo/messaging/dynamoMessagingSystem.xml with this content:
<?xml version="1.0" encoding="ISO-8859-1" ?> <dynamo-message-system> <patchbay> <!-- DAS Messages --> <message-source> <nucleus-name> /atg/dynamo/service/GSAInvalidatorService </nucleus-name> <output-port> <port-name> GSAInvalidate </port-name> <output-destination> <provider-name> sqldms </provider-name> <destination-name> sqldms:/sqldms/DASTopic/GSAInvalidator </destination-name> <destination-type> Topic </destination-type>
131
10 - SQL Repository Caching
</output-destination> </output-port> </message-source> <message-sink> <port-name> GSAInvalidate </port-name> <input-destination> <provider-name> sqldms </provider-name> <destination-name>
sqldms:/sqldms/DASTopic/GSAInvalidator </destination-name> <destination-type> Topic </destination-type> </input-destination> </input-port> </message-sink> </patchbay> </dynamo-message-system>
132
10 - SQL Repository Caching
The XML document type definition for the SQL repository includes operation tags whose primary purpose is to help you develop, test, and debug your SQL repository template. You can use these tags to modify your repositorys database by: Adding Items Updating Items Removing Items Querying Items Importing and Exporting Items and DDLs
These tags are used by a script named startSQLRepository. The use of this script is described in the startSQLRepository Script and the Template Parser section in this chapter. To use these developmental tags: 1. Go to the repositorys page in the Administration Interface. For example, for the SQL Profile Repository, go to:
hostname:8830/nucleus/atg/userprofiling/ProfileAdapterRepository
2.
In the Run XML Operation Tags on the Repository text area, enter the developmental tags and click Enter.
You can instead use the startSQLRepository script from a command line. Create an XML repository definition file and pass it to the startSQLRepository script with appropriate arguments. See the startSQLRepository Script and the Template Parser section in this chapter for more information. Note that if you add or remove an item descriptor in your repository definition file, you must close and restart the ACC as well as Dynamo. Otherwise, errors may result. For example, if you remove an item descriptor, the item descriptor will still appear as an option in the ACC query editor (List items of type... ) and may cause errors if selected.
133
11 - Development, Testing and Debugging with the SQL Repository
Adding Items
You can use an XML template to add repository items. Use an <add-item> tag for each repository item you want to add. Each <add-item> tag must include an item-descriptor attribute to specify the name of the item descriptor to which this repository item should belong. You can nest <set-property> tags within the <add-item> tag to set property values of the new repository item. Any properties you do not set will have the default property value for that item descriptor. For example, the following tags add to the database an instance of users with id = 1. It sets the
username property to Marty. <add-item item-descriptor="users" id="1"> <set-property name="username" value="Marty"/> </add-item>
Note that <add-item> tags are processed one at a time. They cannot make forward references to other items and no attempt is made to satisfy database integrity constraints (beyond that automatically done with the cascade operator). Use the <import-items> tag if you want to load in items with forward references. Note also that if you specify the ID of an existing repository item, you will update that item, overwriting the values of the existing item with the values you specify in the <add-item> tag. Any add or remove attributes in a <set-property> tag within an <add-item> tag will be ignored.
134
11 - Development, Testing and Debugging with the SQL Repository
The $tag:<name>$ syntax can be used only in: the value attribute or body of a <set-value> tag the query attribute or the body of a <query-items> tag
The template parser substitutes the ID of the item you created using that tag. For example, suppose you want to add an item, one of whose properties is another repository item. Lets say we are adding a book item, and the book item has a author property, which is itself a repository item. If you dont want to supply the repository ID for the author repository item, you can use a tag attribute placeholder like this:
<add-item item-descriptor="author" tag="AUTHORID_TAG"> <set-property name="authorName" value="Arthur Ransome"/> ... </add-item> <add-item item-descriptor="book"> <set-property name="title" value="Swallows & Amazons"/> <set-property name="author" value="$tag:AUTHORID_TAG$"/> </add-item>
135
11 - Development, Testing and Debugging with the SQL Repository
Updating Items
You can update repository items using the <update-item> tag. The <update-item> tag encloses one or more <set-property> tags that specify the properties and values being set. Each <update-item> tag must include an item-descriptor attribute to specify the name of the item descriptor of the repository item being removed. You can also use the skip-update attribute to set properties in the item, but avoid the update item call until the transaction is committed. For example, the following element changes the value of the dependents property of the user with id of
1: <update-item item-descriptor="user" id="1" skip-update="true"> <set-property name="dependents" value="1414,1732"/> </update-item>
You can use the add or remove attributes to add or remove values from multi-item properties without overwriting the whole property value. For example, to add another value to the dependents property:
<update-item item-descriptor="user" id="1" skip-update="true"> <set-property name="dependents" value="1799" add="true"/> </update-item>
Removing Items
You can remove items from the repository using the <remove-item> tag. Each <remove-item> tag must include an item-descriptor attribute to specify the name of the item descriptor of the repository item being removed. For example, the following tag removes any repository item whose item descriptor is named users and whose repository ID is 1.
<remove-item item-descriptor="users" id="1"/>
136
11 - Development, Testing and Debugging with the SQL Repository
Set the reference property to null and update item of type Y, essentially nulling the foreign key Delete the item of type Y, since we cant null the foreign key. Remove the element in the collection property of Y that refers to the item of type X. This is has the effect of deleting the one-to-many or manyto-many row that represents the Y to X reference. We are not deleting the item of type Y
Y has a non-required property whose item-type is X Y has a required property whose item-type is X
Data in an auxiliary table will always be deleted by a <remove-item> tag regardless of this attribute because it is not considered a reference. The anyReferencesToItem method queries whether any cross-references to a repository item exist within the repository that contains that item. It uses the same logic as removeReferencesToItem to determine whether references exist. This method can only detect references through queryable properties. Calling these methods generates multiple repository queries per call, one for each property descriptor that might refer to the item. For example, if the items type is contact-info, one query is performed for each property descriptor whose type is either contact-info or any multi-valued type that might contain a list of items of type contact-info. The queries each fetch at most one item from the repository, so the effect on the repositorys cache should be minimal.
Querying Items
You can perform queries against the repository using the <query-items> tag. The query itself may be specified as a query attribute of the <query-items> tag or as PCDATA between the opening and closing tags. The query uses the Repository Query Language (RQL) described in the Repository Query Language section of the Repository Queries chapter. For example, the following tag queries the database for any repository items whose item descriptor is named users and whose username property is Marty:
<query-items item-descriptor="users">username="Marty"</query-items>
Queries can be used in this way to pre-load repository caches. See Cache Loading in the SQL Repository Caching chapter.
137
11 - Development, Testing and Debugging with the SQL Repository
Importing and Exporting Items and DDLs
remove-all-items Tag export-items Tag import-items Tag print-ddl Tag
Other operations tags let you import items from another repository or export items. You can also print out the DDLs used in setting up the tables corresponding to the repository template. See the descriptions of the following tags in the SQL Repository Definition Tag Reference:
Note that you can also use the startSQLRepository script to export, import, and print repository items, as described in the next section.
138
11 - Development, Testing and Debugging with the SQL Repository
In addition, you need to make sure that repository IDs in the source repository do not collide with repository IDs in the target repository. Make sure that both the source database and target database already contain IdSpaces. The IdSpaces in the source database and target database must have the same names. Furthermore, the name of the IdSpaces used by each item descriptor should be the same in the source repository for the export and the target repository for the import. If you do this, then the import operation reserves all the IDs it encounters for repository items it creates in the target database. See ID Generators in the Core Dynamo Services chapter of the ATG 6 Dynamo Programming Guide for more information about IdSpaces. The startSQLRepository scripts use the following syntax:
startSQLRepository <arguments> [file name.xml]
(Note that in Windows, you must run the script from the <ATG6dir>\home directory.) For example, to load an XML template whose CONFIGPATH name is /atg/test.xml in a repository with a Nucleus address of /atg/userprofiling/ProfileAdapterRepository:
startSQLRepository -m DPS repository /atg/userprofiling/ProfileAdapterRepository /atg/test.xml
Note that the repository template file name you provide is a reference to a CONFIGPATH location, and not a pathname in your file system. For example, you can put a file in your localconfig directory and refer to it as /<your-file>. You can also use a file with the same name as your repositorys existing definition file and then omit the file name argument from the startSQLRepository command. The startSQLRepository script uses XML file combination to combine all files with the same name into a single repository definition. See XML File Combination in the Nucleus: Organizing JavaBean Components chapter of the ATG 6 Dynamo Programming Guide. For example, if you include the following file at <ATG6dir>/home/localconfig/atg/userprofiling/userProfile.xml and run the startSQLRepository script, it will print all profiles in your Profile Repository:
You can use the startSQLRepository script together with the test operation tags described earlier in this chapter to quickly test a query, or add, update, remove, or print an item.
139
11 - Development, Testing and Debugging with the SQL Repository
Argument
-m <module name>
Purpose The modules to load. The -m argument must precede all other arguments if it is present. You can specify a list of module names separated by your operating systems file delimiter (generally colon or semicolon). If you omit the -m argument, then the default set of modules for your Dynamo installation is used. For more information, see the Starting and Stopping Dynamo chapter in the ATG 6 Installation and Configuration Guide for your application server. The name of the Dynamo server to start. This optional argument lets you specify a Dynamo server that is configured to use a Dynamo data source that points to the repositorys database. See the note on Server Conflicts, however.
-s <server name>
Repository>
If you run the script with the DPS module or a module which requires the DPS module, you can omit this argument, and the script will use the first repository registered in the
/atg/registry/ContentRepositories
component.
-outputSQL
Outputs a DDL (SQL) file for the XML templates in the repository to standard out. Outputs a DDL (SQL) file for the XML templates in the repository to the specified file. Customizes the DDL for the SQL variant used by the specified vendors database software. Possible values are: db2 microsoft oracle solid sybase
-outputSQLFile <file>
-database <vendor>
-output <file>
Sends all output from <print-item> and <query-item> tags to a file (instead of standard out).
140
11 - Development, Testing and Debugging with the SQL Repository
Outputs additional logging information. This option is equivalent to setting the loggingSQLInfo and loggingSQLDebug properties to true for the JTDataSource used by the repository. All SQL emitted will be logged. Exports all of the items of one or more item descriptors to an XML repository definition file. The location of the resulting file is relative to the <ATG6dir>/home directory. Exports all of the items of all item descriptors in this repository to a file. The location of the resulting file is relative to the <ATG6dir>/home directory. To export data from more than one repository into the same file, use this option. This may be preferable to the -export option if your repositories are linked as it prevents duplicating the linked item descriptors in more than one file. Exports all repositories into one file. The location of the resulting file is relative to the <ATG6dir>/home directory. By default, when you use one of the export arguments, all referenced item descriptors are automatically added to the list of item descriptors to be exported. If you use the -skipReferences argument, referenced item descriptors are added only if you affirmatively include them. If you use this argument, this operation wont be wrapped in a transaction. Using a transaction for large operations can run into database limitations on transaction sizes and numbers of permitted row-level locks. Imports XML definitions from a file created with the export or export all arguments. Outputs additional logging information. This option is equivalent to setting the loggingDebug property to true for the repository. If the content you are exporting contains nonASCII characters, use this option to specify an encoding such as 8859_1 or SJIS in which to save your content. The file names of the XML files to parse.
-verboseSQL
-exportRepositories <path-to-
repository1,path-to-repository2> <file>
-skipReferences
-noTransaction
-import <file>
-debug
-encoding <encoding>
<file names>
141
11 - Development, Testing and Debugging with the SQL Repository
In addition, you can use other arguments, such as -workspace, -project, and projectType if you are using the startSQLRepository script with ATG Publishings versioned repository. See Export and Import Tools in the ATG 6 Publishing Administration and Development Guide for more information. You can pass more than one XML file to the startSQLRepository script, and the Template Parser handles them in the order you specify them. This enables you, for example, to pass your full repository definition file together with a test file that uses the test operation tags to manipulate repository items. See the following SQL Repository Test Example.
Server Conflicts
Note that if you start the startSQLRepository script on a server that is already running, errors can result. Use the -s argument to specify a server that is not currently running.
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <!-This is a simple xml template demonstrating add-item, remove-item, and query-items tags. --> <!DOCTYPE gsa-template SYSTEM "http://www.atg.com/dtds/gsa/gsa_1.0.dtd"> <gsa-template> <header> <name>Test 1</name> <author>Marty</author> <version>1</version> </header> <!-- This defines the item-descriptor --> <item-descriptor name="users" default="true"> <table name="users" id-column-names="id" type="primary"> <property name="prop1"/> </table> </item-descriptor> <!-- This removes from the database any of 'users' with id = 1 --> <remove-item item-descriptor="users" id="1"/>
142
11 - Development, Testing and Debugging with the SQL Repository
<!-- This adds to the database an instance of 'users' with id = 1 --> <add-item item-descriptor="users" id="1"> <set-property name="prop1" value="Marty"/> </add-item> <!-- This queries the database for any of 'users' with prop1 = "Marty" --> <query-items item-descriptor="users"> prop1="Marty" </query-items>
</gsa-template>
The location of the resulting file is relative to the <ATG6dir>/home directory. Exporting Individual Repositories To specify individual repositories to export, issue the following command:
bin/startSQLRepository -m module -export all file_name.xml -repository /atg/Path_to_Repository
For example, to export the Product Catalog from the Pioneer Cycling store database into a file called products.xml, issue the following command:
bin/startSQLRepository -m PioneerCycling -export all products.xml -repository /atg/commerce/catalog/ProductCatalog
For additional information about exporting Pioneer Cycling repositories individually, see the ATG Consumer Commerce Solution Set Documentation. For information about
143
11 - Development, Testing and Debugging with the SQL Repository
4. 5. 6.
exporting Motorprise repositories, see the ATG 6 Business Commerce Reference Application Guide. Add a JDBC driver for your database and configure the JDBC connection pool. For more information, see the Configuring Databases and Database Access chapter in the ATG 6 Installation and Configuration Guide for your application server. From the command prompt, switch to the <ATG6dir>/home directory. Add the contents of the XML file to the destination database. Importing All Files To import the content from all the XML files, issue a command like this (type the following on one line, with no line breaks):
bin/startSQLRepository -m PioneerCycling -import all.xml -repository /atg/userprofiling/ProfileAdapterRepository
Importing Individual Files To import content from individual files, issue the following command:
bin/startSQLRepository -m module -import file_name.xml -repository /atg/Path_to_Repository
For example, to import the content from the Pioneer Cycling products.xml file, issue the following command:
bin/startSQLRepository -m PioneerCycling -import products.xml -repository /atg/commerce/catalog/ProductCatalog
Note that when binary data in repository items exported, it is represented using base64 encoding.
In the text field, you can enter any XML operations tags against the current repository as if they were commands. Click the Enter button and the page displays the output you would obtain by running the startSQLRepository script against the repository.
144
11 - Development, Testing and Debugging with the SQL Repository
Debug Levels
The SQL Repository component has a debugLevel property you can use to adjust the debug log entries. This property is an integer from 0 to 15, with 15 resulting in the greatest frequency of debug log entries. The default level is 5; this typically is the level desired when contacting ATG support to diagnose problems. Level 3 is the lowest level at which SQL debugging statements are issued. At level 3 and above when loggingDebug is true on the SQL Repository component, it also turns on loggingDebug on its Transaction Manager, since Transaction Manager debugging is often needed in conjunction with SQL debugging. Even at level 0, some debug statements are issued. You can also get debugging messages for an individual item descriptor or property. Turn on logging debug in the Dynamo Administration Interface page for the Repository component. You can also turn on debug messages by including a loggingDebug attribute tag in the repository definition for that item descriptor or property. For example:
<item-descriptor name="user" ...> <attribute name="loggingDebug" value="true" /> ...<property ... /> </item-descriptor>
145
11 - Development, Testing and Debugging with the SQL Repository
146
11 - Development, Testing and Debugging with the SQL Repository
This chapter includes reference information about the SQL repository: SQL Repository Definition Tag Reference Document Type Definition for SQL Repository Definition Files Sample SQL Repository Definition Files Configuring the SQL Repository Component
147
12 - SQL Repository Reference
input-parameter-types Tag returns Tag dependencies Tag Development Operation Tags: transaction Tag rollback-transaction Tag import-items Tag add-item Tag update-item Tag query-items Tag print-item Tag set-property Tag remove-item Tag remove-all-items Tag export-items Tag load-items Tag dump-caches Tag print-ddl Tag
DOCTYPE Tag
All XML documents declare their doctype, referencing a document type definition file. The DTD for SQL repository templates is installed within the <ATG6dir>/DAS/lib/classes.jar archive, but can be referred to with this URL:
http://www.atg.com/dtds/gsa/gsa_1.0.dtd
You should not need to change the reference to the DTD in the following DOCTYPE tag:
<!DOCTYPE gsa-template PUBLIC "-//Art Technology Group, Inc.//DTD General SQL Adapter//EN" "http://www.atg.com/dtds/gsa/gsa_1.0.dtd">
Note that if your SQL repository definition is composed from more than one file, using XML file combination, only the file that appears first in the CONFIGPATH should include the DOCTYPE declaration. See XML File Combination in the Nucleus: Organizing JavaBean Components chapter of the ATG 6 Dynamo Programming Guide.
148
12 - SQL Repository Reference
gsa-template Tag
The entire template is enclosed in a <gsa-template> tag.
Child Tag header item-descriptor add-item dump-caches export-items import-items load-items print-ddl print-item query-items remove-all-items remove-item transaction update-item
How many? zero or one zero or more zero or more zero or more zero or more zero or more zero or more zero or more zero or more zero or more zero or more zero or more zero or more zero or more
header Tag
The <header> tag provides information that can help you manage the creation and modification of repository definition files.
149
12 - SQL Repository Reference
author zero or more The author or authors of this template. An identifier for the version of this template. version zero or one description zero or one
For example, the header of your template might look like this:
<header> <name>Catalog Template</name> <author>Herman Melville</author> <author>Emily Dickinson</author> <version>$Id: catalog.xml,v 1.10 2000/12/24 03:34:26 hm Exp $</version> <description>Template for the store catalog</description> </header>
item-descriptor Tag
The SQL repository template contains one <item-descriptor> tag for each set of repository items that have common attributes.
Attribute name
Description The item descriptor name. Must be unique within the repository. If you want to refer to the same item descriptor with more than one name, set the itemDescriptorAliases property of the Repository component. The name of the item descriptor in the ATG Control Center interface. If no display-name is specified, the name attribute is used. If you specify a resource bundle for this item descriptor using the <attribute name=resourceBundle> tag, then the display-name-resource attribute specifies the resource bundle key that holds the display name. See Localizing SQL Repository Definitions.
Value Required
display-name
String
display-nameresource
String
150
12 - SQL Repository Reference
An optional attribute to describe purpose of the item descriptor. If you specify a resource bundle for this item descriptor using the <attribute name=resourceBundle> tag, then the description-resource attribute specifies the resource bundle key that holds the description of the item descriptor. See Localizing SQL Repository Definitions. Is this the default item descriptor for this repository? The default item descriptor is used for newly created repository items if a different item descriptor is not specified. If no item descriptor is designated as the default, then the first item descriptor in the repository definition file is the default. The item descriptor from which this item descriptor inherits. See Item Descriptor Inheritance. The name of a property of a parent item descriptor that specifies the names of its child item descriptors. See Item Descriptor Inheritance. The value of its parents sub-typeproperty that triggers use of this
description
description-resource
String
default
super-type
sub-type-property
sub-type-value
String
particular type. See Item Descriptor Inheritance. copy-from The item descriptor from which this item descriptor inherits by copying. See Item Descriptor Inheritance. You can set this property to the name of a repository item property. A user interface can then represent the repository item using this property. For example, a profile item descriptor might use displayproperty="login". Then, each repository item would be represented using the value of the items login property. String
display-property
String
151
12 - SQL Repository Reference
version-property id-separator
A property whose value is used as a version control mechanism for this item. This property must be an int data type. The property is incremented each time the item is updated. A character used to separate elements of a multi-column repository ID when the ID is string encoded. Set this to the name of the ID space to use for this item type. An item descriptor with a single-column repository ID uses the name of the item descriptor as the name of the ID space by default. An item descriptor with a multi-column repository ID uses the name of the primary table and the names of the ID column in that table by default. See IdSpaces and the id Property in the SQL Repository Data Models chapter and the ID Generators section of the Core Dynamo Services chapter of the ATG 6 Dynamo Programming Guide for more information about ID space names and how they affect the IDs of newly generated items. Hidden item types are not displayed in the ATG Control Center. Set this to true if this item descriptor should only be displayed to expert users. If a text search query does not explicitly specify a property to search, then the query searches this list of property names. See Text Search Queries in the SQL Repository Queries chapter. The cache mode for this item descriptor. Caching can be disabled for a single property using the cache-mode attribute of the <property> tag. See the SQL Repository Caching chapter.
String
id-space-names
String
hidden
expert
text-searchproperties
cache-mode
Default is simple.
152
12 - SQL Repository Reference
The maximum number of items belonging to this item descriptor to store in the item cache. When the number of items requested exceeds this number, the least recently accessed item is removed from the cache. See the SQL Repository Caching chapter. The maximum time in milliseconds that an entry can remain in the item cache before it is refreshed. See Cache Timeout for more information. The maximum time in milliseconds that an entry can remain in the query cache before it is refreshed. See Cache Timeout for more information. The time in milliseconds that an item cache entry can remain unused before it is removed from the cache. See the SQL Repository Caching chapter. The maximum number of queries of this item descriptor to store in the query cache. When the number of queries issued against this item descriptor exceeds this number, the least recently used query is removed from the cache. See the SQL Repository Caching chapter. Integer. Default is 1000.
item-cache-size
item-expire-timeout
Integer. Default is -1, which means items remain in the cache indefinitely, until otherwise invalidated. Integer. Default is -1, which means items remain in the cache indefinitely, until otherwise invalidated. Integer. Default is 0, which means items remain in the cache indefinitely, until otherwise invalidated. Integer. Default is 0, effectively disabling the query cache.
query-expire-timeout
item-cache-timeout
query-cache-size
153
12 - SQL Repository Reference
content Are the items defined by this item descriptor ContentRepositoryItems? If you set this to true, you must also define the folder-id-property attribute, the content-property attribute and one or more of contentname-property, content-pathproperty, and use-id-for-path. Are the items defined by this item descriptor FolderItems? There can be at most one folder item descriptor in each repository. If you set this to true, you must also define the folder-idproperty attribute, the contentproperty attribute and one or more of content-name-property, contentpath-property, and use-id-forpath. Specifies the name of a property that defines the name of this content item or folder item in the folder hierarchy. Unlike content-path-property, the value of this attribute should not include any path separator characters or the names of any parent folders. Specifies a property of this item that defines the absolute path name of this item in the folder hierarchy. This name should include a leading path separator character. Set this to true if the Repository ID for this item is the relative path name of the item in the folder hierarchy. Use this attribute if the column used to store the content-path-property happens to be the primary key for the table containing the item. The name of the property that specifies the ID of the folder containing this folder or content item. Both content item descriptors and folder item descriptors must specify a folder-idproperty.
folder
content-nameproperty
String
content-pathproperty
String
use-id-for-path
folder-id-property
String.
154
12 - SQL Repository Reference
The name of the property that defines the content of a ContentRepositoryItem. If the item descriptor uses the content="true" attribute, then you must set this attribute. String. The type of the property itself may be File, byte[] or String. If the property is byte[] or String, then the content is stored in the database. If the property is File, then the content is retrieved from the file system, using the FilePropertyDescriptor. String
content-property
content-lengthproperty
An optional attribute that specifies a property that contains the number of bytes in the content. This property is used by the getContentLength() method of the ContentRepositoryItem interface. An optional attribute that specifies a property that contains the last modified time for that piece of content. The propertys data-type should be date or timestamp. This property is used by the getContentLastModified() method of the ContentRepositoryItem interface.
last-modifiedproperty
String
155
12 - SQL Repository Reference
rql-filter named-query zero or one zero or more
If a <property> tag appears as a direct child of an <item-descriptor> tag (rather than a child of a
<table>tag), then the property is a transient property of the repository item and is not stored
persistently in the database. Transient properties are readable and writable, but cannot be made queryable. See the Transient Properties section of this chapter.
table Tag
The <table> tag defines the SQL database tables that store properties of repository items.
Description The tables name in the database. The name or names of the database columns that correspond to the repository ID. For multi-valued properties that are Arrays, Lists or Maps, specifies which column is to be used as the sorting value to determine the order of the Array or List and the key for the Map. The tables type.
multi-column-name
String
type
Default is auxiliary.
156
12 - SQL Repository Reference
property Tag
A <property> tag that is a child of a <table> tag defines one of the persistent characteristics of a repository item. A <property> tag that is a direct child of an <item-descriptor> tag defines a transient characteristic of a repository item. Since such a transient property is not associated with any database table, it is not stored when the repository item is updated in the database.
Purpose Required. The column name or names in the SQL database; defaults to the value of name. An optional name to use to identify the property in the user interface; defaults to the value of name. If you specify a resource bundle for this property using the <attribute name=resourceBundle> tag, then the display-name-resource attribute specifies the resource bundle key that holds the display name. See Localizing SQL Repository Definitions. An optional attribute to describe purpose of the property; defaults to the value of name.
display-name
String
display-name-resource
String
description
String
157
12 - SQL Repository Reference
description-resource String
If you specify a resource bundle for this property using the <attribute name=resourceBundle> tag, then the description-resource attribute specifies the resource bundle key that holds the description of the property. See Localizing SQL Repository Definitions. You can group repository item properties into categories, so that a user interface will display related properties together, rather than in alphabetical order according to their display-name attributes. See Sorting Properties in the SQL Repository Item Properties chapter. If you specify a resource bundle for this property using the <attribute name=resourceBundle> tag, then the category-resource attribute specifies the resource bundle key that holds the category of the property. See Localizing SQL Repository Definitions. Required, unless item-type or propertytype is specified. See Data-Type Correspondences in this section for information about how these values map to Java and SQL data-types.
category
String
category-resource
String
data-types
string big string enumerated boolean int short long float double byte binary date timestamp array set list map
item-type
If the value of this property is another repository item, specifies the item descriptor type of that repository item. Required if data-type or property-type is not specified.
158
12 - SQL Repository Reference
SQL data type. The SQL type of the corresponding column if it is different from the default type for the data-type, as specified in the Data-Type Correspondences topic in this section. Specifies the Java class of a user-defined property. See the User-Defined Property Types: Using the property-type Attribute section in this chapter. If the data-type is an array, list, set, or map of items, this attribute is the name of the item descriptor of the items these collections contain. All of the items must be of the same base type. If the data-type is an array, list, set or map of primitive values, this is the type of the primitive value. The data-type can be any valid value other than array, list, set or map. Every element in the collection must be of the same type. A default value for the property, if one is not specified when the repository item is created. Note that you cannot set a default value for multi-valued attributes (array, set, list, or collection type properties). Use this attribute to group properties together for the purpose of loading them in the same select statement. By default, each property is in a group with the same name as its table. You can set the group for a property to add or remove properties from these default groups. This gives you a simple way to optimize the SQL generated by the repository. See Cascading Data Relationships. Java type
sql-type
property-type
component-item-type
component-data-type
Data-type name
default
A string
group
A string
cascade
Zero or more of insert, update or delete, separated by commas The Nucleus address of another Repository.
repository
Specifies that the value of this property refers to one or more repository items that are part of another Repository. If you specify a relative path, it is relative to this Repository. See Linking between Repositories.
159
12 - SQL Repository Reference
cache-mode
disabled inherit
The cache mode for this property. If you set the cache-mode, it overrides the item descriptor cache mode, if they are different. This can be set on a specific property to disable caching. The only cache mode you can set on a per-property basis is disabled. You can restore the default cache mode by setting this to inherit. See SQL Repository Caching. The class name of a PropertyEditor to use for this property. See the JavaBeans specification for a description of PropertyEditors. Defaults to true. Transient properties cannot use queryable="true" unless the entire item descriptor is transient. See Transient Properties in this chapter. Defaults to true Defaults to true Defaults to false. This should be set to true for all properties whose corresponding database columns are defined as NOT NULL. Hidden properties are not displayed in the ATG Control Center; defaults to false Defaults to false
editor-class
queryable
Boolean
hidden
Boolean
expert
Boolean
Note that the values of name attributes are case-insensitive. Therefore, you cannot have, for instance, two different properties named gender and GENDER.
160
12 - SQL Repository Reference
The following table shows how the data-type attribute names for the primitive types correspond to Java object types and SQL data types. Note that some SQL data types may vary, depending on the specific SQL implementation you are using. You can specify a different SQL data type correspondence by using the sql-type attribute of the <property> tag.
Recommended SQL Data Type VARCHAR VARCHAR or CLOB (Oracle) LONG VARCHAR or CLOB DATETIME (Sybase or MS) DATE (DB2 or Oracle) DATETIME (Sybase or MS) DATE (Oracle 8i) TIMESTAMP (DB2 or Oracle 9i) INTEGER NUMERIC(1) INTEGER INTEGER LONG VARBINARY (Sybase or MS) LONG RAW or BLOB (Oracle) BLOB (DB2) INTEGER SMALLINT (DB2) FLOAT (DB2, Sybase or MS) NUMBER (Oracle) DOUBLE (DB2, Sybase or MS) NUMBER (Oracle) NUMERIC(19) BIGINT (DB2) none none none none
String java.util.Date
timestamp
java.sql.Timestamp
short
Short
float
Float
double
Double
long
Long
161
12 - SQL Repository Reference
Clob and Blob Limitations
If you plan to use BLOBs (Binary Large Objects) or CLOBs (Character Large Objects), be sure that your database and JDBC driver work with the data and queries you plan to use. Comparison queries (=, !=, <, <=, >, >=) do not work with BLOBs or CLOBs. In addition, Oracle versions before 9.2 do not support pattern-match queries (CONTAINS, STARTS_WITH, ENDS_WITH) against CLOBs. BLOBs and CLOBs are not supported if your Oracle JDBC driver is a Thin driver (any version) or an OCI driver version before 8.1.7.
option Tag
For properties whose data type is enumerated, use <option> tags to indicate the possible values of the enumerated properties. For example:
<property name="gender" data-type="enumerated"> <option value="male" code="0"/> <option value="female" code="1"/> </property>
Description The value of the enumerated option. The integer code that represents the enumerated option in the database.
Value Required. Optional. If no code is specified, an appropriate code is generated by the SQL repository. The value of the code attribute is a sequential integer, with the first option beginning at 0. Optional.
resource
If you specify a resource bundle for this property using the <attribute name=resourceBundle> tag, then the resource attribute specifies the resource bundle key that holds the string value of this property option. See Localizing SQL Repository Definitions.
attribute Tag
A Java Beans PropertyDescriptor can store an arbitrary set of name value pairs called feature descriptor attributes. You can use the <attribute> tag in the SQL repository as a child of a <property> or an <item-descriptor> tag to supply parameters that affect the behavior of properties or item types in your repository definition. A <property> tag or an <item-descriptor> tag can contain zero or more <attribute> tags. The <attribute> tag is an empty tag that defines the parents feature descriptor value or values. This tag allows you to associate arbitrary name/string value pairs with any property or
162
12 - SQL Repository Reference
item type. The name/value pairs are added to the property descriptor via the setValue method of java.beans.FeatureDescriptor, and can later be used by the application. Here is an example:
<property name="employeeNumber" data-type="string"> <attribute name="PCCExpert" value="true" data-type="boolean"/> </property>
See User-Defined Property Types for more information. You can refer to values of Nucleus components with the bean attribute of the <attribute> tag. For example:
<attribute name="documentRootPath" bean="/atg/demo/QuincyFunds/repositories/FeaturesDataStore. relativePathPrefix" />
Attribute Name
Description The name of the name/value pair. You can specify any name here and it will be added to the list of feature descriptor attributes for your property. The value of the name/value pair. The data type of this value is defined by the data-type attribute supplied to this tag. If no data-type attribute is provided, the value of the attribute is a string. The primitive data-type of the value.
Value Required.
Value
Required.
data-type
Default is string.
163
12 - SQL Repository Reference
Bean The name of a Nucleus component or property that is the value of the attribute.
A Nucleus address. If a relative address is specified, the address is relative to the Repository component. See the Assigning FeatureDescriptorValues with the <attribute> Tag section in this chapter.
derivation Tag
The <derivation> tag is used for derived properties. See the Derived Properties section in the SQL Repository Properties chapter. The <derivation> tag can have one or more <expression> tags as children.
Attribute method
Value The derivation method to use. By default, this is firstNonNull. A property name.
overrideproperty
The name of a property that, when set explicitly, overrides the derived property value that would have been used.
expression Tag
The <expression> tag is a child of a <derivation> tag. It encloses a repository item property name. One or more <expression> tags define how the value of a derived property is determined. The <expression> tag has no attributes or child tags. See the Derived Properties section in this chapter.
rql-filter Tag
The <rql-filter> tag can be used to define a filter for database read operations. The <rql-filter> tag has no attributes. It encloses a Repository Query Language (RQL) string that defines the filter query. See the Repository Filtering section in this chapter.
164
12 - SQL Repository Reference
sql-query Tag sql Tag input-parameter-types Tag returns Tag dependencies Tag
named-query Tag
Child of: item-descriptor Tag An <item-descriptor> tag can contain any number of <named-query> elements.
A <named-query> tag can have either an <rql-query> child element or a <sql-query> child element, not both.
rql-query Tag
Child of: named-query Tag
rql Tag
Child of: rql-query Tag The body of this tag specifies the RQL string to be used in the named query.
sql-query Tag
Child of: named-query Tag
165
12 - SQL Repository Reference
sql-query Child Tags
Child Tag sql returns input-parameter-types dependencies How many? one zero or one one one
This element defines a specific SQL statement to be used in the named query.
sql Tag
Child of: sql-query Tag The body of this tag specifies the SQL string to be used in the named query. For stored procedures, use the appropriate stored procedure invocation syntax along with the storedprocedure attribute in the <sql> tag:
<sql stored-procedure="true">
input-parameter-types Tag
Child of: sql-query Tag The <input-parameter-types> element is a comma-separated list of class names that any parameters in the Query must be an instance of. There must be as many class names as parameters.
returns Tag
Child of: sql-query Tag The body of this optional tag specifies a comma-separated list of Repository property names that are returned by this query.
dependencies Tag
Child of: sql-query Tag If any of the properties specified by the body of the <dependencies> tag are changed, then this query will be flushed from the query cache.
166
12 - SQL Repository Reference
transaction Tag
You can use a <transaction> tag to group a set of test operation tags. A <transaction> tag takes no attributes. If a <transaction> tag appears inside of another transaction, the outer transaction is suspended while the inner transaction occurs, and is resumed again at the end of the inner transaction. Note that <add-item> tags in this element are processed one at a time. They cannot make forward references to other items and no attempt is made to satisfy database integrity constraints (beyond that automatically done with the cascade operator). Use the <import-items> tag if you want to load in items with forward references. All test operation tags are enclosed in a single transaction by default, so you do not need a <transaction> tag. However, to avoid possible database deadlocks, you should place all test operation tags inside <transaction> tags. For example: For example, suppose you have this pattern:
The <print-item> tag will not find item 1 because that item has not been committed yet. In addition, you can run into deadlocks with this pattern if you try to access or modify items that may be locked by operations in the outer tag. Instead, you should typically use a pattern like:
<transaction> <add-item item-descriptor="foo" id="1"/> </transaction> <transaction> <print-item item-descriptor="foo" id="1"/> </transaction>
167
12 - SQL Repository Reference
export-items print-item query-items remove-all-items remove-item rollbacktransaction transaction update-item zero or more zero or more zero or more zero or more zero or more zero or more
rollback-transaction Tag
The <rollback-transaction> tag is used only in the <transaction> test operation tag, to mark the transaction as rollback only. It must be empty and has no child tags or attributes.
import-items Tag
The <import-items> tag is a procedural tag that can be used to add items to a repository in a more complex way than is possible using <add-item> tags in a <transaction> tag.
add-item Tag
add-item Child Tag
168
12 - SQL Repository Reference
Attribute itemdescriptor id
Description The name of the item descriptor to use when adding items to the repository Specify the RepositoryId to use for the added item. You must ensure that this ID is not the same as the ID of an existing item. The Nucleus address of the repository to which the item should be added. You do not need to specify this if you are adding to the base repository specified in the startSQLRepository command. If true, indicates that the item should not be added until the transaction is committed. If true, indicates that the item should not be added, even when the transaction is committed. Use this to create transient items. Use this to add a new item with a guaranteed unique RepositoryId . You can refer to this item using this tag attribute in print-item and update-item tags within the same XML file. This is useful for writing test scripts that are run over and over again on the same database, each time operating on different items.
repository
Optional
on-commit
Boolean
skip-add
Boolean
tag
String
update-item Tag
See Updating Items in the Development, Testing and Debugging with the SQL Repository section of this chapter.
169
12 - SQL Repository Reference
Attribute item-descriptor Description The name of the item descriptor to use when updating items in the repository Specifies a repository ID to use for this item. You must specify either id or tag. If you added your item with an <add-item> tag using the tag attribute, you can refer to that item in the same XML file with the tag attribute in the <update-item> tag. Set this to true if you want to set properties in the item, but avoid the update item call until the transaction is committed. id tag
Value Required.
skip-update
query-items Tag
The <query-items> tag has no child tags. See Querying Items in the Development, Testing and Debugging with the SQL Repository chapter. This tag can also be used for loading caches. See Cache Loading in the SQL Repository Caching chapter.
Attribute item-descriptor
Description The name of the item descriptor to use when querying items in the repository. The query to issue against the itemdescriptor. Instead of using this attribute, it is usually more convenient to specify the query in the body of the query-items tag in a PCDATA section. If true, logs only the repository ID of the items returned by the query. If true, eliminates log messages for each item returned. If true, prints the content property of the repository items returned by the query.
Value Required.
query
An RQL query.
id-only
quiet
print-content
170
12 - SQL Repository Reference
Description Specifies a folder to print. When a folder is printed, each of its children is displayed, using the display-property attribute. Specifies an id to use for this item. If you dont set id, tag, or path all items in the descriptor are printed. The name of the item descriptor to use when printing an item. If you omit the id, tag, and path attributes, then all items in this item descriptor are printed. Specifies an item or folder to print. When a folder is printed, each of its children is displayed, using the display-property attribute. Set this to true to print all of the content of the item in addition to the properties. If you added your item with an <add-item> tag using the tag attribute, you can refer to that item in the same XML file with the tag attribute in the <print-item> tag. Value Optional. Path name of folder.
print-item Tag
The <print-item> tag is always empty and has no child tags.
Attribute folder
id
item-descriptor
Required.
path
print-content
tag
set-property Tag
The <set-property> tag is used only in the <add-item> and <update-item> test operation tags. It has no child tags. To set the value of an Array, List, or Set property, use a comma-separated list of values:
<set-property name="interests" value="fishing,fussing,wrassling"/>
To set the value of a Map property, use a comma-separated list of key=value pairs:
<set-property name="homes" value="Jefferson=Monticello,Jackson=Hermitage,Madison=Montpelier"/>
To add or remove a value to a multi-valued property, use the Boolean add or remove attributes. For example, to add a value to the preceding example:
171
12 - SQL Repository Reference
To set a property to null, use this form:
<set-property name="foo" value="__NULL__"/>
To set the value of a property that refers to another repository item, use the ID of the other repository item:
<set-property name="bestBuddy" value="10022349_5"/>
Description The name of the property to set. The value to assign to the property. If true, add this value to a multi-valued property. If true, remove this value from a multi-valued property.
remove-item Tag
The <remove-item> tag is a procedural tag for removing items from the repository. It has no child tags and the following attributes:
Attribute item-descriptor
Description The name of the item descriptor to use when removing an item. Specifies an id to use for this item If you added your item with an <add-item> tag using the tag attribute, you can refer to that item in the same XML file with the tag attribute in the <remove-item> tag. If true, items that refer to the item to be removed will also be removed.
Value Required.
id tag
RepositoryId of item.
Optional.
removereferences-to
172
12 - SQL Repository Reference
remove-all-items Tag
The <remove-all-items> tag is a procedural tag for removing all items in the repository. This tag is enabled only if the system property atg.allowRemoveAllItems is set on startup. You can set this property by adding -Datg.allowRemoveAllItems to the JAVA_ARGS in your <ATG6dir>/home/localconfig/environment.bat or environment.sh file
export-items Tag
The <export-items> tag is a procedural tag for exporting the data required to recreate one or more item descriptors. The data is exported as XML to standard output. Using this tag is similar to running the startSQLRepository script with the -export argument. See the startSQLRepository Script and the Template Parser section in the Development, Testing and Debugging with the SQL Repository chapter.
item-descriptors
Specifies a comma-separated list of one or more item descriptor names. For example:
<export-items item-descriptors="authors,books"/>
By default, when you use <export-items> tag, all referenced item descriptors are automatically added to the list of item descriptors to be exported. If you use the skip-references="true" attribute, referenced item descriptors are added only if you affirmatively include them.
load-items Tag
The <load-items> tag can be used to load the item cache. Loading an item cache can improve performance, since otherwise it can take some time for the normal run of queries to fill the caches.
Description The item descriptor whose item cache should be loaded A list of properties to cache. If no properties are specified, then no properties of the items are cached. If this attribute is set to true, the <load-items> tag loads all the items for the given item descriptor, ignoring the list of repository IDs in the body of the tag. If this attribute is set to true, the <load-items> tag produces no output.
load-all-items
quiet
173
12 - SQL Repository Reference
dump-caches Tag
The <load-items> tag has no child tags. Its body is a comma-separated list of the repository IDs of the items that should be loaded into the item cache.
The <dump-caches> tag can be used to print out the contents of the item cache for one or more item descriptors. It has no child tags.
Attribute dump-type
Description The debug value causes the items in the cache to be logged. The queries value creates a log entry consisting of the <load-items> tag that would be used to re-load the cache. See the example below. The both value both logs the items in the cache and logs the <load-items> tag.
Value
debug, queries, or both
item-descriptors
A comma-separated list of one or more item descriptor names. If no item descriptors are specified, all item descriptor caches are exported.
might cause the following to be logged, if there were four product items in the cache:
============== START BUFFER PRECACHE ============== <load-items item-descriptor="product"> prod100003,prod100002,prod100001,prod10001 </load-items> =============== END BUFFER PRECACHE===============
print-ddl Tag
The <print-ddl> tag has no child tags. You can use it to print the DDLs used. The data is exported to standard output. Using this tag is similar to running the startSQLRepository script with the -
174
12 - SQL Repository Reference
Description The name of the database vendor. This attribute lets you generate SQL appropriate for your production database software. Value Optional. Valid values are: db2 microsoft oracle solid sybase
outputSQL or -outputSQLFile <file> argument. See the startSQLRepository Script and the Template
Parser section in the Development, Testing and Debugging with the SQL Repository chapter.
Attribute database-name
<!-==================================================================== gsa_1.0.dtd - document type for GSA templates @version $Id: //product/DAS/main/Java/atg/dtds/gsa/gsa_1.0.dtd#56 $$Change: 290980 $ ==================================================================== --> <!-- Flag datatype, and values --> <!ENTITY % flag "(true | false)"> <!-- The whole template --> <!ELEMENT gsa-template (header?, (item-descriptor | add-item | update-item | print-item | remove-item | transaction | query-items | remove-all-items | export-items | import-items | print-ddl | dump-caches | load-items | workspace | add-layer | checkin-all)*)> <!-- The header --> <!ELEMENT header (name?, author*, version?, description?)>
175
12 - SQL Repository Reference
<!-- Name of template --> <!ELEMENT name (#PCDATA)> <!-- The author(s) --> <!ELEMENT author (#PCDATA)> <!-- Version string --> <!ELEMENT version (#PCDATA)> <!-- Description string --> <!ELEMENT description (#PCDATA)>
<!-- cache-mode datatype and values --> <!ENTITY % cache-mode "(disabled | simple | locked | distributed)"> <!ENTITY % property-cache-mode "(disabled | inherit)"> <!-- Item descriptors --> <!ELEMENT item-descriptor ((property | table | attribute | named-query)*, rql-filter?, (property | table | attribute | named-query)*)> <!ATTLIST item-descriptor name CDATA #REQUIRED display-name CDATA #IMPLIED display-name-resource CDATA #IMPLIED default %flag; "false" super-type CDATA #IMPLIED sub-type-property CDATA #IMPLIED sub-type-value copy-from content folder CDATA CDATA %flag; %flag; #IMPLIED #IMPLIED "false" "false"
use-id-for-path %flag; "false" content-name-property CDATA #IMPLIED content-path-property CDATA #IMPLIED content-property CDATA #IMPLIED content-length-property CDATA #IMPLIED folder-id-property CDATA #IMPLIED last-modified-property CDATA #IMPLIED display-property CDATA #IMPLIED version-property CDATA #IMPLIED hidden expert writable %flag; "false" %flag; "false" %flag; "true" #IMPLIED CDATA #IMPLIED "simple" #IMPLIED
%cache-mode;
176
12 - SQL Repository Reference
id-space-names
text-search-properties CDATA #IMPLIED item-cache-size CDATA #IMPLIED item-cache-timeout CDATA item-expire-timeout CDATA #IMPLIED query-cache-size CDATA #IMPLIED query-expire-timeout CDATA id-separator CDATA versionable %flag;
layer-property CDATA #IMPLIED version-deleted-property CDATA #IMPLIED working-version-property CDATA #IMPLIED > <!-- Property tag - defines one property descriptor for an item descriptor --> <!ELEMENT property (derivation?, (option | attribute)*)> <!ATTLIST property name column-name CDATA CDATA #REQUIRED #IMPLIED
column-names CDATA #IMPLIED property-type CDATA #IMPLIED data-type data-types item-type sql-type CDATA #IMPLIED CDATA #IMPLIED CDATA #IMPLIED CDATA #IMPLIED
sql-types CDATA #IMPLIED component-item-type CDATA #IMPLIED component-data-type CDATA #IMPLIED display-name CDATA #IMPLIED display-name-resource CDATA #IMPLIED description CDATA #IMPLIED description-resource CDATA #IMPLIED required %flag; "false" readable writable queryable default hidden expert editor-class category cascade repository cache-mode group > <!-- Derived properties have an associated derivation which specifies how the dervied property values are derived --> %flag; "true" %flag; "true" %flag; "true" CDATA #IMPLIED %flag; "false" %flag; "false" CDATA CDATA CDATA #IMPLIED #IMPLIED CDATA #IMPLIED #IMPLIED
category-resource
177
12 - SQL Repository Reference
<!ELEMENT derivation (expression+)> <!ATTLIST derivation method user-method override-property > <!-- A derived property expression, when evaluated specifies a value used in deriving a derived property value --> <!ELEMENT expression (#PCDATA)> <!-- Defines a table for an item descriptor --> <!ELEMENT table (property | attribute)*> <!ATTLIST table name CDATA #REQUIRED CDATA #IMPLIED (primary|auxiliary|multi) CDATA CDATA CDATA #IMPLIED #IMPLIED #IMPLIED multi-column-name type id-column-name id-column-names layer-column-name > CDATA CDATA CDATA #IMPLIED #IMPLIED #IMPLIED
"auxiliary"
<!-- Options are possible values for enumerated attributes --> <!ELEMENT option EMPTY> <!ATTLIST option value resource bean code CDATA CDATA CDATA CDATA #IMPLIED #IMPLIED #IMPLIED #IMPLIED>
<!-- The attribute tag is used to specify the list of feature descriptor values --> <!ELEMENT attribute EMPTY> <!ATTLIST attribute name value bean data-type CDATA CDATA CDATA CDATA #REQUIRED #IMPLIED #IMPLIED #IMPLIED>
<!-- this tag specifies an RQL statement to be used as a filter for an item descriptor --> <!ELEMENT rql-filter (rql,param*)> <!-- RQL query string itself --> <!ELEMENT rql (#PCDATA)> <!-- RQL query parameters --> <!ELEMENT param EMPTY>
178
12 - SQL Repository Reference
<!-- The named-query element. This specifies an association between a user-defined name and a Query representation --> <!ELEMENT named-query (rql-query | sql-query)> <!-- The rql-query element. Identifies an association between a user-defined name and an RQL query string, that can later be retrieved by name from the corresponding repository view that this tag is found under --> <!ELEMENT rql-query (query-name, rql)> <!ELEMENT sql-query (query-name, sql, returns?, input-parameter-types?, dependencies?)> <!ELEMENT sql (#PCDATA)> <!ATTLIST sql stored-procedure %flag; #IMPLIED > <!ELEMENT returns (#PCDATA)> <!ELEMENT input-parameter-types (#PCDATA)> <!ELEMENT dependencies (#PCDATA)> <!-- The query-name element, which indicates the user-defined name of a named query instance --> <!ELEMENT query-name (#PCDATA)> <!-- The transaction element. add-item, print-item etc. It surround the operation elements Note that add-item tags in this element
are processed one at a time. They cannot make forward references to other items and no attempt is made to satisfy database integrity constraints (beyond that automatically done with the cascade operator) Use the import-items tag if you want to load in items with forward references. --> <!ELEMENT transaction (add-item | update-item | print-item | remove-item | transaction | query-items | remove-all-items | export-items | load-items | rollback-transaction)*> <!-- The workspace element. In versioning repositories, it sets the current workspace used to process operation elements. It surrounds the these operation elements: add-item, print-item, ... --> <!ELEMENT workspace (add-item | update-item | print-item | remove-item | transaction | query-items | remove-all-items | export-items | load-items)*> <!ATTLIST workspace
179
12 - SQL Repository Reference
id CDATA #REQUIRED> <!-- The add-layer element. and branches. --> <!ELEMENT add-layer EMPTY> <!ATTLIST add-layer id type previous-layer-id <!-- The checkin-all element. versions in a workspace. --> <!ELEMENT checkin-all EMPTY> <!ATTLIST checkin-all workspace-id comment <!-The import-items element. CDATA CDATA CDATA
#REQUIRED
(workspace|branch) #REQUIRED CDATA #REQUIRED> This tag is used to check in all the working
#REQUIRED #REQUIRED>
These
tags can contain forward references. passes - pass one creates all items.
The tags are processed in three Pass two, sets required properties Pass three
and optional properties which do not refer to other items. sets the remaining properties and updates the item --> <!ELEMENT import-items (add-item)*> <!-- Procedural tags for adding and modifying items --> <!ELEMENT add-item (set-property*)> <!ATTLIST add-item item-descriptor CDATA id CDATA tag on-commit skip-add repository no-checkin > <!-- Procedural tags for adding and modifying items --> <!ELEMENT update-item (set-property*)> <!ATTLIST update-item item-descriptor CDATA id tag skip-update > <!-- Procedural tag for removing an item --> <!ELEMENT remove-item EMPTY> <!ATTLIST remove-item item-descriptor CDATA #REQUIRED CDATA CDATA CDATA #REQUIRED #IMPLIED #IMPLIED #IMPLIED CDATA CDATA CDATA CDATA %flag; #REQUIRED #IMPLIED #IMPLIED #IMPLIED #IMPLIED #IMPLIED "false"
180
12 - SQL Repository Reference
id
tag CDATA remove-references-to %flag; > <!-Procedural tag for removing all items.
property atg.allowRemoveAllItems is set on startup --> <!ELEMENT remove-all-items EMPTY> <!ATTLIST remove-all-items item-descriptor CDATA > <!-- Procedural tag for exporting the data required to recreate one or more item-descriptors. The item-descriptors attribute specifies a comma If none are separated list of one or more item descriptor names. specified, all item-descriptors are exported --> <!ELEMENT export-items EMPTY> <!ATTLIST export-items item-descriptors CDATA skip-references > <!-- Procedural tag for querying and printing an item --> <!ELEMENT query-items (#PCDATA)> <!ATTLIST query-items item-descriptor CDATA query print-content quiet id-only > <!-- Procedural tag for caching a list of items --> <!ELEMENT load-items (#PCDATA)> <!ATTLIST load-items item-descriptor CDATA properties CDATA load-all-items quiet > <!-- Procedural tag for printing an item --> <!ELEMENT print-item EMPTY> <!ATTLIST print-item item-descriptor CDATA path CDATA folder id CDATA CDATA #IMPLIED #IMPLIED #IMPLIED #IMPLIED %flag; %flag; #REQUIRED #IMPLIED "false" "false" CDATA CDATA %flag; %flag; #REQUIRED #IMPLIED #IMPLIED "false" "false" %flag; #IMPLIED "false" #REQUIRED
181
12 - SQL Repository Reference
tag print-content > CDATA CDATA #IMPLIED #IMPLIED <!ATTLIST set-property name value add remove > <!-- Sets a property value. CDATA CDATA %flag; %flag; #REQUIRED #IMPLIED "false" "false"
<!-- Sets a property value. Used only in the add-item and update-item tags --> <!ELEMENT set-property (#PCDATA)>
<!ELEMENT rollback-transaction EMPTY> <!-- Procedural tag for printing the DDL needed --> <!ELEMENT print-ddl EMPTY> <!ATTLIST print-ddl database-name CDATA > <!-- Procedural tag for dumping the caches of one or more item-descriptors. The item-descriptors attribute specifies a comma separated list of one or more item descriptor names. If none are specified, all of the caches for the repository are dumped. The dump-type attribute specifies if the output should be formatted as a list of item ids or as XML that can be later used to pre-cache the items --> <!ELEMENT dump-caches EMPTY> <!ATTLIST dump-caches item-descriptors CDATA #IMPLIED dump-type (debug|queries|both) "debug" > #IMPLIED
182
12 - SQL Repository Reference
Repository Example C: One-to-Many with an Array maps a repository item to a primary table and a multi-value table using an array property. This demonstrates a one-tomany relationship. The multi table, named subjects_tbl, contains a list of a users favorite subjects (simple strings). When using an array or list type property, the multi table requires a multi-column-name attribute (in this example, seq_num) to ensure that the ordering of the multi-values are maintained. Repository Example D: One-to-Many with a Set maps a repository item to a primary table and a multi-value table using a set type property. This is another example of oneto-many relationship. Since we are using a set, we are not required to use a multicolumn-name attribute. Repository Example E: One-to-Many with a Map maps a repository item to a primary table and a multi-value table using a map property. When using a map type property, the multi table requires a multi-column-name attribute (in this example, card_key). This column contains keys that uniquely identify each of the multi-values. For example, each user has many credit cards; the keys are strings that identify each of the users cards (like business card, frequent flyer card, personal card. Repository Example F: One-to-Many Mapping to Other Repository Items maps a oneto-many relationship. It defines two item types, user and address. Each user can have many addresses. Repository Example G: Ordered One-to-Many demonstrates an ordered one-to-many relationship using a list type property. It defines two item types, author and book. Each author can have many books, and the order of the books is considered significant. Repository Example H: Many-to-Many maps a many-to-many relationship. It defines two item types, user and address. Each user can have many addresses. Many users may live at the same address. Repository Example I: Multi-Column Repository IDs demonstrates the use of composite repository IDs.
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <!DOCTYPE gsa-template PUBLIC "-//Art Technology Group, Inc.//DTD Dynamo Security//EN" "http://www.atg.com/dtds/gsa/gsa_1.0.dtd"> <gsa-template> <header> <name>Repository Example Version A</name> <author>Pat Durante</author> <description> This template maps a repository item to a single table row. Just a primary table...no joins with
183
12 - SQL Repository Reference
other tables. </description> </header> Simplest case. <property name="id" data-type="string"/>
<item-descriptor name="user" default="true"> <table name="usr_tbl" type="primary" id-column-names="id"> <property name="name" column-names="nam_col" data-type="string"/> <property name="age" column-names="age_col" data-type="int"/> </table> </item-descriptor> </gsa-template>
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <!DOCTYPE gsa-template PUBLIC "-//Art Technology Group, Inc.//DTD Dynamo Security//EN" "http://www.atg.com/dtds/gsa/gsa_1.0.dtd"> <gsa-template> <header> <name>Repository Example Version B</name> <author>Pat Durante</author> <description> This template maps a repository item to a primary table and an auxiliary table (a one-to-one relationship.) function. </description> </header> <item-descriptor name="user" default="true"> <table name="usr_tbl" type="primary" id-column-names="id"> Each user has a job title and
184
12 - SQL Repository Reference
<property name="id" data-type="string"/> <property name="name" column-names="nam_col" data-type="string"/> <property name="age" column-names="age_col" data-type="int"/> </table> <table name="job_tbl" type="auxiliary" id-column-names="id"> <property name="function"/> <property name="title"/> </table> </item-descriptor> </gsa-template>
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <!DOCTYPE gsa-template PUBLIC "-//Art Technology Group, Inc.//DTD Dynamo Security//EN" "http://www.atg.com/dtds/gsa/gsa_1.0.dtd"> <gsa-template> <header> <name>Repository Example Version C</name> <author>Pat Durante</author> <description> This template maps a repository item to a primary
185
12 - SQL Repository Reference
(simple strings). multi-values are maintained. </description> </header>
table and a multi-value table using an array property. A one-to-many relationship. The "multi" table contains a list of a user's favorite subjects When using an "array" property, the "multi" table requires a "multi-column-name" (e.g., seq_num) to ensure that the ordering of the
<item-descriptor name="user" default="true"> <table name="usr_tbl" type="primary" id-column-names="id"> <property name="id" data-type="string"/> <property name="name" column-names="nam_col" data-type="string"/> <property name="age" column-names="age_col" data-type="int"/> </table> <table name="subjects_tbl" type="multi" id-column-names="id" multi-column-name="seq_num"> <property name="favoriteSubjects" column-names="subject" data-type="array" component-data-type="string"/> </table> </item-descriptor> </gsa-template>
186
12 - SQL Repository Reference
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <!DOCTYPE gsa-template PUBLIC "-//Art Technology Group, Inc.//DTD Dynamo Security//EN" "http://www.atg.com/dtds/gsa/gsa_1.0.dtd"> <gsa-template> <header> <name>Repository Example Version D</name> <author>Pat Durante</author> <description> This template maps a repository item to a primary table and a multi-value table using a set property. A one-to-many relationship. Since we are using a Demonstrates that D5 "set", we are not required to use a "multi-column-name" attribute. "seq_num" column. </description> </header> <item-descriptor name="user" default="true"> <table name="usr_tbl" type="primary" id-column-names="id"> <property name="id" data-type="string"/> <property name="name" column-names="nam_col" data-type="string"/> <property name="age" column-names="age_col" data-type="int"/> </table> <table name="subjects_tbl" type="multi" id-column-names="id"> <property name="favoriteSubjects" column-names="subject" data-type="set" component-data-type="string"/> </table> </item-descriptor> </gsa-template> repositories (unlike D4.5) do not require a
CREATE TABLE usr_tbl ( id nam_col age_col primary key(id) ); CREATE TABLE subjects_tbl (
187
12 - SQL Repository Reference
id VARCHAR(32) subject VARCHAR(32) primary key(id, subject) );
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <!DOCTYPE gsa-template PUBLIC "-//Art Technology Group, Inc.//DTD Dynamo Security//EN" "http://www.atg.com/dtds/gsa/gsa_1.0.dtd"> <gsa-template> <header> <name>Repository Example Version E</name> <author>Pat Durante</author> <description> This template maps a repository item to a primary table and a multi-value table using a map property. A one-to-many relationship. When using a "map" property, the "multi" table requires a "multi-column-name" (e.g., card_key). This column will contain keys that uniquely identify each of the multi-values (For example, each user has many credit cards...the keys are strings that identify each of the user's cards (like business card, frequent flyer card, personal card.) </description> </header> <item-descriptor name="user" default="true"> <table name="usr_tbl" type="primary" id-column-names="id"> <property name="id" data-type="string"/> <property name="name" column-names="nam_col" data-type="string"/> <property name="age" column-names="age_col" data-type="int"/> </table> <table name="credit_card_tbl" type="multi" id-column-names="id" multi-column-name="card_key"> <property name="card_num" column-names="card_num" data-type="map" component-data-type="string"/> </table> </item-descriptor> </gsa-template>
188
12 - SQL Repository Reference
CREATE TABLE usr_tbl ( id nam_col age_col primary key(id) ); CREATE TABLE credit_card_tbl ( id card_key card_num ); CREATE INDEX credit_card_tbl_idx ON credit_card_tbl(id); VARCHAR(32) VARCHAR(32) VARCHAR(32) not null references usr_tbl(id), not null, null, VARCHAR(32) null, INTEGER null,
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <!DOCTYPE gsa-template PUBLIC "-//Art Technology Group, Inc.//DTD Dynamo Security//EN" "http://www.atg.com/dtds/gsa/gsa_1.0.dtd"> <gsa-template> <header> <name>Repository Mapping Example Version F</name> <author>Ben Erwin</author> <description> This template maps out a one-to-many relationship between user items and address items. It demonstrates the use of the component-item-type attribute (which allows one repository item to contain other repository items.) Each user item will contain many address items (home address, business address, etc.) </description> </header> <item-descriptor name="address">
189
12 - SQL Repository Reference
<property name="city" data-type="string"/> </table> </item-descriptor> <item-descriptor name="user" default="true">
<table name="addr_tbl" type="primary" id-column-name="addr_id"> <property name="user" column-name="user_id" item-type="user"/> <property name="street" data-type="string"/>
<table name="usr_tbl" type="primary" id-column-name="id"> <property name="id" data-type="string"/> <property name="name" column-name="nam_col" data-type="string"/> <property name="age" column-name="age_col" data-type="string"/> </table> <table name="addr_tbl" type="multi" id-column-name="user_id"> <property name="addresses" column-name="addr_id" data-type="set" component-item-type="address"/> </table> </item-descriptor> </gsa-template>
<item-descriptor name="author"> <table name="author" type="primary" id-column-name="author_id"> </table> <table name="book" type="multi" id-column-name="author_id"
190
12 - SQL Repository Reference
multi-column-name="sequence_num"> <property name="books" data-type="list" component-item-type="book" column-name="book_id"/> </table> </item-descriptor> <item-descriptor name="book"> <table name="book" type="primary" id-column-name="book_id"> <property name="author" item-type="author" column-name="author_id"/> <property name="seq" data-type="int" column-name="sequence_num"/> </table> </item-descriptor>
Note some limitations for this data model: You need to use the List data type to represent the ordered many side of the relationship. The sequence_num and author_id columns in the book table cant be specified as not null, since the SQL Repository will try to set these fields to null when items in the List are removed. The book item descriptor needs to define a property to point to the sequence_num field, like this:
<property name="seq" data-type="int" column-name="sequence_num"/>
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <!DOCTYPE gsa-template PUBLIC "-//Art Technology Group, Inc.//DTD Dynamo Security//EN"
191
12 - SQL Repository Reference
"http://www.atg.com/dtds/gsa/gsa_1.0.dtd"> <gsa-template> <header> <name>People Repository Version H</name> <author>Pat Durante</author> <description>
This template maps out a many-to-many relationship between user items and address items. Each user can have many addresses. same address. </description> </header> <item-descriptor name="address"> <table name="addr_tbl" type="primary" id-column-names="address_id"> <property name="street" data-type="string"/> <property name="city" data-type="string"/> </table> <table name="user_address_tbl" type="multi" id-column-names="addr_id"> <property name="users" column-names="user_id" data-type="set" component-item-type="user"/> </table> </item-descriptor> <item-descriptor name="user" default="true"> <table name="usr_tbl" type="primary" id-column-names="id"> <property name="id" data-type="string"/> <property name="name" column-names="nam_col" data-type="string"/> <property name="age" column-names="age_col" data-type="int"/> </table> <table name="user_address_tbl" type="multi" id-column-names="user_id"> <property name="addresses" column-names="addr_id" data-type="set" component-item-type="address"/> </table> </item-descriptor> </gsa-template> Many users may live at the
192
12 - SQL Repository Reference
VARCHAR(32) null, null, not null references addr_tbl(address_id), not null references usr_tbl(id), VARCHAR(32)
street
city VARCHAR(32) primary key(addr_id) ); CREATE TABLE user_address_tbl ( addr_id VARCHAR(32) user_id ); primary key(addr_id, user_id)
CREATE INDEX user_address_tbl_user_idx ON user_address_tbl(user_id); CREATE TABLE usr_tbl ( id nam_col age_col primary key(id) ); VARCHAR(32) VARCHAR(32) INTEGER not null, null, null,
<item-descriptor name="typeX" id-separator=":"> <table name="TYPEX" type="primary" id-column-names="TYPEX_ID"> <property name="id" column-names="TYPEX_ID" data-type="string" /> <property name="name" column-names="NAME" data-type="string" /> </table> <table name="TYPEXY" type="multi" id-column-names="TYPEX_ID"> <property name="typeXYs" component-item-type="typeXY" column-names="TYPEX_ID,TYPEY_ID" data-type="set" /> </table> </item-descriptor> <item-descriptor name="typeY" id-separator=":"> <table name="TYPEY" type="primary" id-column-names="TYPEY_ID"> <property name="id" column-names="TYPEY_ID" data-type="string" /> <property name="name" column-names="NAME" data-type="string" /> </table> </item-descriptor> <item-descriptor name="typeZ" id-separator=":"> <table name="TYPEZ" type="primary" id-column-names="TYPEZ_ID"> <property name="id" column-names="TYPEZ_ID" data-type="string" /> <property name="name" column-names="NAME" data-type="string" /> </table> </item-descriptor>
193
12 - SQL Repository Reference
<item-descriptor name="typeXY" id-separator=":"> data-types="string,string" />
<table name="TYPEXY" type="primary" id-column-names="TYPEX_ID,TYPEY_ID"> <property name="id" column-names="TYPEX_ID,TYPEY_ID" <property name="name" column-names="NAME" data-type="string" /> <property name="x" column-names="TYPEX_ID" item-type="typeX" /> <property name="y" column-names="TYPEY_ID" item-type="typeY" /> </table> <table name="TYPEXYZ" type="multi" id-column-names="TYPEX_ID,TYPEY_ID"> <property name="typeXYZs" component-item-type="typeXYZ" column-names="TYPEX_ID,TYPEY_ID,TYPEZ_ID" data-type="set" /> </table> </item-descriptor> <item-descriptor name="typeXYZ" id-separator=":"> <table name="TYPEXYZ" type="primary" id-column-names="TYPEX_ID,TYPEY_ID,TYPEZ_ID"> <property name="id" column-names="TYPEX_ID,TYPEY_ID,TYPEZ_ID" data-types="string,string,string" /> <property name="name" column-names="NAME" data-type="string" /> <property name="x" column-names="TYPEX_ID" item-type="typeX" /> <property name="y" column-names="TYPEY_ID" item-type="typeY" /> <property name="z" column-names="TYPEZ_ID" item-type="typeZ" /> <property name="xy" column-names="TYPEX_ID,TYPEY_ID" item-type="typeXY" /> </table> </item-descriptor>
primary key(TYPEX_ID)
primary key(TYPEY_ID)
194
12 - SQL Repository Reference
VARCHAR(32) not null, null, VARCHAR(32) VARCHAR(32) VARCHAR(32) not null, not null, null,
TYPEZ_ID
NAME VARCHAR(32) primary key(TYPEZ_ID) ); CREATE TABLE TYPEXY ( TYPEX_ID TYPEY_ID NAME
primary key(TYPEX_ID, TYPEY_ID), foreign key (TYPEX_ID) references TYPEX(TYPEX_ID), foreign key (TYPEY_ID) references TYPEY(TYPEY_ID) ); CREATE TABLE TYPEXYZ( TYPEX_ID TYPEY_ID TYPEZ_ID NAME VARCHAR(32) VARCHAR(32) VARCHAR(32) VARCHAR(32) not null, not null, not null, null,
primary key(TYPEX_ID, TYPEY_ID,TYPEZ_ID), foreign key (TYPEX_ID) references TYPEX(TYPEX_ID), foreign key (TYPEY_ID) references TYPEY(TYPEY_ID), foreign key (TYPEZ_ID) references TYPEZ(TYPEZ_ID) );
195
12 - SQL Repository Reference
<task>commerce</task> ... </custom-admin>
The repository is displayed in the ATG Control Center under the name specified by the <display-name> tag. The repositorys repositoryName property must match the value specified by the <repositoryname> tag in the /atg/devtools/admins.xml file. For example:
<default-admin id="StandardProductCatalog" xml-combine="replace"> <display-name>Catalog Elements (En)</display-name> <task>commerce</task> <repository-name>ProductCatalog</repository-name> <folder-view>true</folder-view> <create-bean-displays> ... </create-bean-displays> <standard-bean-displays> ... </standard-bean-displays> </default-admin>
Property
Description
$class
class name
autoCommitInitialization
If setAutoCommit and
localTransactionModeInit ialization are both true,
boolean true
boolean true
database table?
196
12 - SQL Repository Reference
If restoreCacheOnRestart is true, then an XML file used to reload item caches on restart is written to this location. Should the repository caches be flushed when the data source is switched? With cacheSwitchHot=false, the repository caches are flushed when the Repository receives a
SwitchingDataSourceEvent File boolean false
cacheRestoreFile
cacheSwitchHot
of type PREPARE from the SwitchingDataSource. With cacheSwitchHot=true, the Repository pre-populates an on-deck set of caches with data from the next DataSource. See Database Switching and Repository Caches in the Copying and Switching Databases section of the Configuring Databases chapter in the ATG 6 Installation and Configuration Guide for the application server youre using.
checkTables
If set to true, then the GSARepository verifies each database table with a simple SQL query at startup. To skip the validity check and achieve faster startups, set this to false. This property is used by the startSQLRepository script. Do not change its value. This property is used by the startSQLRepository script. Do not change its value. This refers to a Data Source to use for obtaining connections. Data Sources should typically implement resource pooling for best performance.
boolean false
databaseName
databaseTableInfo
dataSource
197
12 - SQL Repository Reference
debugLevel
An integer value that indicates the detail of debugging messages printed out when the Repositorys loggingDebug property is set to true. Higher values generate more messages. The range is from 015. Note that you can also set the debug level for an individual item descriptor or property in the Dynamo Administration Interface or by using the loggingDebug attribute tag. See Debug Levels in the Development, Testing and Debugging with the SQL Repository chapter.
int 5
definitionFiles
The location of the repository definition XML files, specified as an absolute CONFIGPATH name. Dynamo uses XML file combination to collate multiple definition files into a single repository definition. If true the repository will disable all of its item caches when it starts up. This overrides all item cache size settings in the definition file. The caches can still be turned on later programmatically. This is mostly for debugging. If true the repository will disable all of its query caches when it starts up. This overrides all query cache size settings in the definition file. The caches can still be turned on later programmatically. This is mostly for debugging.
XMLFile[]
No default
disableItemCaches AtStartup
false
disableQueryCaches AtStartup
false
198
12 - SQL Repository Reference
If true, the repository checks to make sure all required properties are present when adding repository items and forbids the setting of a required property to null. The characters % and _ are typically treated as wildcards in database queries. If this property is set to true, then the GSARepository uses an escape character before % and _ in all pattern-match queries. The one exception is where we use a pattern-match query to simulate a text search query, since in that case we want to allow wildcards to be passed through. The escape character is specified by the
wildcardEscapeCharacter boolean true true
enforceRequiredProperties
escapeWildcards
The GSAEventServer component that handles cache invalidation messages for item descriptors using distributed cache mode. If you want to define profile groups or content groups, set this to the RepositoryGroups component. See Setting Up Targeting Services in the ATG 6 Personalization Programming Guide for more information about profile groups and content groups. An IdGenerator to use for generating unique IDs for items.
/atg/dynamo/server/ SQLRepositoryEventServer
groupContainer
/atg/registry/Repository Groups
idGenerator
199
12 - SQL Repository Reference
itemDescriptorAliases
A map that you can use to allow one item descriptor to be accessed by more than one name. You configure it as a Map that maps the alias to the existing item descriptor name that is its equivalent. For example:
itemDescriptorAliases= All Profiles=user
Properties
The maximum number of items to load from the database at one time. This property is consulted by getItems() and the hot cache switching logic. If true, then sorted query results are sorted in a locale sensitive manner. More specifically, String values will be compared using java.text.Collator. Since most databases cannot handle sorting with multiple locales, setting this option to true also means that the repository will perform all sorting in memory. If false, database sorting (via ORDER BY) is used where applicable and Strings are compared using String.compareTo(). If database sorting is adequate for your purposes, leaving this property set to false will result in better performance.
int 200
localeSensitiveSorting
boolean false
200
12 - SQL Repository Reference
If true, use local transaction mode for initializing the service. Some database/JDBC driver combinations require this mode for JDBC meta-data queries when the GSARepository initializes. If false, a
TransactionDemarcation boolean true
localTransactionModeIniti alization
A ClientLockManager to use for locked mode caching. See the SQL Repository Caching chapter. The name of the database account that was used to create the tables that underlie the repository. See Table Ownership Issues. The name of a metadata catalog. See Table Ownership Issues in the SQL Repository Queries chapter. Configures the syntax to use for outer joins. See Outer Joins in the SQL Repository Queries chapter in the SQL Repository Queries chapter.
ClientLockManager
metaDataSchemaPattern
String DYNAMO
metaDataCatalogName
String
outerJoinSupport
Sybase)
informix none
pathSeparator
If paths in your content folders use a separator other than the default /, set this property to the appropriate separator. Name of repository
String /
repositoryName
String SQLRepository
201
12 - SQL Repository Reference
restoreCacheOnRestart
If true, the repository automatically dumps the contents of its item caches when it is stopped and reloads the same items into the caches when it is started again. Tags that reload the caches are written into the file specified by the cacheRestoreFile property. A comma-separated list of SQL types for which the repository always uses the default JDBC type. You can set this property either to string values of SQL types, like varchar, or to the corresponding integer values specified in the class java.sql.Types (e.g. -4). If true, the Repository calls
Connection.setAutoCommit () as needed. If false, the
boolean false
safeSQLTypes
null
setAutoCommit
boolean false
repository will not call that API. Some JDBC drivers, due to bugs, may cause errors in the
GSARepository.initialize () method unless this property
is set to false. If you are using a Sybase database, this property is automatically set to true. If you need to set it to false, then set either
autoCommitInitialization
or localTransaction
ModeInitialization to false. simulateTextSearchQueries
If true, substitute pattern match queries for text search queries. This setting is not supported for production Dynamo applications. See Text Search Queries.
boolean false
202
12 - SQL Repository Reference
If true, multi-valued properties whose component type is RepositoryItem are guaranteed to not contain null entries. Instead, entries that would be null as a result of repository filtering are not placed into the multi-valued property and the property is marked read-only. The name of the SQL function to use to lower-case an expression. This is used for case-insensitive querying. If this property is null, no attempt will be made to lower-case database expressions. If true, the getItem method returns items that are cached, but not yet added. If you use distributed cache mode, Dynamo maintains an item descriptor for the das_gsa_subscriber table. This property specifies which repository that item descriptor belongs to. By default, this item descriptor is in the
/atg/dynamo/service/jdbc /SQLRepository repository. If boolean false
skipNullProperties
SQLLowerFunction
String lower
storeTransientItems
subscriberRepository
for any reason you desire to use a different repository instance, you must make sure that each repository that uses distributed cache mode has the same value for its subscriberRepository property.
synchronousInvalidation Events
For distributed cache mode, should invalidation events be sent asynchronously, for better performance, or synchronously, to avoid a slight window of stale cache?
boolean false
203
12 - SQL Repository Reference
transactionManager updateSchemaInfoCache
A TransactionManager to use for all transactions. All code in the same server should typically use the same TransactionManager. If true, then the Repository creates files that store the SQL type for each column in the database schema. If true, then the Repository attempts to optimize certain SQL delete operations based on the values in the cache. For certain usage patterns, such as when there are many multivalued properties, setting this to true can result in a significant performance gain. Set this property to true only when either (a) you have defined a version property for each item descriptor or (b) you are using locked cache mode. Setting this property causes it to be set in each of the item descriptors defined in the Repository. The Java class names of user defined property descriptors that should be loaded for this repository. User defined property descriptors register themselves in a static systemwide table. This property enables you to ensure that these classes are loaded before the repository loads any XML definitions that might refer to them.
useCacheForDelete
boolean false
userPropertyDescriptors
Class[]
204
12 - SQL Repository Reference
If useSetAsciiStream is set to true, then the SQL repository always uses setAsciiStream() instead of setString() in prepared statements. You can useSetAsciiStream instead of useSetUnicodeStream, but then you lose the ability to handle internationalized values in the database. If useSetBinaryStream is set to true, then the SQL repository always uses setBinaryStream() instead of setBytes() in prepared statements. The setBinaryStream() is required for large byte arrays in some JDBC drivers. If useSetObject is set to true, then the SQL repository always uses setObject() instead of setInt(), setFloat(), setDouble(), or setString() in prepared statements. If useSetUnicodeStream is set to true, then the SQL repository always uses setUnicodeStream() instead of setString() in prepared statements. The setUnicodeStream() method is required for large Strings in some JDBC drivers. Setting useSetUnicodeStream to true is recommended if you are using Oracle with internationalized content, but is not recommended if you do not have internationalized content in your database. Note that if you are using MS SQL Server, you must set useSetUnicodeStream to false.
boolean false
useSetAsciiStream
useSetBinaryStream
boolean false
useSetObject
boolean false
useSetUnicodeStream
boolean true
205
12 - SQL Repository Reference
useTransactionsForCached Reads wildcardEscapeCharacter
By default, the SQL repository does not use transactions when reading from the cache. This improves performance. To disable this optimization, set this property to true. This character is used in queries to escape characters that would otherwise be treated as wildcards. See the description of the escapeWildcards property. The parser used to parse the XML definition file. This value is read-only An XMLToolsFactory to use in parsing XML templates.
boolean false
XMLToDomParser
atg.xml.tools. XMLToDomParser
XMLToolsFactory
206
12 - SQL Repository Reference
A content repository is a collection of repository items that correspond to documents maintained in a hierarchical name space. A content repository typically serves as a source of content items to be displayed to a user, either directly or as an element in a page. A SQL repository implemented through the Generic SQL Adapter connector can act as a content repository, storing content items that are displayed in pages. Since the GSARepository class implements both the atg.repository.Repository interface and the atg.repository.content.ContentRepository interface, and a repository can contain multiple repository item types, a single repository can contain both content repository items (arranged in a hierarchical structure with folders that can contain repository items and other folders) and non-content repository items (arranged in a flat structure). You can use a content repository to serve targeted content, as described in the Creating Rules for Targeting Content and Setting Up Targeting Services chapters of the ATG 6 Personalization Programming Guide. A product catalog in a commerce application is also typically a content repository, as described in the Using and Extending the Default Catalog chapter of the ATG 6 Commerce Administration and Development Guide. Note that the essential feature of a content repository is that it represents a hierarchical structure of folders and repository items, like a directory structure. The repository items themselves do not necessarily represent content that is displayed in a Web application, although in most cases they will. What is significant is whether the repository items are maintained in a hierarchical structure. You can define one or more item descriptors in a SQL repository to be a content item descriptor that defines a type of ContentRepositoryItem. When you retrieve one of these items using any of the Repository method calls, the repository item implements the atg.repository.content.ContentRepositoryItem interface. You can have other item descriptors in the same repository that do not implement this interface and do not define content items. The Repository Loader is a utility that handles the work of creating and updating content repository items from documents on your file system. The repository template can be configured so that the loader assigns the values of your content repository items properties from selected portions of these documents while still allowing access to the entire document. These properties include metadata about the document file such as its length and the time it was last modified. The Repository Loader can be configured to periodically scan the file system and synchronize it with the repository representation, adding, updating and deleting content repository items as necessary. See the Repository Loader chapter for more information.
207
13 - SQL Content Repositories
storys byline, dateline, length, and keywords, while the content would include the text of the story itself. You can adopt one of two basic architectural styles when you set up a SQL content repository: You can store both the content and the metadata in your SQL database. A content repository item would include a property whose value was the content. You can store the metadata in the database, and the content in your file system. In this case, the metadata would include properties that indicate how to look up the content in the file system. A content repository item would include a property whose value was a pointer to the content in the file system.
As with other repositories, setting up a SQL content repository involves the following steps: 1. Design the item types you want to include in your content repository. For each type of repository item, decide what sorts of properties you want to have available to you for searching and targeting content in the repository. Set up a SQL database containing content repository items, to act as the data store of the repository. Create a repository definition. This is an XML file that describes the repositorys item descriptors and property descriptors, and defines the relationship among these items and the rows and tables of the database. See Creating a SQL Content Repository Definition. Configure a SQL Repository component that interacts with the data store you set up in step 2 to create, modify, and retrieve repository items. See Configuring a SQL Content Repository.
2. 3.
4.
A repository that contains content items must include item descriptors flagged with the folder and content attributes of the <item-descriptor> tag in the SQL repository definition.
208
13 - SQL Content Repositories
Items defined by the content item descriptor implement the atg.repository.content.ContentRepositoryItem interface. Items defined by the folder item descriptor implement the atg.repository.content.FolderItem interface, as well as the atg.repository.MutableRepositoryItem interface.
Regardless of how you store path information in the database, you can get the path of an item with this method in the atg.repository.content.FolderItem interface (which is extended by the ContentRepositoryItem interfaces):
public String getItemPath()
This method returns the path of this item, represented as a relative path from the repositorys root folder
use-id-for-path
This is the simplest mechanism. In this mode, the relative path name of the item is used as the ID of the repository item. Your database must then include an ID column that is a string large enough to hold the entire relative path name of each folder item and content item. Put this ID column in your primary table and set the id-column-name attribute of the primary table to point to this column. You then set useid-for-path="true" for that item descriptor. For example:
<item-descriptor name="folder" folder="true" use-id-for-path="true" folder-id-property="folder-id"> <table name="folder" id-column-name="id"> <property name="id" column-names="id"/> <property name="folder-id" column-names="folder_id"/> ... </table> </item-descriptor>
209
13 - SQL Content Repositories
<property name="id" column-names="id"/> ... </table> </item-descriptor>
<item-descriptor name="article" content="true" use-id-for-path="true" folder-id-property="folder-id"> <table name="articles" id-column-names="id"> <property name="folder-id" column-names="folder_id"/>
The use-id-for-path mode may not work if you have an existing database schema that does not follow this format. This approach also might not work if path names in your repository could be longer than the size of varchar you can efficiently store and query against in your database. Some databases impose a 255-character limit on the size of queryable columns. This may be too small to hold the entire path for some content repositories. Note that even though you put the entire path name in the property designated by the id-columnnames attribute, you still need to use the folder-id-property attribute to designate a property that holds the name of the parent folder of the item. In the preceding example, the folder-id property holds the name of the folder.
content-name-property
You can set the item descriptors content-name-property attribute. In this case, you can store just the name of the repository item itself (rather than the entire path name) in one property of the repository item and use the content-name-property to designate the name of this property. The content-nameproperty specifies the property representing the name of the folder item or content item, while the folder-id-property specifies the property representing the parent folder of the folder item or content item. From these two pieces of information, we can compute the path for a given item by walking up the content hierarchy. The operation of computing the path for this item will be more expensive in this mode, since we have to query up the folder hierarchy to compute the path for an item rather than getting the path from a single property. However, this mode can overcome the problem of the size limitation on queryable columns. Now your column size for the content name limits the size of each individual component of the file name, not the size of the entire path. A folder-id-property is required for all content repositories, whichever method they use to specify how path information is stored in the database. The data-type of the folder-id-property can be either data-type="string" (or whatever type you define your repository IDs to be), or you can specify that its item-type is the name of the folder item descriptor. This enables you to conveniently access folder information from the item itself. For example:
<item-descriptor name="folder" folder="true" content-name-property="filename" folder-id-property="folder-id"> <table name="folder" id-column-names="id"> <property name="filename" data-type="string"/> <property name="folder-id" item-type="folder"/> ... </table> </item-descriptor>
210
13 - SQL Content Repositories
Since this content-name property is not guaranteed to be unique across the repository, you have a separate column for the ID of this repository
content-path-property
You might not be able to use the repository items path as its repository ID. If that is the case, perhaps due to references to these rows from other tables, and if you can store the entire path name of the item as a column in your table, then you can use a third alternative. In this mode, you can set the content-pathproperty to refer to a property that holds the path name of the item. You would then use a separate property and column in your table to refer to the ID for this item. For example:
<item-descriptor name="folder" folder="true" content-path-property="pathname"> <table name="folder" id-column-names="id"> <property name="id" data-type="long"/> <property name="pathname" data-type="string"/> ... </table> </item-descriptor>
211
13 - SQL Content Repositories
<item-descriptor name="files" content-length-property="length" last-modified-property="lastModified" content-property="data"> <table name="media_files" type="auxiliary" id-column-names="media_id"> <property name="length" data-type="long" column-names="length"/> <property name="lastModified" data-type="timestamp" column-names="last_modified"/> <property name="data" data-type="binary" column-names="data"/> </table> </item-descriptor>
You configure your content item descriptors by naming the properties that are to be used to retrieve each of these values. This is done using the following attributes in the <item-descriptor> tag:
Attribute Name
content-path-property content-name-property
Description Specifies the ID of the folder containing this folder or content item. Refers to the name of this folder or content item (not the full path name) For content item descriptors, this is the name of the property that holds the content itself. For content item descriptors, this optionally is used to specify a property that can be used to retrieve the last modified time for that piece of content.
content-property
last-modifiedproperty
212
13 - SQL Content Repositories
<item-descriptor name="book" display-property="title" content="true" content-property="bookcover_image" content-path-property="filePath" folder-id-property="parentFolder"> <table name="book_info" id-column-name="id" type="primary"> <property name="filePath" data-type="big string"/> <property name="parentFolder" item-type="book_folder"/> <property name="title" data-type="big string"/> <property name="author" data-type="big string"/> </table> <property name="bookcover_image" property-type="atg.repository.FilePropertyDescriptor"/> </item-descriptor>
213
13 - SQL Content Repositories
property-type="atg.repository.FilePropertyDescriptor"
As described in the Storing Content on a File System section, this property type indicates that the repository should use the propertys path name and convert it to a java.io.File object.
<item-descriptor name="book_folder" display-property="folderPath" folder="true" content-path-property="folderPath" folder-id-property="parentFolder"> <table name="book_folder" id-column-name="id" type="primary"> <property name="parentFolder" item-type="book_folder"/> <property name="folderPath" data-type="big string"/> </table> </item-descriptor>
This item type is specified to be a folder, using the attribute folder="true". The folder-id-property attribute in the item-descriptor tag indicates that this item stores its parent folder ID in the database using a property named parentFolder. Both the book and the book_folder item type stores their paths in the database using the contentpath-property attribute. The content-path-property attribute indicates the property of this item that defines the absolute path name of this item in the folder hierarchy. In this example, the path is stored in the property named folderPath. If our example had especially deep hierarchies, resulting in excessively long path names, we might instead store just the name of this item using the content-nameproperty attribute, and have the repository calculate the items absolute path by determining its parent folders, using the property indicated by the folder-id-property attribute.
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <!DOCTYPE gsa-template PUBLIC "-//Art Technology Group, Inc.// DTD Dynamo Security//EN" "http://www.atg.com/dtds/gsa/gsa_1.0.dtd"> <gsa-template> <item-descriptor name="book_folder" display-property="folderPath" folder="true" content-path-property="folderPath"
214
13 - SQL Content Repositories
folder-id-property="parentFolder">
<table name="book_folder" id-column-name="id" type="primary"> <property name="parentFolder" item-type="book_folder"/> <property name="folderPath" data-type="big string"/> </table> </item-descriptor> <item-descriptor name="book" display-property="title" content="true" content-property="bookcover_image" content-path-property="filePath" folder-id-property="parentFolder"> <table name="book_info" id-column-name="id" type="primary"> <property name="filePath" data-type="big string"/> <property name="parentFolder" item-type="book_folder"/> <property name="title" data-type="big string"/> <property name="author" data-type="big string"/> </table> <property name="bookcover_image" property-type="atg.repository.FilePropertyDescriptor"/> </item-descriptor> </gsa-template>
-- drop table book_folder; -- drop table book_info; CREATE TABLE book_folder ( id parentFolder folderPath primary key(id) ); CREATE TABLE book_info ( id filePath parentFolder title VARCHAR(32) long varchar VARCHAR(32) long varchar not null, null, null references book_folder(id), null, VARCHAR(32) VARCHAR(32) long varchar not null, null references book_folder(id), null,
215
13 - SQL Content Repositories
author primary key(id) ); long varchar null,
2.
3.
// repository id of the item we want to get String id = "1001"; // name of item descriptor describing the type of item we want String descriptorName = "book"; // get the item from the repository
216
13 - SQL Content Repositories
RepositoryItem item = pRepository.getItem(id, descriptorName); // make sure we have an item if (item == null) { pln("Item not found, descriptor=" + descriptorName + ", id=" + id); return; } // get the author property of the item String author = (String)item.getPropertyValue("author");
217
13 - SQL Content Repositories
218
13 - SQL Content Repositories
14 Repository Loader
In many applications, it makes sense from a workflow and production point of view to create the content of repository items outside of ATG, on a file system, and subsequently load the items into the repository. The ATG Repository Loader provides a flexible way to take files that are stored in a file system, convert them into repository items, and load the items into the repository. In ATG 6, the Repository Loader has been implemented to handle the loading of HTML files, XML files, and binary media items (like image or audio files) into a SQL repository. The Repository Loader uses ATGs xml2repository system to transform files into XML files that use the xml2repository schema, which are then transformed into repository items. The xml2repository system is described in full in the Repository to XML Data Binding chapter of the ATG 6 Web Services and Integration Framework Guide. Make sure you are familiar with that chapter, as it is essential to understanding how to use the Repository Loader. The Repository Loader has been designed to be flexible. When you configure the Repository Loader, you can choose to let the Repository Loader scan the file system, identify the files that need to be loaded, and load the files, either on command or on a specified schedule. Alternatively, you can submit to the Repository Loader a manifest file that specifies the files to be loaded. The first alternative requires less effort on the part of the Web site administrators; the second alternative loads faster and puts less of a burden on your system. Which to choose depends in part on the size of your repository. The larger the number of files that need to be scanned, the longer it will take to load them, and the greater the burden that file system scanning will place on your system. This chapter includes the following sections: Repository Loader Architecture Using the Repository Loader Configuring the Repository Loader Components Repository Loader Example
219
14 - Repository Loader
LoaderManager Type Mapper and TypeMappings Content Handlers and Back Ends User Interface Elements A fully-configured Repository Loader setup might look like this:
RLClient
FileSystemMonitorService
ContentHandler TypeMappings
Add
Update
Remove
xml2repository services
Repository
Data base
220
14 - Repository Loader
FileSystemMonitorService
The FileSystemMonitorService implements the interfaces atg.repository.loader.FileSystemMonitor and atg.repository.loader.FileSystemManager. A component of the FileSystemMonitorService class can be configured to scan a specified file directory and identify the files that need to be uploaded. You can limit the files to be uploaded in three ways, so that the files include: only files in a specified folder (including, optionally, its subfolders); only files modified since the time the file system was last scanned; only files with specified file extensions.
Once the files have been identified by the FileSystemMonitorService component, the list of files is passed to a component of the LoaderManager class. See Configuring the FileSystemMonitorService for details about this components properties.
FileSystemMonitorScheduler
The FileSystemMonitorService can be set to scan the file system on a periodic schedule. The schedule is set and controlled in a FileSystemMonitorScheduler component. Note that the FileSystemMonitorService can also be set to scan the file system programmatically, using its add, delete, and update methods, or through the Repository Loader administration application, which provides JSP pages and formhandlers that invoke FileSystemMonitorService methods. See Repository Loader Formhandlers and Administration Pages. See Configuring the FileSystemMonitorScheduler for details about this components properties.
LoaderManager
The LoaderManagerImpl implements the interface atg.repository.loader.LoaderManager. A component of the LoaderManagerImpl class accepts Repository Loader jobs, initiated either by the FileSystemMonitorService or, if you are supplying a manifest of files to be loaded, from the Repository Loaders RMI client (see Repository Loader RMI Client). The LoaderManger component passes the files to a TypeMapper component, which determines how the file should be processed. Or, the TypeMapping can be specified directly or in a manifest file. See Configuring the Loader Manager for details about this components properties.
221
14 - Repository Loader
Content Handlers and Back Ends
Files are routed to ContentHandlers, based on the mapping of file types and item descriptors established by the TypeMappings. The ContentHandler is responsible for carrying out the process of transforming the file into a repository item and invoking a back end system to perform add, update, and remove operations on the repository. In ATG 6, a single class of ContentHandler is included named Xml2RepositoryContentHandler. This class uses ATGs xml2repository system to transform files into XML files that use the xml2repository schema, which are then transformed into repository items. The xml2repository system is described in full in the Repository to XML Data Binding chapter of the ATG 6 Web Services and Integration Framework Guide. Make sure you are familiar with that chapter, as it is essential to understanding how to use the Repository Loader.
For example, http://localhost:8840/rl The Repository Loader Administration consists of a set of JSP pages that use Repository Loader formhandlers. On these pages, you can create, delete, monitor, and view the results of Repository Loader jobs. The Repository Loader Administration reports on the status of Repository Loader jobs, including the success or failure, number of files, and errors encountered. Click the Get Detail link for a job to get more information. For more details about using the Repository Loader Administration, see the Repository Loader Example section.
222
14 - Repository Loader
Error Policies
The LoaderManager uses a configurable error handling policy, defined by the Repository Loaders ErrorPolicy component. Each job processed by the LoaderManager might contain files that cause exceptions to be thrown. The LoaderManager consults the ErrorPolicy component to determine how to handle exceptions. All exceptions are logged. The Repository Loader sets a success state for each job and each batch. In addition, the following ErrorPolicy methods are used to determine how to proceed after encountering a problem while processing a job:
checkIsExceptionFatal
Returns true if the provided exception should terminate processing of a job. If false, then we proceed to the checkRequiresNewTransaction method. Returns true if the provided exception requires a new transaction in order to reasonably continue. Returns true if transaction demarcations should be ended using the value of the Jobs batchFailed property. In this case, if a batch has failed, then all subsequent batches in the job will be rolled back, whether or not they contained any errors.
checkRequiresNewTransaction
checkEndTransactionWithBatchState
Make sure you have set your DYNAMO_HOME variable before you run the script. The RLClient arguments are as follows:
Argument -m <manifestFilePath>
Description The server-side path to the manifest containing the desired load commands. See Using a Repository Loader Manifest for information about the manifest file format.
223
14 - Repository Loader
-mp <manifestFilePath> -p <propertiesFilePath>
The server-side path to the manifest containing the desired load commands. Use this argument, rather than the -m argument, if your RLClient and host are running on different operating systems. The manifest path should be enclosed in double quotes. See Using a Repository Loader Manifest for information about the manifest file format. Path to a properties file containing additional parameters and hints to the LoaderManager. See RLClient Hints for more information. Optional. Name of the host where the RemoteManifestProcessor is running. RMI port of the host where the RemoteManifestProcessor is running. Optional. The port defaults to 8860. A colon-separated username:password pair. By default, the username and password is tested against the /atg/dynamo/security/AdminUserAuthority. You can specify a different user authority by setting the userAuthority property of
/atg/dynamo/repository/loader/RemoteManifestProcessorS ervice.
-h <hostname> -r <RMIPort>
-auth <username:password>
-s <servicename>
RLClient Hints
You can optionally pass the address of a properties file as an argument to the RLClient containing additional parameters and hints to be used by the LoaderManager. This hints file can include the following:
batchSize numElementsInManifest
The number of files to process in each transaction. The total number of files in the manifest
For example:
atg.repository.loader.batchSize=2 atg.repository.loader.numElementsInManifest=725
If you dont specify a batch size, the RLClient uses the LoaderManagers defaultBatchSize property.
224
14 - Repository Loader
The elements in a Repository Loader manifest file are handled in the order that they appear. As a result, you need to take care that an element does not depend on a subsequent element. For example, remember that a content repository requires a folder hierarchy; do not attempt to add a content item before you add the folder that is to contain it. The next two sections, Repository Loader Manifest File Document Type Definition and Repository Loader Manifest File Tags, provide more details about the XML format used in creating a Repository Loader manifest file.
<!-==================================================================== rl-manifest_1.0.dtd - document type for Repository Loader manifests Version: $Id: //product/DAS/main/Java/atg/dtds/rl/rl-manifest_1.0.dtd#1 $ $Change: 286550 $ ==================================================================== --> <!-- A single manifest composed of any number of add, update, remove tags --> <!ELEMENT manifest (add | update | remove)*> <!ATTLIST manifest num-elements CDATA #IMPLIED>
225
14 - Repository Loader
<!ELEMENT add (#PCDATA)> <!ATTLIST add type-mapping CDATA #IMPLIED> <!ELEMENT update (#PCDATA)> <!ATTLIST update type-mapping CDATA #IMPLIED> <!ELEMENT remove (#PCDATA)> <!ATTLIST remove type-mapping CDATA #IMPLIED>
manifest Tag
A <manifest> tag wraps the entire manifest. It contains one or more of the following child tags:
<add> <remove> <update>
The add, remove, and update tags simply contain in their bodies the pathname of the file or folder to be added, removed, or updated:
<add>/home/Dynamo/RL/sample-data/user001.xml</add> <update>/home/Dynamo/RL/sample-data/user002.xml</update>
type-mapping Attribute
An add, remove, or update tag can include an attribute named type-mapping. The value of the typemapping attribute is assumed to be the absolute Nucleus path of a component that implements atg.repository.loader.TypeMapping. If the component can be found and is of the correct class, it will be used to process the file specified in the tag.
num-elements Attribute
A manifest tag can include an attribute named num-elements. This is an optional attribute that indicates the total number of add, remove, and update elements in the manifest file.
226
14 - Repository Loader
Configuring the FileSystemMonitorService Configuring the LoaderManager Configuring the FileSystemMonitorScheduler Configuring the TypeMapper Configuring a TypeMapping Configuring the Xml2RepositoryContentHandler
Instances of these components can be found in Nucleus in the /atg/dynamo/service/loader folder. This section also describes two sample configurations: Automatic Mode Manifest Mode
Property createManifestMode
Description If true, indicates that we are processing a large file system. The FileSystemMonitorService will pass files to the LoaderManager in the form of a manifest file, rather than as an array of files. An array of file extension strings to use as filter criteria when gathering updates. Only files whose file extensions match one of the strings in this property will be scanned. Example:
filters=.html,.htm,.wml
Type, Default
boolean false
filters
String[]
includeFolders
Should we include the content folders in the scan results? This must be true in order for folders to be created as folder repository items in a content repository. The time that the
FileSystemMonitorService last
boolean true
lastScanned
227
14 - Repository Loader
manifestFile The file used if
createManifestMode="true". This
File RLxxxx.xml
references need not actually exist - it will be created if it is not found. rootPath The root path that should be monitored. All files to be scanned must be in or under this path. The Nucleus address of the TypeMapper component
(where xxxx is the unique name assigned to the file using the File.createTempFile method)
typeMapper
Property cancelledJobsFifoSize
Description The number of cancelled jobs that should be kept in the queue viewable in the Repository Loader Administration. The number of completed jobs that should be kept in the queue. The number of files to handle in a single transaction. This value can be overridden by a batch size argument provided in the LoaderManagers load() and remove() methods. A batch size of -1 means to handle the entire job in one transaction. A batch size of 0 or 1 means to treat each file as a separate transaction. The default TypeMapper the LoaderManager will use if no TypeMapper is provided in the LoadManagers load() or processManifest() methods. An array of components that listen to
JobEvents
completedJobsFifoSize
int 20 int -1
defaultBatchSize
defaultTypeMapper
jobEventListeners
atg.repository.loader. JobEventListener[]
228
14 - Repository Loader
jobIdGenerator
jobIdPrefix
jobIdSpace
The name of the IdSpace used to generate JobIds The number of threads used in the Job queue An array of components that listen to error LoaderEvents.
jobQueueSize
loaderErrorEventListeners
null loaderEventListeners An array of components that listen to add, update and remove LoaderEvents. The number of suspended jobs that should be kept in the queue Should failed jobs be suspended or cancelled?
atg.repository.loader. LoaderEventListener[]
suspendedJobsFifoSize
suspendFailedJobs
Description Boolean. Is the scheduler enabled? The location of a file that stores the time of the last scan Boolean. Should the FileSystemMonitorService recursively scan the supplied paths? The schedule on which to scan the file system. See Scheduler Services in the Core Dynamo Services chapter of the ATG 6 Dynamo Programming Guide for information about schedule formats. The Nucleus address of the Scheduler component.
schedule
scheduler
229
14 - Repository Loader
Configuring the TypeMapper
Property folderTypeMapping Description
The Nucleus address of the TypeMapping that handles folder item descriptors. This property is required if any of the files to be loaded are content item types. An array of the Nucleus address of each TypeMapping used by this TypeMapper component.
typeMappings
The ExtFilterTypeMapper class uses the extension of the file name to locate TypeMappings. It includes the following additional property:
extensions
An array of strings that correspond to the file extensions used by the TypeMapper. For example:
extensions=.xml,.html
The DirFilterTypeMapper class matches regular expressions (which correspond to directory names) to
TypeMappings. It includes the following additional property:
directories
Configuring a TypeMapping
The Repository Loader includes an implementation of the TypeMapping interface named TypeMappingImpl. You can configure an instance of TypeMappingImpl for a particular item descriptor using the following properties:
Property contentIsXML
Description Is the content of files assigned to this mapping XML? Should the content of files assigned to this mapping be parsed for property values?
parseContent
boolean
230
14 - Repository Loader
The name of the item descriptor handled by this TypeMapping. The Nucleus address of the ContentHandler component that handles content for this mapping. The Nucleus address of the
ContentRootPathProvider
itemDescriptorName
String
contentHandler
ContentHandler
contentRootPathProvider
ContentRootPathProvider
component used by this mapping, if any. encodingTyper The Nucleus address of the encoding typer used by this mapping, if any. If the item descriptor is not a content item descriptor, you need to use a repository item property to hold the file path of the items. This property is the name of that repository item property. The Nucleus address of the repository that contains the item descriptor handled by this TypeMapping. Sets a code that describes how this mapping uses ID and path properties for the purposes of item creation, update, and removal. See Setting Repository IDs.
PageEncodingTyper
pathPropertyName
String
repository
MutableRepository
updatePropertyConfiguration
231
14 - Repository Loader
Value CONTENT_ITEM_DESCRIPTOR_ID_AND_PATH_PROP
Description For content item descriptor types only, use descriptor metadata to determine which properties should be used for assigning the repository item ID and path properties. IDs may still be assigned from content data. The repository item ID property is to be set as part of the file parsing process. Since the files content uniquely and persistently defines the repository items ID no path property need be assigned. The repository item ID property is to be set as part of the file parsing process. Set the repository item property specified by the pathPropertyName property of the TypeMapping with the files path. Use a value generated by the IdGenerator for the repository item ID, using the IdGenerator specified by the TypeMappings idGenerator property. If no idGenerator is specified, errors will result. No path property will be set. As a consequence, files assigned to this mapping cannot be updated or removed using the Repository Loader. Use a value generated by the IdGenerator for the repository item ID, using the IdGenerator specified by the TypeMappings idGenerator property. If no idGenerator is specified, errors will result. Set the repository item property specified by the TypeMappings pathPropertyName property with the files path. For non-content item descriptors. Use the files path as both its repository item ID and its path value.
CONTENT_DEFINED_ID_AND_NO_PATH_PROP
CONTENT_DEFINED_ID_AND_NAMED_PATH_PROP
GENERATED_ID_AND_NO_PATH_PROP
GENERATED_ID_AND_NAMED_PATH_PROP
ID_EQUALS_FILE_PATH
Note that if a file includes a tag that corresponds to the ID property defined in an item descriptor, then the value in this tag will be used by the xml2repository function in setting the repository item ID, rather than the generated ID or the file path ID. Therefore, if you are using any of the following values for the updatePropertyConfiguration property, you need either to make sure your documents do not include an ID tag, or else set the parseContent property of the TypeMapping for these documents to false:
232
14 - Repository Loader
CONTENT_ITEM_DESCRIPTOR_ID_AND_PATH_PROP ID_EQUALS_FILE_PATH
properties:
Description The component that handles repository add operations. The component that handles repository remove operations. The component that handles repository update operations. Boolean. Is the XMLTransformer enabled? The FileMappingFilter that determines whether a file should be run through the XMLTransformer. Boolean. Should we throw an exception if a remove operation found and removed zero repository items? The name of the property to be used for queries where the updatePropertyConfiguration property of the TypeMapping is CONTENT_DEFINED_ID_AND_NO_PATH_PROP or CONTENT_DEFINED_ID_AND_NAMED_PATH_PROP.
exceptionOnZeroRemove
idPropertyNameForQueries
For more information about these components, see Repository to XML Data Binding: Repository Operations in the ATG 6 Web Services and Integration Framework Guide. The XMLTransformer is a component that can define a set of style sheets to be applied to an XML source. You can set its styleSheets property to an array of style sheet files to use in transforming XML files. An instance of the XMLTransformer can be found in Nucleus at /atg/dynamo/service/loader/typemapping/SampleXMLTransformer. This lets us transform an XML file at load time into the format required by the xml2repository system. Depending on the number and complexity of the stylesheets, this might be an expensive operation.
233
14 - Repository Loader
Using automatic mode may require less effort on the part of the Web site administrators. Using manifest mode results in faster loading and puts less of a burden on your system. If you use a content management system that can identify for you the files that have changed since your last Web site update, you should be able to generate a manifest of files to be loaded, updated, or removed fairly easily. Note that you will need to configure more than just the few properties described in this section. This section is intended only to bring out the most significant differences between these two modes of operation.
Automatic Mode
If the file system handled by the Repository Loader is comparatively small, you can configure the Repository Loader for more automatic operation. You can set the Repository Loader to scan the file system on command or according to a prescribed schedule and automatically identify the files that need to be loaded into the repository. As mentioned earlier, what constitutes a comparatively small set of files will vary, but in any case, a set of 1000 or fewer files may be considered small. To configure the Repository Loader for automatic operation, set the following properties: FileSystemMonitorScheduler:
enabled="true"
Set the schedule property to the schedule on which you want the file system to be scanned. For example:
schedule=every 2 hours in 15 minutes
Manifest Mode
Use the RLClient to load large numbers of files. Use this configuration setting: FileSystemMonitorScheduler:
enabled="false"
You also need to create a manifest file that lists all the folders and files to be loaded into the repository by the Repository Loader. See Using a Repository Loader Manifest for information about the manifest file format.
234
14 - Repository Loader
Description A content folder item type. A content item type. This is the super-type for a series of item types that inherit from the fileAsset item type. The type property specifies which subtype (textFileAsset, binaryFileAsset, htmlArticle, txtPressRelease, xmlPressRelease, gifImage, or jpgImage) an item belongs to. A fileAsset item also defines lastModified, size, and parentFolder properties.
following item types in its item descriptors. Note how the content item types use item descriptor inheritance:
textFileAsset
A content item type that inherits from fileAsset. It is designed for text files. The text content is stored in the content big string property. It has subtypes named htmlArticle, txtPressRelease, and xmlPressRelease. A content item type that inherits from fileAsset. The content is stored in the content binary property. It has subtypes named gifImage and jpgImage. A content item type that inherits from fileAsset and from
textFileAsset. It defines a published timestamp property and a keywords string property.
binaryFileAsset
htmlArticle
txtPressRelease
xmlPressRelease
gifImage
jpgImage
address
A simple non-content item type. Used by the user item types addresses property. A non-content item type. Used by the user item types contacts property. A simple non-content item type. Used by the user item types numbers property. A complex non-content item type. The user item type is described in detail in the User Example Item Type section.
contact phone
user
235
14 - Repository Loader
User Example Item Type
The user item type demonstrates a variety of data relationships. It shows how an item type can use properties that nest other item types. The user item descriptor is defined as follows:
<item-descriptor name="user" default="true"> <table name="rlex_user" type="primary" id-column-name="id"> <property name="id" data-type="string"/> <property name="name" column-name="nam_col" data-type="string"/> <property name="age" column-name="age_col" data-type="string"/> </table> <!-- a set of address items --> <table name="rlex_address" type="multi" id-column-name="user_id"> <property name="addresses" column-name="addr_id" data-type="set" component-item-type="address" cascade="delete,update"/> </table> <!-- a set of contact items --> <table name="rlex_contact" type="multi" id-column-name="user_id"> <property name="contacts" column-name="con_id" data-type="set" component-item-type="contact" cascade="delete,update"/> </table> <!-- a map of phone items --> <table name="rlex_phone" type="multi" id-column-name="user_id" multi-column-name="kind"> <property name="numbers" column-name="phone_id" data-type="map" component-item-type="phone" cascade="delete,update"/> </table> <!-- a one-to-one mapping in an aux table --> <table name="rlex_job" type="auxiliary" id-column-names="id"> <property name="jobtype"/> <property name="title"/> </table> <!-- a multivalue property (array) --> <table name="rlex_subjects" type="multi" id-column-names="id" multi-column-name="seq_num"> <property name="favoriteSubjects" column-names="subject" data-type="array" component-data-type="string"/> </table> <!-- a multivalue property (list) -->
236
14 - Repository Loader
component-data-type="string"/> -->
multi-column-name="seq_num"> <property name="worstSubjects" column-names="subject" data-type="list" </table> <!-- a multivalue property (map)
<table name="rlex_credit_card" type="multi" id-column-names="id" multi-column-name="card_key"> <property name="card_num" column-names="card_num" data-type="map" component-data-type="string"/> </table> </item-descriptor>
The user item type defines three string properties, id, name, and age, in its primary table. Contact information about the user is defined in three multi-valued properties, addresses, contacts, and numbers. The values of each of these multi-valued properties are collections of other repository items (address, contact, and phone, respectively). In addition, the user item type defines the following properties:
String properties, using a one-to-one mapping in an auxiliary table. Multi-valued collections of strings properties, using a one-to-many mapping in a multi table.
237
14 - Repository Loader
Item Type fileFolder textFileAsset htmlArticle txtPressRelease xmlPressRelease gifImage jpgImage user TypeMapping Component FolderTypeMapping UnparsedContentTypeMapping HtmlArticleTypeMapping PressReleaseTXTTypeMapping PressReleaseXMLTypeMapping GifImageTypeMapping JpgImageTypeMapping UserTypeMapping
ContentHandler Component ContentHandler ContentHandler HtmlArticleContentHandler ContentHandler ContentHandler ContentHandler ContentHandler UserContentHandler
The ContentHandler component is used for all the type mappings that dont require XSL preprocessing.
Example TypeMapper
The Repository Loader Example is configured with an ExtFilterTypeMapper component that maps file extensions to TypeMappings, using the following property configuration:
extensions=.xml,.txt,.gif,.jpg,.html,.eml typeMappings=pressReleaseXMLTypeMapping,pressReleaseTXTTypeMapping,\ gifImageTypeMapping,jpgImageTypeMapping,htmlArticleTypeMapping,\ userTypeMapping
238
14 - Repository Loader
xml2repository Schemas
The xml2repository system uses XML schemas to represent repository items as XML files. This system lets you load the repository using XML files that conform to the schema, or export existing repository items in the form of XML files that can later be loaded. The Repository Loader Example includes an XML schema for each of the following repository item types. The schemas can be found in the <ATG6dir>/RL/Example/repository2xml/schemas directory.
Dont start up any other ATG demo, example, or reference application modules at the same time, since the database connection configurations may conflict. Once the server is started, you can navigate to this page for the Repository Loader Administration user interface:
http://<hostname>:<port number>/rl
Then, use the Repository Loader Administration user interface to load the sample files included in the FileSystemMonitorService root path: <ATG6dir>/RL/Example/j2ee-apps/example/webapp\public : 1. If you are running on Windows, set the pathSeparator property of the /atg/rlexample/ExampleRepository component to \:
pathSeparator=\
Remember that \ is an escape character in properties files, so if you are setting this property in a properties file rather than in the ACC, you need to use this format:
pathSeparator=\\
239
14 - Repository Loader
2. 3. 4. Click the Create Job link. Set the Recurse option to yes. Click the Add Files button.
This loads the files from the root path into the ExampleRepository as repository items. The Repository Loader Administration should show the job as completed. In addition, the Repository Loader example shows an example of how you can use the xml2repository feature to export a repository item in the form of an XML file. Go to the following page in your browser:
http://localhost:8840/rl-example/itemAsXml.jsp
This page takes a hard-coded repository item (user001) and outputs it as an XML file. Depending on your browser settings, you will likely have to view the resulting page as source in order to see the generated XML. This demonstrates an example of the format used by the xml2repository system. Examining the XML format generated for a particular item descriptor might help you if you want to generate compliant XML or write an XSL stylesheet for the Repository Loader to apply before processing a files contents.
240
14 - Repository Loader
ATG 6 provides infrastructure and tools for accessing ATG repositories with Web services. The Creating Repository Web Services chapter in the ATG 6 Web Services and Integration Framework Guide describes how to create a Web service that exposes a particular repository item descriptor, or an individual repository item property. Repository items can be passed via a Web service in the form of an XML file created with the ATG xml2repository feature, described in the Repository to XML Data Binding chapter of the ATG 6 Web Services and Integration Framework Guide. This chapter describes three generalized Web services you can use to provide access to ATG repositories: GetRepositoryItem Web Service PerformRQLQuery Web Service PerformRQLCountQuery Web Service
These Web services are packaged in the <ATG6dir>/DAS/WebServices directory. For information about how to deploy these and other Web services, see the ATG 6 Web Services and Integration Framework Guide and the ATG 6 Installation and Configuration Guide for your application server. In addition to these repository Web services, ATG Personalization and ATG Commerce contain several Web services that provide access to specific ATG repositories and repository item types. A complete list of the Web services included in ATG 6 can be found in the ATG 6 Web Services and Integration Framework Guide.
WebService Implementation
http://hostname:8840/repository/generic /getRepositoryItem/getRepositoryItem
webservice.GetRepositoryItemSEIImpl
241
15 - Repository Web Services
Nucleus Component Method Name Input Parameters
getRepositoryItem String pRepositoryPath String pItemDescriptorName
/atg/repository/RepositoryServices
the path of the repository component from which to retrieve the item
the found RepositoryItem in XML format, or null if no item with that repository ID exists Exceptions
atg.repository.RepositoryException if a repository error occurs atg.repository.xml.GetException if an error occurs translating the
GetRepositoryItem Exceptions
Exceptions generated by the GetRepositoryItem Web service may occur for the following reasons: RepositoryException The pRepositoryPath specified by the argument is null, or empty The pRepositoryPath specified by the argument does not resolve to a component The pRepositoryPath specified by the argument does not resolve to a Repository The call to Repository.getItem throws a RepositoryException
Parameter
NucleusPath MethodName EarFileName
Value
/atg/repository/RepositoryServices getRepositoryItem repositoryWebServices.ear
242
15 - Repository Web Services
RepositoryWebServices
AppName AppDescription ServletName ServletDisplayName ServletDescription URLPattern WebURI ContextPath WebAppDisplayName WebAppDescription
A collection of generic repository Web services, where the user must provide information about which repository is being acted upon null (will be dynamic) null (will be dynamic)
pRepositoryPath, pItemDescriptorName, pRepositoryId true
243
15 - Repository Web Services
Web Service Implementation
Web Service URL Web Service Class Name Nucleus Component Method Name Input Parameters
The RQL string to execute. Note that this string cannot contain parameters. Output
String[]
The found RepositoryItems in XML format, or null if no items satisfy the given query. Exceptions
atg.repository.RepositoryException if a repository error occurs atg.repository.xml.GetException if an error occurs translating the
PerformRQLQuery Exceptions
Exceptions generated by the PerformRQLQuery Web service may occur for the following reasons: RepositoryException The pRepositoryPath specified by the argument is null, or empty. The pRepositoryPath specified by the argument does not resolve to a component. The pRepositoryPath specified by the argument does not resolve to a Repository. The pItemDescriptorName specified by the argument does not identify an ItemDescriptor of the given repository. The pRQLString specified by the argument is null, or empty. The RQL code throws an exception, either during parsing or execution.
GetException The call to GetService.getItemAsXML throws a GetException for any of the found items.
244
15 - Repository Web Services
Value
/atg/repository/RepositoryServices performRQLQuery repositoryWebServices.ear RepositoryWebServices
WebService Generation
The following parameters are used to generate this Web service, using the WebServiceGeneratorImpl class:
Parameter
NucleusPath MethodName EarFileName AppName AppDescription ServletName ServletDisplayName ServletDescription URLPattern WebURI ContextPath WebAppDisplayName WebAppDescription
When called, executes the given query against the specified repository
PerformRQLQuery generic.war repository/generic RepositoryServices
A collection of generic repository Web services, where the user must provide information about which repository is being acted upon
null (will be dynamic) null (will be dynamic) pRepositoryPath, pItemDescriptorName, pRQLString true
245
15 - Repository Web Services
WebService Implementation
Web Service URL Web Service Class Name Nucleus Component Method Name Input Parameters
The RQL string to execute. Note that this string cannot contain parameters. Output
int
PerformRQLCountQuery Exceptions
Exceptions generated by the PerformRQLCountQuery Web service may occur for the following reasons: RepositoryException The pRepositoryPath specified by the argument is null, or empty. The pRepositoryPath specified by the argument does not resolve to a component. The pRepositoryPath specified by the argument does not resolve to a Repository. The pItemDescriptorName specified by the argument does not identify an ItemDescriptor of the given repository. The pRQLString specified by the argument is null, or empty. The RQL code throws an exception, either during parsing or execution.
WebService Generation
The following parameters are used to generate this Web service, using the WebServiceGeneratorImpl class:
246
15 - Repository Web Services
Value
/atg/repository/RepositoryServices performRQLCountQuery repositoryWebServices.ear RepositoryWebServices
Parameter
NucleusPath MethodName EarFileName AppName AppDescription ServletName ServletDisplayName ServletDescription
Performs a repository count query using the given repository path, item descriptor name and RQL string
generic.war repository/generic RepositoryServices
A collection of generic repository Web services, where the user must provide information about which repository is being acted upon null (will be dynamic) null (will be dynamic)
pRepositoryPath, pItemDescriptorName, pRQLString true
247
15 - Repository Web Services
248
15 - Repository Web Services
16 Composite Repositories
All ATG repositories provide a means for representing information in a data store as Java objects. The composite repository provides a means for using more than one data store as the source for a single repository. The composite repository consolidates all data sources in a single data model, making the data model flexible enough to support the addition of new data sources. Additionally, the composite repository allows all properties in each composite repository item to be queryable. Thus, from the point of view of your ATG application, the composite repository presents a consistent view of your data, regardless of which underlying data store the data may reside in. The composite repository is a repository that unifies multiple data sources. Its purpose is to make any number of repositories appear in an ATG application as a single repository. The composite repository defines a mapping between item descriptors and properties as they appear to facilities that use the composite repository and item descriptors and properties of the data models that comprise the composite data model. A composite repository is composed of any number of composite item descriptors. Each item descriptor can draw on different data models from different repositories, and map underlying data model attributes in different ways.
Use Example
Suppose you maintain profile data both in a SQL database and an LDAP directory. Dynamos profile repository ships with a user composite item descriptor comprised of just one primary item descriptor and no contributing item descriptors. The primary item descriptor is the user item descriptor. You can add to the composite item descriptor the user item descriptor from the LDAP repository as a contributing item descriptor. If there are any property name collisions between the SQL repository and the LDAP repository, you can resolve them by mapping the properties explicitly to different names in the composite repository configuration. Once youve done this, your Dynamo applications can view both LDAP profile information and SQL database profile information as properties of composite items in the composite user item descriptor.
249
16 - Composite Repositories
the repository item ID or a unique property. A contributing item is linked to a primary item if the value of its unique ID attribute matches the value of the primary items unique ID attribute. If multiple relationships are defined, then they are AND-ed together. For example, suppose you have a contributing item descriptor that defines two relationships to the primary item descriptor. One says that a primary items firstName property must match the contributing items userFirstName property and the other says that the primary items lastName property must match the contributing items userLastName. These two relationships together mean that a users first names and last names must each match for two items to be related. This is useful in situations where no one property uniquely identifies a user. See link-via-property Tag for an example of defining a relationship with two or more properties.
Property Derivation
The properties in a composite item descriptor are determined as follows: 1. If configured to do so, all properties from the primary and contributing item descriptors are combined into the composite item descriptor, with each property retaining its property name and property type. Any properties marked as excluded are removed from the composite item descriptor. See Excluding Properties.
2.
250
16 - Composite Repositories
3.
All property mappings are performed. This means that a primary or contributing property that is to be mapped gets renamed in the composite item descriptor. See Property Mappings. If there are any two properties in the composite item descriptor that have the same name, an error results. The composite repository requires that all composite property names map explicitly to only one primary or contributing property.
4.
8.
Property Mappings
The composite repository requires that all composite property names map explicitly to only one primary or contributing property. If your primary or contributing item descriptors contain one or more properties that have the same name, you need either to exclude one of the properties (see Excluding Properties) or map it to a new name. You can map a property using the mapped-property-name attribute in a property tag in an item descriptor. For example, suppose you have two contributing item descriptors, each of which has a property named login. You can map one of the properties to a new name like this:
<property name="ldapLogin" ... mapped-property-name="login"/>
In this example, the name attribute specifies the property name in the composite item descriptor and the mapped-property-name attribute specifies the name of the property in the primary or contributing item descriptor to which this property maps.
251
16 - Composite Repositories
Excluding Properties
<property name="password ... exclude="true"/>
Sometimes you may not want to expose absolutely every property from the underlying primary and contributing item descriptors in the composite item descriptor. You can configure the item descriptor to exclude those contributing properties that are not desired. You do this using a property tag with its exclude attribute set to true:
Link Methods
The link-method attribute determines what happens when the composite repository needs to get a property value that belongs to a contributing repository item. For example, suppose a process calls
CompositeItem.getPropertyValue("ldapFirstName");
where ldapFirstName is a property of a contributing repository item in an LDAP repository. The CompositeItem that is being asked for the property needs to look for this contributing item. If it is able to find it, it retrieves the property value and then does one of two things based on the value of the linkmethod attribute. If the link-method attribute is set to static, then the contributing item is stored in a member variable of that composite repository item, so that the next time a property is requested from that same item, it just retrieves it from this variable instead of finding it again from the underlying contributing repository. This saves some computational effort and results in faster property retrieval. However, if for any reason, the value of the property or properties used to link to the underlying contributing item have been changed, the data in this member variable will be stale. Note that this can only occur if a linking property the underlying data store is altered. For example, if you link to a contributing item descriptor using a login property, static linking could result in stale data only if the login property is changed in one in the underlying repositories. If the link-method attribute is set to dynamic, then the composite repository queries the underlying repository for the contributing item every time a property is requested from it. This might result in slower performance, but it also means that data would never be out of sync at the repository level. Dynamic link mode might seem like the most technically correct implementation, since the data model is guaranteed to reflect the most up-to-date information. However, since it requires a query each time information is needed from a composite item, it can impair performance. Remember also that in most cases, the information that links items together changes only rarely. Static linking is more often than not sufficient to provide correct data model linking.
When users create a new composite item via the createItem() method in
MutableCompositeRepository, then new instances of the primary item and of all contributing items
will be created. So, for example, if you have a user item type defined in your composite repository that
252
16 - Composite Repositories
borrows properties from the SQL and LDAP repositories, then any new user composite repository item that is created will create both a SQL repository item and an LDAP repository item. However, before these items can be added to their respective repositories, the correct link needs to exist between them. If the items are linked by a certain property, then this property needs to be set on the primary item before the items are added, otherwise an error will occur since theres no way in the future to link those two items back together.
lazy
If this option is chosen, contributing items will be created only when they are needed. In this case, when users call setPropertyValue on a property that is defined in the contributing repository, the composite repository creates the item in the contributing then and there. There are two different behaviors depending on whether the CompositeItem is transient or not. If the item is transient, then we wait until the item is persisted before checking to see that all of the appropriate linking properties have been set, so that we may propagate them to the new contributing item. If the item is not transient, we check to see if the correct linking properties are set on the primary item and then add the contributing item to its repository. If there any properties used for linking are missing, then an error is returned. The check for valid linking properties occurs during the updateItem call, and not during the setPropertyValue call on the contributing item. So if you use lazy item creation and call
setPropertyValue on a persistent item, you dont need to already have valid values set for any linking
properties on the primary item at that exact point in time. As long as the values of the linking properties are set before updateItem is called, then the item should be successfully created.
none
If this option is chosen, then no repository items will be created in the underlying repositories under any circumstance. Any contributing items used in the composite repository must already exist in order for valid results to be returned from property value requests.
If there is no contributing item found, then the default value for that property in the contributing item descriptor is returned. If there is no default value, then null is returned
null
repository. Create a component of this class and set its configurationFile property to the Nucleus
253
16 - Composite Repositories
Property
configurationFile
address of the composite repository definition file. You can configure the following properties of this component:
Description The Nucleus address of an XML file that uses the Composite Repository DTD. See the Composite Repository Definition File Tag Reference. If true, output from all debug levels lower than the current debug level will be printed to the log. An integer from 0 to 23 that indicates the frequency with which debug log messages are generated. The higher the value, the greater the frequency of debug log entries. See Debug Levels in the Development, Testing and Debugging with the SQL Repository chapter. The maximum number of items that will be returned by a single query to an underlying repository. The name of the composite repository
Value
cumulativeDebug
Boolean. Default is
true
debugLevel
Integer. Default is 5
queryBatchSize
repositoryName
String
254
16 - Composite Repositories
composite-repository-template Tag
The composite-repository-template tag encloses the whole composite repository definition. The composite-repository-template tag encloses a <header> tag and one or more <itemdescriptor> tags: header Tag item-descriptor Tag
Example:
<composite-repository-template> <header> ... </header> <item-descriptor name="..." /> ... </item-descriptor> </composite-repository-template>
header Tag
The <header> tag provides information that can help you manage the creation and modification of repository definition files.
Description The name of this template. The author or authors of this template.
255
16 - Composite Repositories
version zero or one An identifier for the version of this template. description zero or one For example, the header of your template might look like this:
<header> <name>Catalog Template</name> <author>Neal Stephenson</author> <author>Emily Dickinson</author>
<version>$Id: catalog.xml,v 1.10 2000/12/24 03:34:26 hm Exp $</version> <description>Template for the store catalog</description> </header>
item-descriptor Tag
The item-descriptor tag defines a composite item descriptor.
Attribute name
Description The name of the composite item descriptor. Is this the composite repositorys default item descriptor? You can set this property to the name of a repository item property. A user interface can then represent the repository item using this property. For example, a profile item descriptor might use displayproperty="login". Then, each repository item would be represented using the value of the items login property.
Value Required
default
display-property
String
256
16 - Composite Repositories
If you specify a resource bundle for this property using the <attribute
name=resourceBundle>
display-nameresource
String
tag, then the displayname-resource attribute specifies the resource bundle key that holds the display name. See the Localizing SQL Repository Definitions chapter. link-method The method used in retrieving properties from contributing repository items. See Link Methods. Specifies how and whether contributing repository items are created. See Creating Composite and Contributing Items. Specifies what to do if a contributing repository items is requested, but not found in the underlying repository. See Missing Contributing Items.
static (default) or dynamic
contributing-itemcreation-policy
null-contributingitem-policy
Example:
<item-descriptor name="compositeUser" default="true" display-property="fooProperty" display-name-resource="itemDescriptorUser"> <attribute name="resourceBundle" value="atg.userprofiling.CompositeProfileTemplateResources" data-type="string"/> <primary-item-descriptor.../> <contributing-item-descriptor.../> ... </item-descriptor>
257
16 - Composite Repositories
contributing-item-descriptor Tag attribute Tag
primary-item-descriptor Tag
Used in: item-descriptor Tag
One and only one item descriptor must be designated as the primary item descriptor. The function of the primary item descriptor is to provide the ID space for the composite item descriptor.
Attribute name
Description A unique name for the item descriptor The Nucleus address of the repository component in which the item descriptor resides. The name of this primary or contributing item descriptor in its source repository. If true, the composite repository attempts to make all properties of the primary or contributing item descriptor available to the composite item descriptor. This attribute determines whether the properties of this item descriptor are queryable by default. This setting can be overridden by explicitly setting the queryable attribute on the property.
repository-nucleus-name
repository-item-descriptorname
Required. String.
all-properties-propagate
all-properties-queryable
258
16 - Composite Repositories
contributing-item-descriptor Tag
The contributing-item-descriptor element specifies Used in: item-descriptor Tag
attribute Tag
The attribute tag is used to specify the list of feature descriptor values. The <attribute> tag is an empty tag that defines the item descriptors feature descriptor value or values. Here is an example:
<property name="employeeNumber" data-type="string"> <attribute name="PCCExpert" value="true" data-type="boolean"/> </property>
Attribute tags must be empty and have no child tags. Used in: item-descriptor Tag property Tag
Attribute name
Description The name of the name/value pair. You can specify any name here and it will be added to the list of feature descriptor attributes for your property. The value of the name/value pair. The data type of this value is defined by the data-type attribute supplied to this tag. If no data-type attribute is provided, the value of the attribute is a string.
Value Required.
value
Required.
259
16 - Composite Repositories
data-type The primitive data-type of the value.
Default is string.
property Tag
The property tag allows you to explicitly map a property in a composite repository to a property in a primary item descriptor or a contributing item descriptor. This usage handles the case where two or more contributing item descriptors have properties with the same name. It can have one or more <attribute> tags. Used in: primary-item-descriptor Tag contributing-item-descriptor Tag
Attribute name
Description The name of this composite property. The name of the property in the primary or contributing item descriptor to which this property maps.
Value Required
mapped-propertyname
Required
queryable required expert Expert properties are not displayed in the default view of the ATG Control Center. Hidden properties are not displayed in the ATG Control Center.
hidden
260
16 - Composite Repositories
Boolean. Default is false. Boolean. Default is false. If you specify a resource bundle for this property using the
<attribute name=resourceBundle> tag, then
String
the category-resource attribute specifies the resource bundle key that holds the category of the property. See the Localizing SQL Repository Definitions chapter. display-nameresource If you specify a resource bundle for this property using the
<attribute name=resourceBundle> tag, then
String
the display-name-resource attribute specifies the resource bundle key that holds the display name. See the Localizing SQL Repository Definitions chapter. exclude Set this attribute to true to exclude it from the composite item descriptor. See Excluding Properties. Boolean. Default is false.
Example
<property name="ldapFirstName" mapped-property-name="firstName" queryable="false" required="false" expert="false" hidden="false" readable="true" writable="true" category-resource="categoryBasics" display-name-resource="ldapFirstName"> ... </property>
primary-item-descriptor-link Tag
The primary-item-descriptor-link tag determines which method is used to link items in contributing item descriptors to items in the primary item descriptor. It has no attributes.
261
16 - Composite Repositories
Used in: contributing-item-descriptor Tag
If the primary-item-descriptor-link tag encloses a link-via-id tag, then the repository ID of the item is used for linking. If the primary-item-descriptor-link tag encloses a link-via-property tag, then a unique property of the item, specified in the link-via-property tag, is used for linking.
Examples In this example, the contributing item descriptors items are linked to the primary item descriptors items by the common repository ID of the items:
<primary-item-descriptor-link> <link-via-id/> </primary-item-descriptor-link>
In this example, a primary item is linked to an item in this contributing item descriptor if the value of the primary items firstName property matches the value of the contributing items userFirstName property AND the value of the primary items lastName property matches the value of the contributing items userLastName property. This is useful in the case where no one property in the primary item descriptor or the contributing item descriptor is uniquely valued. The relationships are AND-ed together.
<primary-item-descriptor-link> <link-via-property primary="firstName" contributing="userFirstName"/> <link-via-property primary="lastName" contributing="userLastName"/> </primary-item-descriptor-link>
link-via-id Tag
The link-via-id tag is an empty tag with no attributes. If it is present, it indicates that the repository ID of the item is used for linking the item in the primary item descriptor to the item in the contributing item descriptor. Used in: primary-item-descriptor-link Tag
link-via-property Tag
The link-via-property tag is an empty tag. If it is present, it indicates that one or more properties are used to link the item in the primary item descriptor to the item in the contributing item descriptor. See the second example under primary-item-descriptor-link Tag. Used in: primary-item-descriptor-link Tag
262
16 - Composite Repositories
Description The name of the property in the primary item descriptor used for linking. The property name used is the name in the underlying repository and not the names in the composite repository. The name of the property in the contributing item descriptor used for linking. The property name used is the name in the underlying repository and not the names in the composite repository. Value Required.
Attribute primary
contributing
Required.
sort-property
<?xml encoding="UTF-8"?> <!-- =============================================================== --> <!-- composite-repository_1.0.dtd - Composite Repository configuration spec --> <!-- Version: $Change: 213147 $$DateTime: 2001/10/01 11:59:44 $$Author: gm $--> <!-- =============================================================== --> <!-- Flag datatype, and values --> <!ENTITY % flag "(true | false)"> <!-- The attribute tag is used to specify the list of feature descriptor values --> <!ELEMENT attribute EMPTY> <!ATTLIST attribute name value data-type > CDATA CDATA CDATA #REQUIRED #REQUIRED #IMPLIED
<!-- =============================================================== --> <!-- composite-repository-configuration - top level element --> <!-- =============================================================== --> <!ELEMENT composite-repository-template (header?, item-descriptor*)>
263
16 - Composite Repositories
<!-- The header --> <!-- Name of template --> <!ELEMENT name (#PCDATA)> <!-- The author(s) --> <!ELEMENT author (#PCDATA)> <!-- Version string --> <!ELEMENT version (#PCDATA)> <!-- Description string --> <!ELEMENT description (#PCDATA)>
<!-- =============================================================== --> <!-- composite-view element: --> <!-The definition of a view as it appears to code that calls the --> <!-- composite repository. --> <!-- =============================================================== --> <!ELEMENT item-descriptor (attribute*, primary-item-descriptor, contributing-item-descriptor*)> <!ATTLIST item-descriptor name CDATA default %flag;
#REQUIRED "false"
display-property CDATA #IMPLIED display-name-resource CDATA #IMPLIED link-method CDATA #IMPLIED contributing-item-creation-policy CDATA #IMPLIED null-contributing-item-policy CDATA #IMPLIED > <!-- =============================================================== --> <!-- The primary item descriptor definition <!-<!-<!-<!-<!-<!---> The primary view's property values take precedence over --> contributing views' property values. Also, a composite item's --> primary item provides the composite item's id. The repository-nucleus-name and view-name specify the primary view. The unique-id-property specifies which property in the uniquely identifies items in the primary view. --> --> --> -->
<!-- =============================================================== --> <!ELEMENT primary-item-descriptor (property*)> <!ATTLIST primary-item-descriptor name CDATA #REQUIRED repository-nucleus-name CDATA #REQUIRED CDATA #REQUIRED repository-item-descriptor-name
264
16 - Composite Repositories
<!ELEMENT contributing-item-descriptor (property*, primary-item-descriptor-link)> <!ATTLIST contributing-item-descriptor name CDATA #REQUIRED repository-nucleus-name CDATA #REQUIRED repository-item-descriptor-name CDATA #REQUIRED all-properties-propagate %flag; "false" all-properties-queryable > <!ELEMENT property (attribute*)> <!ATTLIST property name CDATA #IMPLIED CDATA "true" "false" "false" "false" "true" "true" CDATA CDATA "false" #IMPLIED #IMPLIED #REQUIRED mapped-property-name queryable %flag; required expert hidden readable %flag; %flag; %flag; %flag; %flag; "true"
<!ELEMENT primary-item-descriptor-link (link-via-id | link-via-property+)> <!ELEMENT link-via-id EMPTY> <!ELEMENT link-via-property EMPTY> <!ATTLIST link-via-property primary CDATA #REQUIRED contributing sort-property > CDATA %flag; #REQUIRED #IMPLIED
265
16 - Composite Repositories
<!-- composite repository definition --> <composite-repository-template> <!-- Header similar to GSA DTD --> <header> <!-- name of this document -->
<name>A sample Composite Repository template</name> <!-- author of this document --> <author>Graham Mather</author> <!-- version of this document --> <version>$Change: 226591 $$DateTime: 2002/01/22 15:50:56 $$Author: gm $ </version> </header> <!-- composite item descriptor definition --> <!-- name: name of the composite item descriptor --> <!-- default: is this the composite repository's default item descriptor? --> <!-- display-property: the property used when display items of this type --> <!-- display-name-resource: resource which defines the display name --> <item-descriptor name="compositeUser" default="true" display-property="fooProperty" display-name-resource="itemDescriptorUser"> <!-- resource bundle from whence this item descriptor's resources come --> <attribute name="resourceBundle" value="atg.userprofiling.CompositeProfileTemplateResources" data-type="string"/> <!-- icon for items of this type --> <attribute name="icon" value="userIcon" data-type="string"/> <!-- "basics" category sort priority --> <attribute name="categoryBasicsPriority" value="10" data-type="int"/> <!-- primary view definition --> <!-- name: the name of the primary view, as it appears internally to the composite repository. The primary view and all composite views must have unique internal view names --> <!-- repository-nucleus-name: the nucleus path of the repository in which the primary view resides --> <!-- repository-item-descriptor-name: the name of the view in the given repository which acts as the primary item descriptor for this composite item descriptor --> <!-- all-properties-propagate: if true, composite repository attempts to make all properties in the primary item descriptor available in the composite item descriptor. Default is false --> <!-- all-properties-queryable: if true, all properties in the view are queryable unless otherwise specified. If false, all properties are not queryable unless otherwise specified. default is true -->
266
16 - Composite Repositories
<primary-item-descriptor name="user" repository-nucleus-name="/atg/userprofiling/ProfileAdapterRepository" repository-item-descriptor-name="user" all-properties-propagate="true" all-properties-queryable="true"> <!-Can also contain explicit property mappings and explicit property exclusions --> <property mapped-property-name="lastName" exclude="true"/> <property mapped-property-name="email" exclude="true"/> </primary-item-descriptor> <!-- contributing view definition --> <!-- name: the name of this contributing view, as it appears to the composite repository --> <!-- repository-nucleus-name: the nucleus path of the repository in which the primary view resides --> <!-- repository-item-descriptor-name: the name of the view in the given repository which acts as the primary item descriptor for this composite item descriptor --> <!-- all-properties-propagate: if true, composite repository attempts to make all properties in the primary item descriptor available in the composite item descriptor. Default is false --> <!-- all-properties-queryable: if true, all properties in the view are queryable unless otherwise specified. If false, all properties are not queryable unless otherwise specified. default is true --> <contributing-item-descriptor name="UserProfile-LDAP" repository-nucleus-name="/atg/adapter/ldap/LDAPRepository" repository-item-descriptor-name="user" all-properties-propagate="true" all-properties-queryable="true">
<!-- explicit property mapping sometimes it's advantageous to explicitly map a property in a composite view to a particular property in either the primary or a contributing view. For example, perhaps two contributing views have properties with the same name. This gets around the "no contributing views with same property names" rule. --> <!-- name: name of this composite property --> <!-- mappedPropertyName: the property to which this property maps --> <!-- queryable: property queryable flag --> <!-- required: property required flag--> <!-- expert: property expert flag -->
267
16 - Composite Repositories
<!-- hidden: property hidden flag --> <!-- readable: property readable flag --> <!-- writable: property writable flag --> queryable="false" required="false" expert="false" hidden="false" readable="true" writable="true" category-resource="categoryBasics" display-name-resource="ldapFirstName"> <!-- bundle for this property's resources --> <attribute name="resourceBundle"
<!-- category-resource: resource for category name --> <!-- display-name-resource: resource for display name --> <property name="ldapFirstName" mapped-property-name="firstName"
value="atg.userprofiling.CompositeProfileTemplateResources" data-type="string"/> <!-- flag for ui being able to write this property --> <attribute name="uiwritable" value="true" data-type="boolean"/> <!-- maximum length for this property --> <attribute name="maxLength" value="32" data-type="int"/> <!-- does this property's value have to be unique? --> <attribute name="unique" value="true" data-type="boolean"/> <!-- sort priority --> <attribute name="propertySortPriority" value="10" data-type="int"/> </property> <!-- explicit property exclusion Sometimes users will not want to expose absolutely every property from the underlying primary and contributing views in the composite view. An explicit property removal allows the user to make the composite view contain only those contributing properties that are desired. --> <property mapped-property-name="login" exclude="true"/> <property mapped-property-name="password" exclude="true"/> <property mapped-property-name="id" exclude="true"/> <!-2) a composite view's property names are determined thusly: a) If all-properties-propagate is true, all properties from the primary and contributing views are combined into the composite view, retaining their property names, property types, and any metadata they may have defined. b) All property exclusions are performed. This means that any properties to be excluded are removed from the composite view. c) All property mappings are performed. This means that a primary or contributing property that is to be mapped gets renamed in the composite view.
268
16 - Composite Repositories
d) If there are any two properties in the composite view that have the same name, error. The composite repository requires that all composite property names map explicitly to only one primary or contributing property. --> <!-- the primary view link describes how items in the contributing view are linked to items in the primary view. For each primary-contributing relationship, the user picks a unique id attribute for the primary and the contributing view. The attribute can be either the repository id of the item or a uniquely-valued property of the item (e.g. login). A primary item is linked to a contributing item if its unique id attribute value matches the unique id attribute value of the contributing item. There must be at least one primary view link, but there is primary view link limit. -->
<!-- example: this primary view link defines a relationship where an item in the primary view is linked to an item in this contributing view if the contributing item has a repository id which is the same as the primary item's id. --> <!-<primary-item-descriptor-link> <link-via-id/> </primary-item-descriptor-link> --> <!-- OR: This primary view link defines a relationship where a primary view item is linked to an item in this contributing view if the value of the primary item's "login" property matches the value of the contributing item's "userLoginName" property. --> <primary-item-descriptor-link> <link-via-property primary="login" contributing="login"/> </primary-item-descriptor-link> <!-- OR: This primary view link defines a relationship where a primary view item is linked to an item in this contributing view if the value of the primary item's "firstName" property matches the value of the contributing item's "userFirstName" property AND the value of the primary item's "lastName" property matches the value of the contributing item's "userLastName" property. This is useful in the case where no one property in the primary view or the contributing view is uniquely valued. The relationships are ANDed together
269
16 - Composite Repositories
<primary-item-descriptor-link> </primary-item-descriptor-link> --> </contributing-item-descriptor> </item-descriptor> </composite-repository-template>
270
16 - Composite Repositories
17 Secured Repositories
The ATG secured repository system works in conjunction with the ATG Security System to provide finegrained access control to repository item descriptors, individual repository items, and even individual properties through the use of Access Control Lists (ACLs). This chapter includes the following sections: Secured Repository Features Creating a Secured Repository Modifying the Underlying Repository Configuring the Secured Repository Adapter Component Writing the Secured Repository Definition File ACL Syntax Secured Repository Definition File Tag Reference Secured Repository Definition File Document Type Definition Secured Repository Example
271
17 - Secured Repositories
Access control on properties of all Repository Items in a Repository Item Descriptor The ability to control who can read or write a particular property in any repository item defined by an item descriptor. This is similar to control of a column in a database table. A default ACL may be assigned to all items in the item descriptor that do not have an explicit ACL. Access control on properties of an individual Repository Item The ability to control who can read or write a particular property in a repository item. This is similar to control of a field of a row in a database table. If an ACL is assigned to a property that also has an ACL for the property in the item descriptor, the ACL for the property overrides the ACL defined in the item descriptor. Limitation of query results The ability to control who can receive certain repository items as results from a repository query. Ownership of a Repository Item At creation time the current user is assigned as the owner of the new repository item. The owner has the implicit right to query a repository item and modify its ACL; otherwise this is simply an association of an identity to an Item. Automatic generation of ACLs on new Repository Items When a new repository item is created, it is assigned an ACL that is constructed out of an ACL fragment and a template for the owner (creator) and each group the owner (creator) is a member of. All of these features may be configured or not according to the needs of your application. Some features require additional storage in the underlying repository, or may have significant performance impact. Features that are unnecessary need not be enabled to save space or improve performance. See Performance Considerations for more information about performance issues.
Access Rights
Access control within secured repositories is managed by building ACLs that associate certain access rights with certain identities (not just individual users, but also groups, organizations, and roles that may be associated with many users). The following table lists the access rights defined by the secured repository system. Not all access rights are necessarily available in all implementations or instances of a secured repository.
Description Determines the ability to create a new repository item with an item descriptor. In order to add the new item to the repository you must also have WRITE access to the item descriptor. Determines the ability to remove a repository item from an item descriptor. In order to delete an Item you must also have DESTROY access for that Item.
DELETE
RepositoryItemDescriptor
272
17 - Secured Repositories
RepositoryItem Determines the ability to remove the repository item from the repository, destroying its contents. Note that most secured repositories will also require DELETE access on the item descriptor. Determines the ability for a repository item to be queried. If LIST access is not given for a repository item, the item will never be returned from a query. If you wish to also grant access to a repository items properties, you should additionally use the READ access right. An items owner implicitly has LIST access. Determines the ability to request a repository item from an item descriptor, or to inspect the contents of a repository item or a property in a repository item. Determines the ability to inspect the ACL of a repository item. This access right is implicitly granted to the owner of a repository item. Determines the ability to inspect the owner of a repository item. Determines the ability to add a repository item to an item descriptor, or to change the contents of a repository item or a property in a repository item. If the WRITE access right is granted for the item descriptor, it does not affect the ability to update a repository item, only the ability to add new repository items. Determines the ability to change the ACL of a repository item. This access right is implicitly granted to the owner of a repository item. Determines the ability to change the owner of a repository item. RepositoryItem
DESTROY
LIST
READ
READ_ACL
READ_OWNER
RepositoryItem
WRITE
WRITE_ACL
RepositoryItem
WRITE_OWNER
RepositoryItem
It is important to remember that the secured repository does not provide complete security within an application, since the unprotected repository that it sits on top of is still available within the Nucleus name space. The intent is to provide a repository that aids in creating secure applications, not one that protects the data from rogue programmers. The ATG Control Center may be configured to hide unprotected repositories, and an application may choose not to use an unprotected repository, so as not to expose unprotected data to an end user.
273
17 - Secured Repositories
1. 2.
Modify the underlying repository. For those item descriptors you want to secure, you need to make some minor modifications to the underlying data and item descriptors to add properties with which to store the Access Control List and owner information. This is described in the Modifying the Underlying Repository section of this chapter. Create and configure the Secured Repository Adapter component. This components class is atg.adapter.secure.GenericSecuredMutableRepository. See Configuring the Secured Repository Adapter Component in this chapter. Add the Secured Repository Adapter component to the ContentRepositories registry. See Register the Secured Repository Adapter in this chapter. Create the secured repository definition file. This is an XML file that specifies the access rights and owner information. This is described in the Writing the Secured Repository Definition File section in this chapter. Access rights are specified using the syntax described in the ACL Syntax section.
3. 4.
<item-descriptor name="cheese"> <property name="country" data-type="string" /> <property name="runniness" data-type="int" /> <property name="ACL" datadata-type="string" /> component<property name="cheeseOwner" component -type="user" /> </item-descriptor>
Make sure that the ACL property is of an appropriate data-type. The length of an ACL is limited by the amount of space available in the property for the ACL. An ACL that is too long will generate a repository exception when set. This problem is most likely to arise in cases where you use the create-group-acltemplate in the secured repository definition to define an ACL for the groups of the owner and the owner is a member of many groups. You can avoid this problem by defining the ACL property as an array of strings, rather than a single string. The ACL is then stored as a collection of substrings, which are concatenated to form the ACL. For example:
<item-descriptor name="cheese"> ... <table name="test_items_acls" type="multi" id-column-names="id" multi-column-name="index"> <property name="ACL" column-names="acl" data-type="array" component-data-type="string">
274
17 - Secured Repositories
The maxFragmentSize attribute defines the longest string that will be put in any array index. The default size is 254. You should set this value to the size of the string column in the database. For many databases, if you use a VARCHAR of unspecified length, then 254 will be the appropriate value. These properties in the underlying repository will be identified in the secured repositorys definition file, using the following tags:
owner-property acl-property
For example:
<acl-property name="ACL" /> <owner-property name="cheeseOwner" />
Property
$class
name
A description of the Secured Repository component that appears in the ACC. The name of the Secured Repository component The name of the underlying repository that the secured repository acts on. The repository definition file used by the secured repository. See Writing the Secured Repository Definition File.
SecuredTestRepository
repositoryName
repository
TestRepository
configurationFile
secured-testrepository.xml
275
17 - Secured Repositories
securityConfiguration
The
atg.security.SecurityConf iguration component to use.
For more information about security policies and other security features, see the Managing Access Control chapter in the ATG 6 Dynamo Programming Guide.
Attribute
descriptor-acl
Description The ACL that applies to the item descriptor. This can contain any of the access rights that apply to the item descriptor. The value of this tag is an ACL string, as defined in the ACL Syntax section. The default-acl element specifies the ACL that is applied to either an item or property descriptor when it has no other ACL. This ACL can contain any of the access rights that apply to the item descriptor or property. The value of this tag is an ACL string, as defined in the ACL Syntax section. This defines the name of the string property in the underlying repository that is to be used to store the name of the owner of a repository item.
default-acl
owner-property
276
17 - Secured Repositories
This defines the name of the string property in the underlying repository that is used to store the ACL for an individual repository item. An ACL fragment that is inserted into the default ACL for a newly created repository item. Typically this defines global access rights for administrators and limited access rights for the user base as a whole. This ACL fragment can contain any of the access rights that apply to a repository item. An ACL template that is used to generate an ACL fragment that applies to the owner (creator) of a newly created repository item. This is a standard format ACL string with a dollar sign ($) used to indicate the owner identity. No other identities may be used in the template. An ACL template that is used to generate an ACL fragment that applies to each group that the owner (creator) is a member of in a newly created repository item. This is a standard format ACL string with a dollar sign ($) used to indicate the group identity. No other identities may be used in the template. Because a user may have a great many groups that they are a member of, it is suggested that this feature be used sparingly. For example, the ACC admin user may have enough groups to create an ACL that is too large for our example repository. For a description of what constitutes membership in a group, see Group Membership.
acl-property
creation-base-acl
creation-owner-acltemplate
creation-group-acltemplate
You can use a subset of these options to define ACLs for properties as well as item descriptors:
descriptor-acl default-acl acl-property creation-base-acl creation-owner-acl-template creation-group-acl-template
Group Membership
An identity is considered to be a group that the owner (creator) is a member of if the owners Persona lists it with its getSubPersonae() call. Exactly what is returned by this call will vary according to the implementation of the User Authority. The standard User Authority used here is implemented on top of the User Directory interface, and includes every Effective Principal of the user as a sub-Persona. For the Profile User Directory, this includes all of the Organizations, Roles, and Relative Roles of the user as well as all of the Organizations, Roles and Relative Roles of any Organization they are members of (either explicitly or implicitly). For the Admin User Directory, this includes all of the Groups that the ACC account is a member of, but not the Privileges that the Group has been assigned.
277
17 - Secured Repositories
ACLs and Personae
When creating ACLs, it is imperative that the Personae that are used for user identities be created by the exact same User Authority that is being used by the secured repository. The User Authority may not be a proxy even though the Personae produced by a proxy test is equivalent to the Personae produced by the User Authority that it is a proxy for. This is because the identity name spaces used by a User Authority and its proxies may not be the same, and the ACL parser cannot support multiple identity namespaces.
ACL Syntax
ACL strings in Dynamo are made up of a series of Access Control Entries (ACEs) separated from each other by semicolons:
ACL ::= ACE [ ';' ACE ]+
Each ACE is made up of two parts, an identity and a list of access rights, separated by a colon, and optionally surrounded by an ACE type specifier for determining whether the ACE is to grant or deny rights:
ACE ::= ( ( IDENTITY ':' ACCESS_RIGHTS_LIST ) | ( ( "grant" | "deny" ) '{' IDENTITY ':' ACCESS_RIGHTS_LIST '}' ) )
The "grant" modifier is the default, and may be omitted. If a "deny" ACE exists where a "grant" ACE also applies, the standard security policy will deny access. An identity is the literal string used by the User Authority to look up the identitys Persona. The standard User Authority (/atg/dynamo/security/UserAuthority in Nucleus) encodes the identity as follows:
UD_IDENTITY ::= UD_NAME '$' PRINCIPAL_TYPE '$' UD_PRINCIPAL_KEY
where UD_NAME is the name of the User Directory as configured in the User Directory User Authority (usually either Admin for the ACC account database or Profile for the Profile Repository), PRINCIPAL_TYPE is one of user, org or role, and UD_PRINCIPAL_KEY is the primary key for looking up the principal in the User Directory. The primary key varies between User Directory implementations. The primary key is a numeric ID for Profile User Directories, but is the account name (for example, admin, administrators-group) for the ACC account User Directory. Dynamo comes configured with three other User Authorities:
/atg/dynamo/security/AdminUserAuthority for looking up ACC accounts /atg/userprofiling/ProfileUserAuthority for looking up Profile accounts /atg/dynamo/service/j2ee/J2EEUserAuthority for looking up J2EE accounts and roles. The J2EEUserAuthority is a clone of the AdminUserAuthority in DAS,
278
17 - Secured Repositories
These user authorities look up Persona information based on the unencoded name of the identity and are typically used for performing authentication. They are, however, front-ends for the standard User Authority and produce Personae that are equivalent to those produced by the standard User Authority. (Note the caveat regarding the mixing of User Authorities in the Writing the Secured Repository Definition File: ACLs and Personae topic.) The list of access rights is a comma-separated list of access right names:
ACCESS_RIGHT_LIST ::= ACCESS_RIGHT [ ',' ACCESS_RIGHT ]+
Only the access rights appropriate for the ACL context are allowed. Access right names are tokens and are not internationalizable.
ACL Examples
The following examples are coded using the syntax used by the standard /atg/dynamo/security/UserAuthority component. The following ACL grants everyone with an ACC account the ability to read:
Admin$role$everyone-group:read
The following ACL grants the ACC admin account the ability to read and write, but every other ACC user only the ability to read:
Admin$user$admin:list,read,write;Admin$role$everyone-group:list,read
The following ACL grants the ACC Administrators group the ability to read, write and delete, but denies the ability to write and delete to ACC user Fnord even if he is a member of the Administrators group:
279
17 - Secured Repositories
Admin$role$administrators-group:
list,read,write,delete;deny{Admin$user$Fnord:write,delete}
secured-repository-template Tag
The secured-repository-template tag encloses the whole secured repository definition. The secured-repository-template tag encloses one item-descriptor tag for each item descriptor in the underlying repository for which you want to specify access rights: item-descriptor Tag
Example:
<secured-repository-template> <item-descriptor name="..." /> ... </secured-repository-template>
item-descriptor Tag
You should include one item-descriptor tag for each item descriptor in the underlying repository for which you want to specify access rights. Unlike the item-descriptor tag in the SQL repository, the item-descriptor tag in the secured repository has just one attribute, name, which must be the same as the name attribute in the underlying repositorys item-descriptor tag. Example:
<item-descriptor name="feature"> <descriptor-acl value="..."/> <owner-property name="..."/> <acl-property name="..."/> ... </item-descriptor>
280
17 - Secured Repositories
descriptor-acl Tag owner-property Tag acl-property Tag creation-base-acl Tag creation-owner-acl-template Tag creation-group-acl-template Tag
property Tag
To apply access rights at the property level, rather than to the whole item, you can use the property tag. A property tag can enclose any or all of the following child tags: Example default-acl Tag descriptor-acl Tag creation-base-acl Tag creation-owner-acl-template Tag creation-group-acl-template Tag
<property name="yumminess"> <descriptor-acl value="..."/> <acl-property name="yummy_acl"/> <creation-base-acl value="... ... </property>
default-acl Tag
The default-acl element specifies the ACL that is applied to either an item or property descriptor when it has no other ACL. This ACL can contain any of the access rights that apply to the item descriptor or property. The value attribute of the tag is an ACL string, using the syntax described in the ACL Syntax section. Used in: item-descriptor Tag property Tag
Example:
<default-acl value="Admin$role$everyone-group:list,read"/>
281
17 - Secured Repositories
descriptor-acl Tag
Used in: item-descriptor Tag property Tag
The descriptor-acl element specifies the ACL that applies to the item or property specified by the enclosing item-descriptor tag or property tag. This ACL can contain any of the access rights that apply to the item descriptor or property. The value attribute of the tag is an ACL string, using the syntax described in the ACL Syntax section.
Example:
<descriptor-acl value="Admin$role$administrators-group:list,read,write; Admin$role$everyone-group:list,read"/>
owner-property Tag
The owner-property tag has one attribute, name, which specifies the name attribute of the property that stores the owner of the item in the underlying repository. Used in: Example If the item descriptor in the underlying repository stores the name of the items owner in a property named item_owner, the owner-property tag would look like this:
<owner-property name="item_owner"/>
item-descriptor Tag
acl-property Tag
The acl-property tag has one attribute, name, which specifies the name attribute of the property that stores the ACL of the item in the underlying repository. Used in: item-descriptor Tag property Tag
Example: If the item descriptor in the underlying repository stores the items ACL in a property named item_acl, the acl-property tag would look like this:
<acl-property name="item_acl"/>
282
17 - Secured Repositories
creation-base-acl Tag
The creation-base-acl tag defines an ACL fragment that is inserted into the default ACL for a newly created repository item or property. Typically this defines global access rights for administrators and limited access rights for the user base as a whole. This ACL fragment can contain any of the access rights that apply to a repository item or property. Used in: Example The following example gives all access rights to the administrators group, but only read and list rights to everyone else:
<creation-base-acl value="Admin$role$administrators-group: read,write,list,destroy,read_owner,write_owner,read_acl,write_acl; Admin$role$everyone-group:read,list"/>
creation-owner-acl-template Tag
The creation-owner-acl-template tag specifies an ACL template that is used to generate an ACL fragment that applies to the owner (creator) of a newly created repository item. This is a standard format ACL string with a dollar sign ($) used to indicate the owner identity. No other identities may be used in the template. Used in: Example The following example gives the owners of an item access rights to read, write, list, or destroy items they own:
<creation-owner-acl-template value="$:read,write,list,destroy"/>
creation-group-acl-template Tag
The creation-group-acl-template tag specifies an ACL template that is used to generate an ACL fragment that applies to each group that the owner is a member of in a newly created repository item. This is a standard format ACL string with a dollar sign ($) used to indicate the group identity. No other identities may be used in the template. Because a user may be a member of a large number of groups, you should use this feature sparingly. It can result in ACL strings that are too long to be stored in your repositorys ACL property. For example, the
283
17 - Secured Repositories
Used in: Example item-descriptor Tag property Tag
ACC admin user may have enough groups to create an ACL that is too large for our example repository. For a description of what constitutes membership in a group, see Group Membership.
The following example gives read and list access rights to every member of every group of which the items owner is a member:
<creation-group-acl-template value="$:read,list"/>
<?xml encoding="UTF-8"?> <!-- =============================================================== --> <!-- secured_repository_template.dtd - Definition spec for secured --> <!-- repositories --> <!-- Version: $Change: 166603 $$DateTime: 2001/04/20 20:05:51 $$Author: bbarber $ --> <!-- =============================================================== --> <!-- =============================================================== --> <!-- secured-repository-template - top level element --> <!-- =============================================================== --> <!ELEMENT secured-repository-template (item-descriptor)*> <!-- =============================================================== --> <!-- account specifications - define an account of a particular type --> <!-- =============================================================== --> <!ELEMENT item-descriptor (descriptor-acl | owner-property | default-acl | creation-base-acl | creation-owner-acl-template |
284
17 - Secured Repositories
#REQUIRED
<!-- =============================================================== --> <!-- descriptor-acl - specifies the ACL that is applied to either an --> <!-item or property descriptor --> <!-- =============================================================== --> <!ELEMENT descriptor-acl (#PCDATA)> <!ATTLIST descriptor-acl value > <!-- =============================================================== --> <!-- default-acl - specifies the ACL that is applied to either an --> <!-item or property descriptor when it has no other --> <!-ACL --> <!-- =============================================================== --> <!ELEMENT default-acl (#PCDATA)> <!ATTLIST default-acl value > <!-- =============================================================== --> <!-- owner-property - specifies the name of the property in which --> <!-the name of the owner of the item is stored --> <!-- =============================================================== --> <!ELEMENT owner-property (#PCDATA)> <!ATTLIST owner-property name > <!-- =============================================================== --> <!-- acl-property - specifies the name of the property in which --> CDATA #REQUIRED CDATA #REQUIRED CDATA #REQUIRED
285
17 - Secured Repositories
<!-<!ELEMENT acl-property (#PCDATA)> <!ATTLIST acl-property name > CDATA #REQUIRED
-->
<!-- =============================================================== --> <!-- creation-base-acl - specifies the base ACL fragment that will <!-be applied to all new items or properties <!-when they are created --> --> -->
<!-- =============================================================== --> <!ELEMENT creation-base-acl (#PCDATA)> <!ATTLIST creation-base-acl value > <!-- =============================================================== --> <!-- creation-owner-acl-template - specifies the ACL fragment <!-template that will be applied to <!-<!-<!-<!-all new items or properties when they are created, utilizing the owner (creator) of the object as the identity in all ACEs. --> --> --> --> --> --> CDATA #REQUIRED
<!-- =============================================================== --> <!ELEMENT creation-owner-acl-template (#PCDATA)> <!ATTLIST creation-owner-acl-template value > <!-- =============================================================== --> <!-- creation-group-acl-template - specifies the ACL fragment <!-template that will be applied to <!-<!-<!-<!-<!-<!-all new items or properties when they are created, utilizing each of the groups that the owner (creator) of the object is a member of as the identities in all ACEs. --> --> --> --> --> --> --> --> CDATA #REQUIRED
286
17 - Secured Repositories
CDATA #REQUIRED
value >
-- test-repository.ddl create table test_items ( -- the ID of this item id varchar, -- a secured property of this item secured_property varchar, -- an unsecured property unsecured_property varchar, )
# test-repository.xml <?xml version="1.0" encoding="UTF-8" standalone="no"?> <!DOCTYPE gsa-template PUBLIC "-//Art Technology Group, Inc.//DTD Dynamo Security//EN" "http://www.atg.com/dtds/gsa/gsa_1.0.dtd"> <gsa-template> <header> <name>Test Repository</name> </header> <item-descriptor name="test_items" default="true"> <table name="test_items" type="primary" id-column-names="id"> <property name="secured_property" column-names="secured_property" data-type="string"/> <property name="unsecured_property" column-names="unsecured_property" data-type="string"/> </table> </item-descriptor> </gsa-template>
287
17 - Secured Repositories
# TestRepository.properties Configuration File $class=atg.adapter.gsa.GSARepository definitionFiles=test-repository.xml repositoryName=TestRepository dataSource=/atg/dynamo/service/jdbc/JTDataSource idGenerator=/atg/dynamo/service/IdGenerator lockManager=/atg/dynamo/service/ClientLockManager
XMLToolsFactory=/atg/dynamo/service/xml/XMLToolsFactory transactionManager=/atg/dynamo/transaction/TransactionManager
We need to add fields to the SQL and the repository definition to provide storage space for security information, one each for storing the owner, repository item ACL, and repository item property ACL. The following files show these changes. The SQL now looks like this:
-- Modified test-repository.ddl create table test_items ( -- the ID of this item id varchar, -- a secured property of this item secured_property varchar, -- an unsecured property unsecured_property varchar, -- the owner of this item item_owner varchar, -- the ACL that applies to this item item_acl varchar, -- the ACL that applies to this item's secured value secured_property_acl varchar )
<!-- Modified test-repository.xml --> <?xml version="1.0" encoding="UTF-8" standalone="no"?> <!DOCTYPE gsa-template PUBLIC "-//Art Technology Group, Inc.//DTD Dynamo Security//EN" "http://www.atg.com/dtds/gsa/gsa_1.0.dtd"> <gsa-template> <header> <name>Test Repository</name> </header> <item-descriptor name="test_items" default="true"> <table name="test_items" type="primary" id-column-names="id"> <property name="secured_property" column-names="secured_property" data-type="string"/> <property name="unsecured_property" column-names="unsecured_property"
288
17 - Secured Repositories
<property <property
The properties file for the GSARepository component can remain as it was. The next step is to create the secured repository layer over this SQL repository. The secured repository has an XML definition file, which would look like this:
<!-- secured-test-repository.xml --> <?xml version="1.0" encoding="UTF-8" standalone="no"?> <!DOCTYPE secured-repository-template PUBLIC "-//Art Technology Group, Inc.//DTD Dynamo Security//EN" "http://www.atg.com/dtds/security/secured_repository_template_1.1.dtd"> <secured-repository-template> <item-descriptor name="test_items"> <!-- The ACL that applies to the item view/descriptor --> <descriptor-acl value="Admin$role$administrators-group: read,write,create,delete;Admin$role$everyone-group:read"/> <!-- The property that the ownership will be stored in --> <owner-property name="item_owner"/> <!-- The property that the ACL will be stored in --> <acl-property name="item_acl"/> <!-- An ACL fragment that is assigned to all new items --> <creation-base-acl value="Admin$role$administrators-group: read,write,list,destroy,read_owner,write_owner,read_acl,write_acl; Admin$role$everyone-group:read,list"/> <!-- Access rights that are assigned to the owner when an item is created --> <creation-owner-acl-template value="$:read,write,list,destroy"/> <!-- Access rights that are assigned to all of the owner's groups when an item is created. is potentially dangerous. --> <creation-group-acl-template value="$:read,list"/> <property name="secured_property"> <!-- The ACL that applies to this property across all items in the repository --> <descriptor-acl value="Admin$role$administrators-group: read,write;Admin$role$everyone-group:read"/> <!-- The name of the property in the item where the ACL for this property is stored. --> <acl-property name="secured_property_acl"/> <!-- An ACL fragment that is assigned to this property WARNING: This feature
289
17 - Secured Repositories
whenever a new item is created. --> item is created --> groups when an item is created.
<creation-base-acl value="Admin$role$administrators-group:read,write"/> <!-- Access rights that are assigned to the owner when an <creation-owner-acl-template value="$:read,write"/> <!-- Access rights that are assigned to all of the owner's WARNING: This feature is potentially dangerous. --> <creation-group-acl-template value="$:read,write"/> </property> </item-descriptor> </secured-repository-template>
# SecuredTestRepository.properties $class=atg.adapter.secure.GenericSecuredMutableRepository $scope=global name=Test repository for the secured repository implementation repositoryName=SecuredTestRepository # the repository that we're wrapping repository=TestRepository # The template file that configures the repository configurationFile=secured-test-repository.xml # The security configuration component used by the repository securityConfiguration=/atg/dynamo/security/SecuredRepositorySecurityConfiguration # Various Dynamo services we need XMLToolsFactory=/atg/dynamo/service/xml/XMLToolsFactory transactionManager=/atg/dynamo/transaction/TransactionManager
WARNING: In the above example we make use of the creation-group-acl-template feature for both repository items and the secured property. This setting should generally be removed if you are setting up a repository based on this code. The reason for this is explained at creation-group-acl-template Tag. Finally, in order to expose these repositories to the ATG Control Center Repository Editor, and to start them up when Dynamo is started, you must add each of them to the initialRepositories property of the /atg/registry/ContentRepositories component:
initialRepositories+=/TestRepository,/SecuredTestRepository
Performance Considerations
While care has been taken to maintain high performance, use of the secured repository does have some impact on the performance of the repository and, in some cases, the impact is considerable.
290
17 - Secured Repositories
For access control defined at the item descriptor level (e.g. Repository.getItem(), MutableRepository.createItem(), MutableRepository.addItem(), MutableRepository.updateItem()) the overhead of handling access checks amounts to the testing of the access control list for the item descriptor. This is normally minimal. The exception to this rule is with the use of the RepositoryView.executeQuery() and RepositoryView.executeCountQuery() family of methods whenever ACLs are specified for individual repository items. In this case, the ACL of each repository item must be consulted to determine if it should be allowed in the result of the query, or counted as part of a count query. If the result set is large, the time required to parse and check all of the ACLs may be large. Furthermore, in the count query case, a full query must be done and its results counted. Thus, if your application is using count queries to limit expensive queries, the features afforded by a secured repository will be very expensive. Access control overhead at the repository item level is noticeable, but is incremental. When the repository item is loaded its ACL will be parsed prior to any access checking. Results of ACL parsing are cached to improve performance where possible. If ACLs are not being stored for each individual repository item, no parsing needs to be done beyond what is done during the initialization of the secured repository. Because the secured repository sits on top of an underlying repository, you can consider whether features that need best possible performance should be written to use the underlying repository rather than going through the secured repository at the cost of the security features.
nested exception. The RepositoryItem.getPropertyValue() and MutableRepositoryItem.setPropertyValue() methods are unusual in that they do not throw a RepositoryException. It is necessary, however, for them to throw a SecurityException, so they will throw an atg.security.RuntimeSecurityException. The RuntimeSecurityException.getSecurityException() method can be used to determine the nested exception.
291
17 - Secured Repositories
292
17 - Secured Repositories
18 LDAP Repositories
The ATG LDAP Repository is an implementation of the Repository API that enables you to store and access profile data in an LDAP (Lightweight Directory Access Protocol) directory. The LDAP repository is similar in functionality to the SQL repository, as described earlier in this guide. While by default ATG Scenario Personalization is configured to use a SQL profile repository, you can change the configuration to use an LDAP repository instead. See the ATG 6 Personalization Programming Guide for information about configuring Dynamo to use an LDAP profile repository. LDAP directories are widely used to store personnel information and other kinds of data. Using an LDAP repository enables you to tap into the profile data you already have in an LDAP directory, and to share user information across multiple applications. In addition, you can configure Dynamos application security scheme to use an LDAP repository, rather than a SQL repository. See the Managing Access Control chapter in the ATG 6 Dynamo Programming Guide for more information. Just like the SQL repository, the LDAP repository implements the ATG Repository API to allow you to store, access, modify, and query user profile information. As in the SQL repository, repository items are first created as transient items (RAM profiles); they become persistent after they are added to the database. It is important to note, however, that the LDAP repository implementation is not specific to user profiles in any way. Since an LDAP directory can be used to store any kind of data (people, groups, mailing lists, documents, printers, etc.), you could use the LDAP repository to expose any of that data in Dynamo. In this chapter, we focus on using LDAP as a profile repository, because that is the most common application of LDAP. Keep in mind, however, that other uses are possible. This chapter includes the following sections: Overview: Setting Up an LDAP Repository An overview of the steps you should take in designing and setting up an LDAP repository. LDAP Directory Primer A brief introduction to LDAP concepts and terminology LDAP Repository Architecture A description of the way item descriptors and repository items work in the LDAP repository. Repository Views in the LDAP Repository How multiple Repository Views can support multiple item descriptors in the same repository.
293
18 - LDAP Repositories
LDAP Repository Queries A brief look at how queries work in the LDAP repository.
Configuring the LDAP Repository Components How to configure the components that make up the LDAP repository LDAP Repository Definition Tag Reference A detailed reference on the XML tags used to define an LDAP repository. Sample LDAP Repository Definition File An example of an XML file that defines a simple LDAP repository. Document Type Definition for LDAP Repository Definition Files The XML DTD for LDAP repository definitions.
3. 4.
294
18 - LDAP Repositories
within the organization, such as ou=Finance,o=quincyfunds.com, ou=Marketing,o=quincyfunds.com, and so on. Under the organizational unit subtrees, there might be entries representing individual people, e.g. uid=nat,ou=Finance,o=quincyfunds.com. As you can see above, a DN consists of a series of comma-separated attribute name/value pairs. The hierarchy is represented right-to-left in a DN, with the right-most pair indicating the top of the hierarchy. For example, ou=Finance,o=quincyfunds.com is a child of o=quincyfunds.com. The left-most attribute name/value pair is called a relative distinguished name (RDN). The examples in this section demonstrate some of the standard attribute names, such as: o for organization ou for organizational unit cn for common name These standard attribute names are inherited from the X.500 standard, which preceded LDAP. Their use is not required, but is a good convention to follow when possible. Note that you can also define an organization like this:
dc=quincyfunds,dc=com
The directory tree may be highly branched, with the entire organizational hierarchy reflected in the tree structure, or it may be almost flat, depending on the needs of the organization. An example of an almost flat directory structure is one where all the people entries reside under the same organizational unit entry, such as ou=person,o=quincyfunds.com. There may also be organizational unit entries for storing other
295
18 - LDAP Repositories
types of information, for example, ou=Groups,o=quincyfunds.com, ou=Customers,o=quincyfunds.com, ou=Devices,o=quincyfunds.com, and so on. A directory may have more than one directory suffix. This typically comes into play with very large directories which are spread across multiple machines, extranets, ISPs, etc. For example, an ISP whose directory service needs to support multiple enterprises might have a separate directory suffix for each of the enterprises.
objectClass: top objectClass: person objectClass: organizationalPerson uid: nat cn: Natalya Cohen cn: Nat Cohen sn: Cohen givenName: Natalya givenName: Nat
Many of the attributes in an LDAP directory can be multi-valued (such as the cn, givenName, and objectClass attributes in the example above). One interesting point to note is that the attribute values comprising the entrys distinguished name do not necessarily have to correspond to the attribute values contained in the entry itself. For example, the entry above does not contain an ou attribute or an o attribute, even though the DN implies an ou value of person and an o value of quincyfunds.com. Even more confusing situations are possible (although, of course, not recommended by the directory providers), where the attribute is specified both in the DN and in the entry itself, but the two values differ. For these kinds of cases, the thing to keep in mind is that the actual directory data is contained in the entrys attributes. The distinguished name is simply a name that can be used to uniquely identify the entry; it does not represent the actual attribute values. For example, when the directory is searched, it is not searched against the DN, but against the attribute values stored in the entries themselves. Note however that you do use the DN to access a directory entry directly, without searching. Also, you must specify the DN when you create a new entry.
296
18 - LDAP Repositories
The entrys object class is stored in the entry itself, as the value of its objectClass attribute. When you create an LDAP entry, you must specify values for all the attributes required by the entrys object class, and you may specify values for any optional attributes. The object class type can be a subtype of another object class. For example, the object class
organizationalPerson is a subtype of the object class person. It happens to not add any required
attributes, but it adds a number of optional ones, like title, postaladdress, and so on. The base (abstract) object class that every type inherits from is called top. Its single required attribute is objectClass. Notice that the example entry in the LDAP Data Representation section above has three values for its
objectClass attribute: top, person, and organizationalPerson. The first two values seem
unnecessary, since they are both ancestors of the organizationalPerson type. However, they are required because not all directory servers support type inheritance. The objectClass values in an entry dont all have to be each others ancestors, however. For example, one can create an entry that is both an organizationalPerson and a mailGroupMember, which itself inherits from top. In other words, multiple inheritance of types is allowed.
Directory Schema
The total set of object classes and attributes known to the LDAP directory is referred to as the directory schema. Each LDAP directory server comes with a standard schema that includes predefined object classes and attributes. In addition, you can extend this standard schema to represent information unique to your enterprise. For each object class, the schema contains information such as the names of the superior object classes from which this object class is derived, and the names of the required and optional attributes of the object class. For each of the attributes, the schema contains information about its syntax and whether the attribute is single- or multi-valued. All LDAP directory implementations are expected to support the minimal default schema specified in RFC 2256. The tables below summarize those object classes and attributes in the default schema used by ATGs LDAP repository. For the full list of object classes and attributes, please refer to the RFC.
Name top
Parent
Optional Attributes
297
18 - LDAP Repositories
person top sn, cn organizationalPerson person inetorgPerson organizationalPerson
Name objectClass
Description describes the kind of object an entry represents common name of an object, e.g. persons full name surname, or family name, of a person name of an organization name of an organizational unit or department persons first name user password as an Octet String persons title in organizational context telephone number fax number unique id e-mail address employee number
cn
false
false false false false false false false false false false false
Notice that all of the attributes listed above are multi-valued. There are actually very few single-valued attributes in LDAP, for maximum flexibility.
298
18 - LDAP Repositories
the major JNDI operations are easily mapped onto the corresponding LDAP operations. Thus, JNDI provides a natural way to access LDAP data from Java applications. Note that all the standard attributes are represented in Java as either String or byte[] data types. That is, Suns LDAP service provider for JNDI expects as input and returns as output all attribute values as one of these two types. When using the LDAP provider, you must explicitly configure all the attributes that should be treated as byte[]; the rest will be treated as String.
LDAP Sources
For more information about LDAP, consult the following: RFC 2251: Lightweight Directory Access Protocol (v3) - Specification of the LDAP version 3 protocol. RFC 2252: Lightweight Directory Access Protocol (v3): Attribute Syntax Definitions More of same, with more detail on attributes. RFC 2254: The String Representation of LDAP Search Filters - Describes the LDAP search filters, which are used by JNDI to perform directory searches. The LDAP repositorys QueryBuilder implementation constructs Query objects, which are essentially LDAP search filters. RFC 2256: A Summary of the X.500(96) User Schema for use with LDAPv3 - Default LDAP v3 schema that all LDAP directory servers are expected to support. The Sun JNDI tutorial at http://java.sun.com/products/jndi/tutorial/ - A good guide to both JNDI and LDAP Your LDAP server-specific documentation. For example, if you have access to a Sun ONE Directory Server installation, browse through the Bookshelf. The Deployment Guide, in particular, is a good source of information.
299
18 - LDAP Repositories
Additional Property Tag Attributes New Item Creation
You define the relationship between the LDAP directory schema and an ATG LDAP repository in an XML file called an LDAP repository definition file. This XML template needs to be located in the CONFIGPATH. Note that, unlike the SQL repository, the LDAP repository cannot use XML file combination to combine XML files in the CONFIGPATH that have the same name. Instead, you need to use a single LDAP repository definition file in your CONFIGPATH. This section introduces the principal features of the LDAP repository definition file. A complete reference to the repository definition file syntax is found in the LDAP Repository Definition Tag Reference section of this chapter.
300
18 - LDAP Repositories
<item-descriptor name="user"> <!-- special properties --> ... <!-- object classes --> <object-class>top</object-class> <object-class>person</object-class> <object-class>organizationalPerson</object-class> <object-class>inetorgPerson</object-class> <!-- properties --> <property name="login" ldap-name="uid" data-type="string" required="true"/> <property name="password" ldap-name="userpassword" data-type="string" required="true"/> <property name="lastName" ldap-name="sn" data-type="string" required="true"/> <property name="firstName" ldap-name="givenName" data-type="string"/> <property name="names" ldap-name="cn" data-type="string" multi="true" required="true"/> <property name="email" ldap-name="mail" data-type="string"/> <property name="phone" ldap-name="telephonenumber" data-type="string"/>
301
18 - LDAP Repositories
<property name="title" data-type="string"/> <property name="employeeNumber" data-type="long"/> <!-- new item creation --> ... </item-descriptor>
<property name="fax" ldap-name="facsimiletelephonenumber" data-type="string" default="(617) 555-1211"/> <property name="department" ldap-name="ou" data-type="string"/>
The object-class tags specify all the object class values corresponding to the given item descriptor. If the object class has ancestor object classes, they must all be specified, as demonstrated above. The object class information is required so that when a new item is created for the given item descriptor and added to the repository, the corresponding LDAP entry is created with the given object class values. Thus, for example, if an item is created in the context of the user item descriptor, the new LDAP directory entry has objectclass attribute values of top, person, organizationalPerson, and inetorgPerson. The LDAP repository definition uses <property> tags to map Profile properties to LDAP attributes. Each such <property>tag describes a property descriptor of its item descriptor. The <property>tags in the example above demonstrate that: Repository item property names can be different from LDAP attribute names. For example, the lastName property in the item descriptor maps to the sn attribute in the LDAP directory schema. If the ldap-name tag attribute is not specified, the repository item property name and the LDAP attribute name will be the same. Repository item property types can be different from JNDI service provider types. For example, userpassword is exposed as a binary type by Suns LDAP service provider, but is a String property in the repository; employeeNumber is a String in Suns LDAP service provider, but a Long in the repository. Repository item properties can have default values. For example, the fax property has a default value of (617) 555-1211.
In addition, the user item descriptor exposes only those attributes that we care about, and promotes some of the optional attributes into required ones. For example, the password attribute is optional in LDAP, but required in the Profile repository item. Note that although the attributes such as givenName and sn are multi-valued in the LDAP directory, they are exposed as single-valued properties in the repository. When getting the values for these properties, the LDAP repository will ignore all but one of the returned values. It is not specified which of the values will be returned. On the other hand, the LDAP repository items names property is multi-valued, and corresponds to the LDAP directorys multi-valued cn attribute; in this case, the attribute value is returned as a String array. For all of this to work, the repository item descriptor must not violate the LDAP directory schema. For example, since cn is a required attribute for the inetorgPerson class, one of the properties specified in the item descriptor must map to the cn attribute, and it must be required. As another example, the item descriptor cannot contain a property that does not correspond to an LDAP attribute. That is, the ldapname tag attribute value must be a legal LDAP attribute name. The LDAP repository does no checking to
302
18 - LDAP Repositories
ensure that the item descriptor conforms to the LDAP schema. If the schema is violated, a runtime exception (an object schema violation) is thrown by JNDI.
<item-descriptor name="engineer" parent="user"> <!-- object classes (added to parent classes) --> <object-class>engineeringPerson</object-class> <!-- properties (added to parent properties) --> <property name="engineerType" data-type="enumerated" default="products" description="Type of engineer: products or services"> <option>products</option> <option>services</option> </property> <property name="currentProject" data-type="string" description="Project or product the engineer is currently working on"/> <!-- child properties (override parent properties) --> <child-property name="department" default="Engineering"/> <!-- item creation (overrides parent behavior) --> ... </item-descriptor>
The optional parent property of an <item-descriptor> specifies that the item descriptor inherits all of the parents object classes and properties. Any additional object-class and property values are added to the list of the parents object classes and properties. You can also specify <child-property> tags to override any parent properties that have the same name. The only aspect of the parent property definition that can be overridden is the propertys default value. The propertys data-type and other attributes must stay the same. The example above demonstrates how the <child-property> tag can be used to assign the default value of Engineering
303
18 - LDAP Repositories
Id and ObjectClasses Properties
to the parents department property; the salespeople item descriptor might assign the default value of Sales to the same property.
In addition to the properties you specify, the LDAP repository creates two special properties for every item descriptor: the id attribute and the objectClasses attribute. Here is the relevant XML from the user item descriptor definition that we examined above:
<item-descriptor name="user"> <!-- special properties --> <id-property name="id" in-ldap="true" ldap-name="dpsid"/> <object-classes-property name="objectClasses" ldap-name="objectclass"/> <!-- object classes --> <object-class>top</object-class> <object-class>person</object-class> <object-class>organizationalPerson</object-class> <object-class>inetorgPerson</object-class> <object-class>dpsUser</object-class> <!-- properties --> ... <!-- new item creation --> ... </item-descriptor>
The purpose of the <id-property> tag is to expose the repository ID of a repository item as an attribute (of type String). Thus, assuming the definition above, an item with repository ID uid=nat,ou=Marketing,o=quincyfunds.com would have an LDAP attribute named dpsid with the same value. The attribute value does not need to be set by the user; it is set automatically by Dynamo. Note that the ID property is populated from the DN; you should not attempt to create the DN from the ID property. The rest of the id-property definition above specifies whether the id property of the repository item maps to an actual LDAP attribute, and if so, the LDAP attributes name. If the value of in-ldap is false (the default), the id attribute exists only as a property of the repository item, and does not exist as an attribute in the LDAP entry. In that case, when the items attribute values are written out to the LDAP directory, the ID attribute value is ignored, since there is no equivalent for it in the directory entry. If the value of in-ldap is true, as above, the ldap-name tag attribute specifies the name of the LDAP attribute to which the id should be written. As usual, if ldap-name is not specified, it is assumed to be the same as name. Thus, with our example item descriptor, when an item with ID uid=nat,ou=Marketing,o=quincyfunds.com is created and added to the repository, the resulting LDAP entry has an attribute named dpsid with value uid=nat,ou=Marketing,o=quincyfunds.com.
304
18 - LDAP Repositories
Saving the ID attribute value in the LDAP entry makes it easier to perform ID matching repository queries, as discussed in the LDAP Repository Queries section in this chapter. The <object-classes-property> tag is similar to <id-property>: it exposes the items object class values as an attribute. The attributes type is String[], which allows for a multi-valued attribute. For example, an item with a user item descriptor will have an objectClasses attribute, the value of which will be an array the elements of which are top, person, organizationalPerson, inetorgPerson, and dpsUser. The dpsUser object class supports the dpsid attribute, which allows us to incorporate the repository ID as an attribute in the LDAP entry. The <id-property> and <object-classes-property> tags are both required in a definition of a base item descriptor (that is, an item descriptor that doesnt have a parent); however, they are not allowed in child descriptor definition. The child item descriptors inherit the id and objectClasses properties from their parent.
<property name="department" ldap-name="ou" data-type="string" multi="false" display-name="Department" description="Department within the organization" default="unknown" required="false" readable="true" writable="false" queryable="true" hidden="false" expert="false"/>
See the LDAP Repository Definition Tag Reference in this chapter for full details. For properties with data-type of enumerated, use <option> tags to specify the propertys value choices. We saw an example of this in the engineerType attribute of the engineer item descriptor:
<property name="engineerType" data-type="enumerated" default="products" description="Type of engineer: products or services"> <option>products</option> <option>services</option> </property>
305
18 - LDAP Repositories
This approach is again inherited from the SQL repository definition file. Just like the SQL repository, an LDAP repositorys <property> tags can have zero or more <attribute> child tags. These child tags allow you to associate arbitrary name/string value pairs with any attribute. The name/value pairs are added to the attributes property descriptor via java.beans.FeatureDescriptor.setValue, and can later be used by the application. Here is an example:
<property name="employeeNumber" data-type="string"> <attribute name="unique" value="true"/> </property>
You might use a descriptor like unique to specify that a property value can be assigned to only one repository item within the item type. This LDAP repository feature is similar to the feature described in the User-Defined Property Types section of the SQL Repository Item Properties chapter. You can also specify a property editor class to use with a property, using the editor-class attribute. For example, the following tag associates a special property editor with the password property:
<property name="password" ldap-name="userpassword" data-type="string" required="true" editor-class="atg.beans.PasswordPropertyEditor"/>
<item-descriptor name="user"> <!-- special properties --> ... <!-- object classes --> ... <!-- properties --> ... <!-- new item creation --> <new-items allowed="false"> </item-descriptor> <item-descriptor name="engineer" parent="user"> <!-- object classes ... (added to parent classes) -->
306
18 - LDAP Repositories
<!-- properties (added to parent properties) --> ... <!-- child properties (override parent properties) --> ... <!-- new item creation (overrides parent behavior) --> <new-items parent-dn="ou=Engineering,o=quincyfunds.com" rdn-property="login"> </item-descriptor>
The <new-items> tag in the user descriptor indicates that this descriptor does not allow new items to be created. The user descriptor basically acts as an abstract class it provides a base set of object classes and properties for children descriptors to build on, but it does not allow items with those object classes and properties to be instantiated. The engineer descriptor, on the other hand, does allow new items to be created. The new-items tag specifies where the newly created items should be placed in the LDAP directory. The new items DN is constructed by appending the value of the parent-dn attribute to the RDN. The RDN is created using the value of the LDAP attribute that corresponds to the repository item property specified by the rdnproperty XML attribute. For example, if a new item is created whose login property is nat, the corresponding RDN is uid=nat (since the Profiles login property maps to the uid attribute in the LDAP directory), and the DN is uid=nat,ou=Engineering,o=quincyfunds.com. If a child descriptor definition does not contain a <new-items> tag, it inherits the parents item creation behavior.
307
18 - LDAP Repositories
comprise a Repository View search roots, because they determine which parts of the directory tree are searched when a repository query is constructed on the view. To summarize, the contents of each Repository View are determined by two factors: the object classes of its item descriptor, and its search roots. When a query is performed on the view, only those items that reside in one of the specified search roots and satisfy the views item descriptor are returned. At least one search root must always be specified, but it may well point to the directory suffix (i.e., the search root may span the entire directory tree).
<view name="user" default="true"> <!-- item descriptor --> <item-descriptor name="user"> ... </item-descriptor> <!-- search roots --> <search-root dn="o=quincyfunds.com"/> </view> <view name="engineer"> <!-- item descriptor --> <item-descriptor name="engineer" parent="user"> ... </item-descriptor> <!-- search roots --> <search-root dn="ou=Engineering,o=quincyfunds.com" recursive="false" check-classes="true"/> </view>
In this example, the user view spans all of o=quincyfunds.com, including ou=Engineering,o=quincyfunds.com, ou=Sales,o=quincyfunds.com, and so on, whereas the engineer view is restricted to ou=Engineering,o=quincyfunds.com. Note the default attribute in the user view specification, which designates user as the default view name.
308
18 - LDAP Repositories
If this query were applied to the ou=Engineering,o=quincyfunds.com search root of the engineer Repository View, the query would be turned into
If the value of check-classes is false for the search root, however, the query is left as is, and no object class checking is performed. Obviously, this optimization should only be turned on if you are absolutely sure that the search root contains only entries that satisfy the item descriptor.
ID Matching Queries
The idMatching query is a special case. The LDAP search filter can only search for entries based on their attribute values. However, the LDAP repository uses the entrys DN, rather than any attribute value, as its ID. Thus, ID matching queries cant be constructed using search filters, unless the LDAP entrys DN is also an LDAP attribute.
309
18 - LDAP Repositories
To implement ID matching queries, add an ID attribute to the LDAP entries, as we saw in the Id and ObjectClasses Properties section previously in this chapter. In our example, all user LDAP entries have an attribute called dpsid, which is mapped to the repository items id attribute. The value of dpsid is automatically set to the DN when the item is first added to the repository. Since the ID can now be accessed as an attribute of an LDAP entry, full support for the ID matching query is provided in this case. Note, however, that directory entries that were not created by the repository must be manually modified to include a dpsid attribute, or they will not be returned by the queries on the view. If no ID attribute exists in the LDAP entries, the ID matching query is only supported as the top level query. That is, you can have a targeting rule that matches only items with specified IDs, but you cant have a rule that matches items with specified IDs and satisfies some other criteria. The top level query is implemented by simply calling Repository.getItems with the specified IDs. No checking is done to verify that the resulting items are actually contained by the view. Dynamo doesnt check that they have the correct object classes, and are located inside one of the search roots.
/atg/adapter/ldap/LDAPRepository /atg/adapter/ldap/InitialContextPool
The Repository of LDAP profiles. A resource pool (JNDIInitialContextPool) used to pool connections to the LDAP server. Specifies the JNDI environment properties used to create a JNDI InitialDirContext.
/atg/adapter/ldap/InitialContextEnvironment
310
18 - LDAP Repositories
An LRU cache that maps repository item IDs to persistent repository items. A component used by the LDAPItemCache to retrieve persistent repository items from the directory. An LRU cache that maps repository search queries to the repository item IDs of the query results. A component used by the LDAPQueryCache to perform repository queries.
/atg/adapter/ldap/LDAPItemCache
/atg/adapter/ldap/LDAPItemCacheAdapter
/atg/adapter/ldap/LDAPQueryCache
/atg/adapter/ldap/LDAPQueryCacheAdapter
These LDAP repository components can be configured using the properties described below.
/atg/adapter/ldap/LDAPRepository
This component is the repository of LDAP profiles.
Property
$class
Default Value
atg.adapter.ldap. LDAPRepository
cacheItemProperties
Should repository items cache their properties? Should the repository cache directory items? Should Repository Views cache query results? The location of the XML template in the CONFIGPATH. Note that the LDAP repository uses XML file combination to combine XML files in the CONFIGPATH that have the same name. See the XML File Combination section in the Nucleus: Organizing JavaBean Components chapter of the ATG 6 Dynamo Programming Guide. The Nucleus address of the component that creates repository IDs for new repository items.
true
cacheItems
true
cacheQueries
false
definitionFile
/atg/adapter/ldap/ ldapUserProfile.xml
idGenerator
/atg/dynamo/service/ IdGenerator
311
18 - LDAP Repositories
initialContextPool
/atg/adapter/ldap/ InitialContextPool
objects.
itemCache
The Nucleus address of the repository item cache component. Should repository items prefetch their properties? If true, the first time any of the items properties is accessed, all of the items property values are retrieved and cached. This value is used only if cacheItemProperties is set to true. The Nucleus address of the Query cache component. Name of repository. How long (in seconds) to delay before shutting down. This value is used only if shutdownDynamoOnFatal is set to true. Should Dynamo be shut down on fatal repository errors? The Nucleus address of the Transaction Manager component. The Nucleus address of the XMLToolsFactory component.
/atg/adapter/ldap/ LDAPItemCache
prefetchItemProperties
true
queryCache
repositoryName shutdownDelay
shutdownDynamoOnFatal
true
transactionManager
XMLToolsFactory
/atg/adapter/ldap/InitialContextPool
This component is a JNDIInitialContextPool used to pool connections to the LDAP server. This components class extends atg.service.resourcepool.ResourcePool. See the Core Dynamo Services: Resource Pools section of the ATG 6 Dynamo Programming Guide and the Javadoc for the ResourcePool class in the ATG 6 API Reference for more information about the many properties available for configuring a connection pool. Getting connections from a resource pool results in better performance than would creating a new connection for each request that needs to access the LDAP server. The following properties are particular to the JNDIInitialContextPool:
312
18 - LDAP Repositories
Property
$class
JNDIEnvironment
The Nucleus address of the JNDI environment component to use when creating initial context objects Should InitialDirContext objects be created rather than InitialContext objects? Should the resource pool InitialContext (or InitialDirContext) objects should be wrapped in MonitoredContext (or MonitoredDirContext) objects? If monitored contexts are being created, then any JNDI service provider errors which occur as a result of operations performed on the contexts will be reported, and the associated resource pool objects will be invalidated.
createDirContexts
true
createMonitoredContexts
true
/atg/adapter/ldap/InitialContextEnvironment
This component specifies the JNDI environment properties used to create a JNDI InitialDirContext. You need to configure this component to point to your LDAP directory server. You are most likely to need to set the following properties (other than the class definition):
Property
$class
Default Value
atg.adapter.ldap. LDAPJNDIEnvironment
providerURL
ldap://localhost:389
313
18 - LDAP Repositories
securityAuthentication simple none
Authentication mechanism for the provider to use. Some valid values are:
simple
use the CRAM-MD5 (RFC-2195) SASL mechanism. See securityAuthentication Property below for more information.
securityPrincipal
The identity of the principal to be authenticated, in the form of a distinguished name. The credentials of the principal to be authenticated Any additional environment properties you may need to set. The value of the otherProperties property is one or more commaseparated property/value pairs. For example, you could set:
otherProperties= com.sun.jndi.ldap.someProperty =someValue
cn=ldapadmin
securityCredentials
ldapadmin
otherProperties
null
securityAuthentication Property
The securityAuthentication property must be set to match an appropriate type of security authentication for your LDAP server. For example, you can use the CRAM-MDS setting only if you have configured your LDAP directory server appropriately. See Suns LDAP tutorial on the subject of SASL at http://java.sun.com/products/jndi/tutorial/ldap/security/sasl.html for more information on the valid SASL mechanisms. Note also that if you set this property to none, the LDAP server will treat the LDAP repository as an anonymous client. Depending on how your LDAP server is configured, you may therefore be unable to create, modify, or delete LDAP directory entries through the LDAP repository.
314
18 - LDAP Repositories
initialContextFactory objectFactories controlFactories stateFactories URLPkgPrefixes DNSURL authoritative batchSize referral securityProtocol language
See the Javadoc for javax.naming.Context for more information about these properties. Furthermore, the InitialContextEnvironment component has the following properties that apply to LDAP service providers in general or are specific to Suns JNDI LDAP service provider:
LDAPVersion binaryAttributes connectControls deleteRDN derefAliases typesOnly refSeparator socketFactory referralLimit BERTrace schemaBugs
See Suns JNDI Implementor Guidelines for LDAP Service Providers, section 3, Environment Properties, at http://java.sun.com/products/jndi/jndi-ldap-gl.html#PROP for more information.
/atg/adapter/ldap/LDAPItemCache
This component is an LRU cache that maps repository item IDs to persistent repository items.
Property
$class cacheAdapter
Description class name The Nucleus address of the adapter that knows how to get objects not found in the cache The maximum time in milliseconds an entry will live in the cache. 0 = cache nothing, always get objects from the cacheAdapter -1 = cache entries never expire
Default Value
atg.service.cache.Cache /atg/adapter/ldap/ LDAPItemCacheAdapter
maximumEntryLifetime
-1
315
18 - LDAP Repositories
maximumCacheEntries
The maximum number of entries in the cache. 0 = cache nothing, always get objects from the cacheAdapter -1 = unlimited
500
/atg/adapter/ldap/LDAPItemCacheAdapter
This component is used by the LDAPItemCache to retrieve persistent repository items from the directory.
Property
$class
Default Value
atg.adapter.ldap. LDAPItemCacheAdapter /atg/adapter/ldap/ LDAPRepository
repository
/atg/adapter/ldap/LDAPQueryCache
This component is an LRU cache that maps repository search queries to the repository item IDs of the query results.
Property
$class cacheAdapter
Description class name The Nucleus address of the adapter that knows how to get objects not found in the cache The maximum time in milliseconds an entry will live in the cache. 0 = cache nothing, always get objects from the cacheAdapter -1 = cache entries never expire
Default Value
atg.service.cache.Cache /atg/adapter/ldap/ LDAPQueryCacheAdapter
maximumEntryLifetime
-1
maximumCacheEntries
The maximum number of entries in the cache. 0 = cache nothing, always get objects from the cacheAdapter -1 = unlimited
1000
316
18 - LDAP Repositories
/atg/adapter/ldap/LDAPQueryCacheAdapter
This component is used by the LDAPQueryCache to perform repository queries.
Property
$class
repository
/atg/adapter/ldap/ LDAPRepository
Change this property to ensure consistency with the LDAP password encryption method youve chosen. For Sun ONE Directory Servers, set the passwordHasher property like this:
passwordHasher=/atg/adapter/ldap/NDSPasswordHasher
The NDSPasswordHasher component supports SHA or no encryption. Set the encryption property of the /atg/adapter/ldap/NDSPasswordHasher to the appropriate value:
encryption=SHA
to disable password encryption. For LDAP servers other than Sun ONE Directory Server, you may need to create your own PasswordHasher implementation, if none of the PasswordHasher implementations included in ATG 6 meet your requirements. See the Working with User Profiles chapter of the ATG 6 Personalization Programming Guide for more information about configuring the PropertyManager component.
317
18 - LDAP Repositories
the Sample LDAP Repository Definition File
at the end of this chapter. Note that while the LDAP repository definition is similar in many ways to the SQL repository definition, the LDAP repository definition uses its own XML document type definition and syntax. Note also that, unlike the SQL repository, the LDAP repository cannot use XML file combination to combine XML files in the CONFIGPATH that have the same name. Instead, you need to use a single LDAP repository definition file in your CONFIGPATH.
DOCTYPE Tag
All XML documents declare their doctype, referencing a document type definition file. You should not need to change the reference to the DTD in the following DOCTYPE tag:
<!DOCTYPE ldap-adapter-template PUBLIC "-//Art Technology Group, Inc.//DTD LDAP Adapter//EN" "http://www.atg.com/dtds/ldap/ldap_1.0.dtd">
ldap-adapter-template Tag
The entire repository definition is enclosed in an <ldap-adapter-template> tag.
header Tag
The <header> tag provides information that can help you manage the creation and modification of repository definition files.
318
18 - LDAP Repositories
author version
view Tag
A repository definition file must include one <view> tag for each RepositoryView in your repository.
Attribute name
Value Required. Must be unique within the definition file. Boolean. Default is false.
default
item-descriptor Tag
Each RepositoryView in the LDAP repository includes a single item descriptor, which is defined by an <item-descriptor> tag.
Attribute name
Value Required. Must be unique within the definition file.. Optional. An item descriptor may have zero or one parents.
parent
319
18 - LDAP Repositories
item-descriptor Child Tags
Child Tag id-property object-classes-property object-class property child-property new-items How many? zero or one zero or one zero or more zero or more zero or more zero or one
id-property Tag
The <id-property> tag defines the profile ID property in the RepositoryItem and the LDAP entry. The tag is always empty. For example:
<id-property name="id" in-ldap="true" ldap-name="dpsid"/>
The <id-property> tag is required in a definition of a base item descriptor (an item descriptor that doesnt have a parent) but is not allowed in a child item descriptor that inherits from a parent.
Description The ID propertys name in the RepositoryItem. Does the ID property correspond to a single LDAP attribute? The ID attributes name in the LDAP directory. The text identifying the ID property in the ATG Control Center. The description of the ID property displayed in the ATG Control Center.
ldap-name display-name
description
object-classes-property Tag
The <object-classes-property> tag exposes the object classes of an LDAP entry as a property of a RepositoryItem. This tag is always empty. For example:
320
18 - LDAP Repositories
Like the <id-property> tag, the <object-classes-property> tag is required in a definition of a base item descriptor (an item descriptor that doesnt have a parent) but is not allowed in a child item descriptor that inherits from a parent. The propertys type is String[], a multi-valued String. For example, if an item descriptor definition has the <object-classes-property> tag in the preceding example and has the following object classes definition:
<object-class>top</object-class> <object-class>person</object-class> <object-class>organizationalPerson</object-class> <object-class>inetorgPerson</object-class>
then its repository items will have the following objectClasses property:
objectClasses=top,person,organizationalPerson,inetorgPerson
Attribute name
Description The name of the repository item property that stores the items LDAP object class values. The propertys name in the LDAP directory. The text identifying the object classes property in the ATG Control Center. You can set this property to the name of a repository item property. A user interface can then represent the repository item using this property. For example, an LDAP users item descriptor might use displayproperty="login". Then, each repository item would be represented using the value of the items login property. The description of the object classes property displayed in the ATG Control Center.
Value Required.
ldap-name display-name
display-property
description
object-class Tag
The object-class tags specify all the object class values corresponding to the given item descriptor. If the object class has ancestor object classes, they must all be specified. For example:
<object-class>top</object-class> <object-class>person</object-class>
321
18 - LDAP Repositories
<object-class>inetorgPerson</object-class>
<object-class>organizationalPerson</object-class>
The object class information is required in the item descriptor specification so that when a new item is created for the given item descriptor and added to the repository, the corresponding LDAP entry can be created with the given object class values. Thus, for example, if an item is created in the context of the user item descriptor, the new LDAP directory entry has objectclass attribute values of top, person, organizationalPerson, and inetorgPerson.
property Tag
Property tags define the properties of a repository item and map the repository item properties to LDAP entry attributes.
Description the propertys name The propertys name in the LDAP directory. The propertys Java data-type. See the LDAP Data-type Correspondences table for how the data-type attribute names correspond to Java object types.
data-type
Required. Valid values are: string big string date timestamp enumerated boolean int byte binary short long float double
322
18 - LDAP Repositories
Boolean. Default is false. Defaults to value of name. Defaults to the value of name
multi
Is this a multi-valued property? If true, the type is an array. The text identifying the property in the ATG Control Center. The description of the property displayed in the ATG Control Center. A default value for the property, if another value is not specified when the profile is created
displayname description
default
required readable writable queryable Non-queryable properties are not indexed in the ATG Control Center. Hidden properties are not displayed in the ATG Control Center.
Boolean. Default is false. Boolean. Default is false. Boolean. Default is false. Boolean. Default is false.
hidden
expert editor-class Associates a property editor class with the property. See the JavaBeans specification for a description of PropertyEditors.
Data-type Attribute Value string big string date timestamp enumerated boolean int
323
18 - LDAP Repositories
byte binary short float double long
Byte byte[] Short Float Double Long
option Tag
For properties whose data-type is enumerated, use <option> tags to indicate each of the possible values of the enumerated property. For example:
<property name="gender" data-type="enumerated"> <option>male</option> <option>female</option> </property>
attribute Tag
An <property> tag can contain zero or more <attribute> tags. The <attribute> tag is an empty tag that defines the propertys feature descriptor value or values. This tag allows you to associate arbitrary name/string value pairs with any property. The name/value pairs are added to the propertys property descriptor via the setValue method of java.beans.FeatureDescriptor, and can later be used by the application. Here is an example:
<property name="employeeNumber" data-type="string"> <attribute name="unique" value="true"/> </property>
Description the name of the name/value pair the value of the name/value pair
324
18 - LDAP Repositories
child-property Tag
If an item descriptor has a parent from which it inherits, you can use <child-property> tags to override the default values of properties of the parent item descriptor. The only aspect of the parent property definition that can be overridden is the propertys default value; the propertys data-type, etc. must stay the same. For example, suppose there is a parent item descriptor with the following property:
You can create a child property that overrides the default value of the department property:
<item-descriptor name="engineer" parent="user"> <!-- object classes (added to parent classes) --> <object-class>engineeringPerson</object-class> <!-- properties (added to parent properties) --> ... <!-- child property (overrides parent properties) --> <child-property name="department" default="Engineering"/> </item-descriptor>
See Item Descriptor Hierarchies and Inheritance in the LDAP Repository Architecture section of this chapter.
Attribute name
Description The attribute name, which is the same as the name of an attribute of the parent item descriptor. The default value for the attribute in the child item descriptor.
Value Required.
default
Required. Overrides the default value of the corresponding attribute in the parent item descriptor.
325
18 - LDAP Repositories
new-items Tag
new-items Tag Attributes
Attribute allowed Description If false, no new items can be created in this item descriptor; the item descriptor acts like an abstract class.
The <new-items> tag describes how new items within the item descriptor are created and identified.
parent-dn
The distinguished name (DN) of the parent. The new items DN is constructed by appending the value of parent-dn to the relative distinguished name, specified by the rdn-property. The name of the repository item property that specifies the relative distinguished name of a new item.
rdn-property
a new item whose login property is nat will have a corresponding RDN of uid=nat (since the LDAP repositorys login property maps to the uid attribute in the LDAP directory), and the DN will be uid=nat,ou=Marketing,o=quincyfunds.com. If a child descriptor definition does not contain a <new-items> tag, it simply inherits the parents item creation behavior. See New Item Creation in the LDAP Repository Architecture section of this chapter.
search-root Tag
A Repository Views contents can be restricted to a particular location or set of locations within the directory tree. For example, one might want to specify that the engineer view contains only entries in the ou=Engineering,o=quincyfunds.com branch of the directory tree. Even if other items that satisfy the engineer item descriptor are encountered somewhere in the LDAP directory (perhaps for testing purposes), they are not considered to be part of the engineer view. We call the tree branches that comprise a Repository View search roots, because they determine which parts of the directory tree are searched when a repository query is constructed on the view. The <search-root> tag is a child tag of the <view> tag that limits the Repository View to the specified roots of the LDAP directory tree. When a query is performed on the view, only those items that reside in one of the specified search roots and satisfy the views item descriptor are returned. At least one search root must always be specified, but it may well point to the directory suffix (i.e., the search root may span the entire directory tree). See the Repository Views in the LDAP Repository section of this chapter.
326
18 - LDAP Repositories
Description The distinguished name (DN) of directory branches that can be part of the Repository View Specifies whether the directory tree branch specified by the dn attribute should be searched recursively. You can set this to false if you want to include only the roots immediate children, or if you know for sure that lower levels of the branch do not contain any relevant entries. This might be used for optimization purposes. With check-classes set to true, when a repository query is constructed, it is automatically augmented with the object class constraints, so that items that do not satisfy the item descriptor are not returned by the search. If the value of check-classes is false for the search root, however, the query is left as is, and no object class checking is performed. Obviously, this optimization should only be turned on if you are absolutely sure that the search root contains only entries that satisfy the item descriptor. Value Required.
Attribute dn
recursive
check-classes
<?xml version="1.0" encoding="ISO-8859-1" ?> <!DOCTYPE ldap-adapter-template PUBLIC "-//Art Technology Group, Inc.//DTD LDAP Adapter//EN" "http://www.atg.com/dtds/ldap/ldap_1.0.dtd"> <ldap-adapter-template> <header> <name>ldapUserProfile.xml</name> <author>ATG</author> <version>$Id: ldapUserProfile.xml,v 1.5 2000/06/23 00:16:14 nat Exp $</version> </header>
327
18 - LDAP Repositories
<!-- user view --> <view name="user" default="true"> <!-- item descriptor --> <!-- special properties -->
<id-property name="id" in-ldap="false"/> <object-classes-property name="objectClasses" ldap-name="objectclass"/> <!-- object classes --> <object-class>top</object-class> <object-class>person</object-class> <object-class>organizationalPerson</object-class> <object-class>inetorgPerson</object-class> <!-- properties --> <property name="login" ldap-name="uid" data-type="string" required="true"> <attribute name="unique" value="true"/> </property> <property name="password" ldap-name="userpassword" data-type="string" required="true" editor-class="atg.beans.PasswordPropertyEditor"/> <property name="fullName" ldap-name="cn" data-type="string" required="true"/> <property name="lastName" ldap-name="sn" data-type="string" required="true"/> <property name="firstName" ldap-name="givenName" data-type="string"/> <property name="email" ldap-name="mail" data-type="string"/> <!-- item creation --> <new-items parent-dn="o=yourcompany.com" rdn-property="login"/> </item-descriptor> <!-- search roots --> <search-root dn="o=yourcompany.com"/> </view> </ldap-adapter-template>
328
18 - LDAP Repositories
http://www.atg.com/dtds/ldap/ldap_1.0.dtd
<?xml encoding="UTF-8"?> <!-==================================================================== ldap-adapter-template.dtd - document type for LDAP Adapter templates Version: $Change: 191227 $$DateTime: 2001/07/06 12:42:29 $$Author: nat $ ==================================================================== --> <!-- Flag datatype, and values --> <!ENTITY % flag "(true | false)"> <!-- The whole template --> <!ELEMENT ldap-adapter-template (header, view+)> <!-- The header --> <!ELEMENT header (name?, author*, version?)> <!-- Name of template --> <!ELEMENT name (#PCDATA)> <!-- The author(s) --> <!ELEMENT author (#PCDATA)> <!-- Version string --> <!ELEMENT version (#PCDATA)> <!-- View(s) --> <!ELEMENT view (item-descriptor, search-root*)> <!ATTLIST view name default > <!-- Item descriptor(s) --> <!ELEMENT item-descriptor (id-property?, object-classes-property?, object-class*, property*, child-property*, new-items?)> <!ATTLIST item-descriptor name parent display-name description hidden expert display-property > <!-- Id property --> CDATA CDATA CDATA CDATA %flag; %flag; CDATA #REQUIRED #IMPLIED #IMPLIED #IMPLIED "false" "false" #IMPLIED CDATA #REQUIRED %flag; "false"
329
18 - LDAP Repositories
<!ELEMENT id-property EMPTY> <!ATTLIST id-property name CDATA in-ldap ldap-name display-name description > <!-- Object classes property --> <!ELEMENT object-classes-property EMPTY> <!ATTLIST object-classes-property name ldap-name display-name description > <!-- Object class(es) --> <!ELEMENT object-class (#PCDATA)> <!-- Property(s) --> <!ELEMENT property (option*, attribute*)> <!ATTLIST property name ldap-name data-type multi display-name description default required readable writable queryable hidden expert editor-class > CDATA CDATA CDATA %flag; CDATA CDATA CDATA %flag; %flag; %flag; %flag; %flag; %flag; CDATA #REQUIRED #IMPLIED #REQUIRED "false" #IMPLIED #IMPLIED #IMPLIED "false" "true" "true" "true" "false" "false" #IMPLIED CDATA CDATA CDATA CDATA #REQUIRED #IMPLIED #IMPLIED #IMPLIED %flag; CDATA CDATA CDATA #REQUIRED "false" #IMPLIED #IMPLIED #IMPLIED
<!-- Options are possible values for enumerated properties --> <!ELEMENT option (#PCDATA)> <!-- Feature descriptor values --> <!ELEMENT attribute EMPTY> <!ATTLIST attribute name value bean > CDATA CDATA CDATA #REQUIRED #IMPLIED #IMPLIED
330
18 - LDAP Repositories
<!-- Child property(s) --> <!ELEMENT child-property EMPTY> <!ATTLIST child-property name default > <!-- Item creation --> <!ELEMENT new-items EMPTY> <!ATTLIST new-items allowed parent-dn rdn-property > <!-- Search root(s) --> <!ELEMENT search-root EMPTY> <!ATTLIST search-root dn recursive > CDATA %flag; #REQUIRED "true" "true" %flag; CDATA CDATA "true" #IMPLIED #IMPLIED
check-classes
%flag;
331
18 - LDAP Repositories
332
18 - LDAP Repositories
Index
A
access rights repository items, 272 ACL (Access Control Lists) access rights, 279 syntax, 278 add-item tag, 134 alias derivation method, 61 atg.adapter.gsa.* package, 35 atg.adapter.gsa.event.GSAEventServer, 122 atg.adapter.gsa.GSAPropertyDescriptor, 85 atg.adapter.gsa.invalidator.GSAInvalidatorService. See SQL repositories, distributed cache invalidation atg.adapter.gsa.query.SqlPassthroughQuery, 91 atg.repository.content.ContentRepositoryItem, 207 atg.repository.ItemDescriptorImpl, 7 atg.repository.loader.FileSystemMonitorScheduler, 221 atg.repository.MutableRepository, 9, 11 atg.repository.NamedQueryView interface, 100 atg.repository.ParameterSupportView, 92 atg.repository.PropertiesChangedEvent, 13, 112 atg.repository.PropertiesChangedListener, 14, 113 atg.repository.query.QueryDescriptorImpl, 100 atg.repository.QueryBuilder, 10, 17 atg.repository.QueryDescriptor interface, 100 atg.repository.QueryOptions, 18 atg.repository.Repository, 9 atg.repository.RepositoryItem, 6, 11 atg.repository.RepositoryItemDescriptor, 7 atg.repository.RepositoryUtils, 14, 136 atg.repository.RepositoryView, 10 atg.repository.RepositoryViewContainer, 10
B
BLOBs, 162
C
cache invalidation SQL repository. See caches (SQL repository), flushing Cache Invalidator. See SQL repositories, distributed cache invalidation caches (composite repository), 255 caches (LDAP repository) item cache, 315 query cache, 316
caches (SQL repository), 111 cache contents, viewing, 127, 174 cache invalidation events, 112 cache modes, 113 configuring caches, 123 disabling caching, 115 distributed cache invalidation, 130 distributed cache mode, 121 flushing, 128 grouping properties, 85 invalidation, 112 isolation levels, 116 item caches, 111, 125 item descriptor inheritance, 132 loading, 127, 173 Lock Managers, 118 locked cache mode, 116 query caches, 111, 124 simple cache mode, 115 statistics on caches, 126 timeout for item caches, 125 timeout for query caches, 125 write locks, 116 cascade delete. See SQL repositories, cascading operations cascade delete order. See SQL repositories, cascading operations cascade insert. See SQL repositories, cascading operations cascade update. See SQL repositories, cascading operations ClientLockManagers, 118 CLOBs, 162 cloning repository items. See SQL repositories, cloning repository items collectiveUnion derivation method, 62 composite repositories, 249 caching, 255 configuring, 251 definition files, 255 document type definition (DTD), 263 excluding properties, 252 item descriptors, 249 link methods, 252 link-via-id, 262 link-via-property, 262 non-serializable properties, 250 primary-item-descriptor-link, 261 property mapping, 251 transient properties, 250
333
Index
CompositeRepository components configuring, 253 configuration files, secured repositories. See secured repositories, definition files constraints REFERENCES, 44 content repositories, SQL, 207 ContentHandler, 222 ContentRepositories component secured repositories, 276
F
file combination, XML, 37 FileSystemMonitorScheduler, 221 configuring, 229 FileSystemMonitorService, 221 configuring, 227 filterQuery property, 91 firstNonNull derivation method, 60 firstWithAttribute derivation method, 60 firstWithLocale derivation method, 61 flushing caches. See caches (SQL repository), flushing full text search queries, 26
D
database meta data, 105 data-type correspondences LDAP repository properties to Java types, 323 SQL database to SQL repository, 160 date properties SQL repositories, 68 debugLevel property, 145 definition files LDAP repositories. See LDAP repository definition files secured repositories. See secured repositories, definition files SQL repositories. See SQL repository definition files derivation methods alias, 61 collectiveUnion, 62 firstNonNull, 60 firstWithAttribute, 60 firstWithLocale, 61 union, 62 derived properties, 56, 164 alias derivation method, 61 collectiveUnion derivation method, 62 derivation methods, 59 firstNotNull derivation method, 60 firstWithAttribute derivation method, 60 firstWithLocale derivation method, 61 override properties, 58 union derivation method, 62 distinguished names. See LDAP repositories, distinguished names DNs. See LDAP repositories, distinguished names Document Type Definition (DTD) composite repository, 263 LDAP repository definition files, 328 Repository Loader manifest file, 225 secured repository definition files, 284 SQL repository definition files, 175 DTD. See Document Type Definition (DTD)
G
Generic SQL Adapter (GSA). See SQL repositories GetRepositoryItem Web services, 241 GSACacheInvalidator. See SQL repositories, distributed cache invalidation
I
inheritance, item descriptor LDAP repositories, 303 SQL repositories, 50 item caches. See caches item descriptors, 6, 7 Dynamic Beans, 8 inheritance, LDAP repositories, 303 inheritance, SQL repositories, 50 LDAP repositories, 300
J
java.sql.CallableStatement, 99 java.sql.ResultSet, 99 javax.naming.Context, 314 JNDI accessing an LDAP directory, 298
L
last-modified properties SQL repositories, 68 LDAP (Lightweight Directory Access Protocol), 293, 295 data representation, 296 directory schema, 297 distinguished names, 296 entries, 296 entry attributes, 296 entry types, 296 hierarchical tree structure, 295 JNDI, access through, 298 object classes, 296 LDAP repositories, 293 attribute tags, 306
E
enumerated properties LDAP repository, 305 SQL repositories, 65 events cache invalidation, 112 property changes, 112 EventServer components, 122
334
Index
N
named queries, 95 null values SQL repositories, 69 SQL repository queries, 104
configuring components, 310 creating new repository items, 306 definition file, 300 directory schema sample, 297 distinguished names, 300 enumerated properties, 305 inheritance, item descriptor, 303 InitialContextEnvironment component, 310, 313 InitialContextPool component, 312 item cache, 315 item descriptors, 300, 303 LDAPItemCacheAdapter component, 316 LDAPQueryCacheAdapter component, 317 LDAPRepository component, 311 overview, 294 password encryption, 317 property tag attributes, 305 queries, 309 queries, ID matching, 309 queries, unsupported, 310 query cache, 316 Repository API implementation, 299 repository IDs, 300, 304 repository items, 300 Repository Views, 307 search roots, 307, 309 security authentication, 314 LDAP repository definition files, 318 attribute tag, 324 child-property tag, 325 data-type correspondences, 323 DOCTYPE tag, 318 document type definition (DTD), 318 header tag, 318 id-property tag, 320 item-descriptor tag, 319 ldap-adapter-template tag, 318 new-items tag, 326 object-class tag, 321 object-classes-property tag, 320 option tag, 324 property tag, 322 sample, 327 search-root tag, 326 view tag, 319 Lightweight Directory Access Protocol. See LDAP (Lightweight Directory Access Protocol) LoaderManager, 221 configuring, 228 loading caches. See caches (SQL repository), loading Lock Managers, 118 locked caching mode. See caches (SQL repository), locked caching mode
O
Oracle ConText full text search engine, 27 outer joins in SQL repository queries, 104 override property in derived properties, 58
P
parameterized queries, 92 password encryption LDAP repository, 317 password hashing. See password encryption PerformRQLCountQuery Web services, 245 PerformRQLQuery Web services, 243 PropertiesChangedEvent, 13 property fetching SQL repositories, 85 property mapping Composite repositories, 251
Q
queries. See also Repository Query Language (RQL) ATG Control Center, 22 bypassing RQL, 91 composite repository, 254 LDAP repository, 309 named, 95 parameterized, 92 QueryBuilder interface, 17 QueryOptions, 18 Repository API, 17 Repository API example, 19 stored procedures, 98 queries, unsupported LDAP repositories, 310 SQL repositories, 106 query caches. See caches parameterized queries, 93
R
REFERENCES constraints, 44 remove-item tag, 136 repositories data-types, 8 item descriptors. See item descriptors LDAP. See LDAP repositories mutable, 9, 11 queries. See queries
M
manifests Repository Loader, 225 Repository Loader tags, 226 meta data database, 105
335
Index
Repository API summary, 9 secured. See secured repositories SQL. See SQL repositories repository filtering, 89 repository IDs, 7 repository items, 6, 11 adding, 134 cloning, 14 deleting, 136 IDs, 7 LDAP repository, 300 PropertiesChangedEvent, 13 removing references, 136 updating, 12, 136 Repository Loader, 207, 219 administration UI, 222 client, 223 ErrorPolicy, 223 example, 234 manifest tags, 226 manifests, 225 Repository Query Language (RQL), 22 ALL queries, 26 collection queries, 25 comparison queries, 24 COUNT queries, 26 examples, 30 full text search queries, 26 grammar definition, 30 ID-based queries, 27 INCLUDES ITEM queries, 26 IS NULL queries, 26 limiting result sets, 28 logical operators (AND, OR, NOT), 25 ORDER BY directives, 28 ordering query result sets, 28 parameterized queries, 29 pattern match queries, 24 property of property queries, 24 RANGE directive, 28 text comparison queries, 24 Repository View. See item descriptor Repository Views LDAP repository, 307 RepositoryUtils. See atg.repository.RepositoryUtils required properties SQL repositories, 67 RL module. See Repository Loader RLClient, 223 hints file, 224 RQL. See Repository Query Language (RQL) RQL filters. See repository filtering rqlFilterString property, 91
S
secured repositories, 271 access rights, 272 ACL property, 274, 282 ACLs, 278
configuration files. See secured repositories, definition files, See definition files ContentRepositories component, 276 creating, 273 definition files, 276, 280, 289 document type definition, 284 examples, 287 exceptions, 291 limitations, 273 owner-property, 274, 282 Secured Repository Adapter components, 275 Secured Repository Adapter components configuring, 275 security repositories. See secured repositories security authentication LDAP repository, 314 serializable repository items composite repositories, 250 ServerLockManagers, 118 session backup repository items, 63 SQL content repositories, 207 SQL repositories, 33 <add-item> tag, 134 <remove-item> tag, 136 <update-item> tag, 136 architecture, 35 cache groups, 85 cascading operations, 48 cloning repository items, 14 Content window, 195 database column names, 44 debugLevel property, 145 derived properties, 56, 164 distributed cache invalidation, 130 Document Type Definition, 175 enumerated properties, 65 id properties, 41 idSpaces, 42 large database columns, 86 null values in NOT queries, 104 outer joins, 104 property fetching, 85 property-type attributes, 80 queries, unsupported, 106 registering, 195 removing references to items, 136 Repository component, configuring, 195 rql-filter tag, 90 session backup, 63 setting up, 33 streams, 86 tag attribute, 134 transactions, 35 uiqueryable attribute, 77 uiwritable attribute, 77 user-defined property types, 79 wildcards in queries, 103 SQL repository definition files, 35 DTD, 175 SqlPassThroughQuery, 91
336
Index
unique properties SQL repositories, 68 unsupported queries. See queries, unsupported update-item tag, 136 User Authority components, 278 user-defined property types, 79
T
table ownership, 105 tag attribute, 134 timestamp properties SQL repositories, 68 transactions repositories, 35 transient properties, 75 caching, 113 composite repositories, 250 TypeMapper, 221 configuring, 230 TypeMapping, 221 configuring, 230
W
Web services, 241 GetRepositoryItem, 241 PerformRQLCountQuery, 245 PerformRQLQuery, 243 security policies, 247 wildcard characters SQL repository queries, 103
X
XML file combination, 37 Xml2RepositoryContentHandler, 222 configuring, 233
U
union derivation method, 62
337
Index