Thursday, November 13, 2008

Managing a collection of data and running Query By Example with Coherence

Problem (1): Coherence NamedCache extends java.util.Map i.e, the data to be managed in the cache must be in a <key>, <value> pair. One of the challenges to manage just a Collection of keys or elements in a cache is to associate these keys with some values. If Values are not given we need to use a Serializable Object that can become it's values (or Nulls). So your collection {a, b, c ..., z} becomes [{a, <o>}, {b, <o>}, ... {z, <o>}] with o being:

public class o implements Serializable { }
Problem (2): Running Queries against the Keys. Coherence supports KeyExtractor that extracts attribute values from cache keys instead of on the Values. The Filter will look like filter = new EqualsFilter(new KeyExtractor("getAge"), value); that matches if the getAge () method of a cache key Object is equal to the value.
Problem (3): Query By Example. This is another challenge. Coherence cache supports Filter mechanism that can be run on existing value for its attributes. Out of the box there is no mechanism that can parse Query By Example. QBE means an Object with partial attribute set is passed to find out all or any cache keys that have same attribute values. Lets introduce a Discoverer class that has following:
protected Filter getFilter(String method, String value) {
Filter filter = null;
if (!(value.equals("*") || value.equals(""))) {
filter = new EqualsFilter(new KeyExtractor(method), value);
} else {
filter = new GreaterEqualsFilter(new KeyExtractor(method), "");
}
return filter;
}
Problem (4): Combining Key searches into one service to support multiple key types. First step is to make sure that each cache stores a specific type of Keys. And if one type of Key is very different from another type of key then create dedicated Discoverer classes that specializes in finding a specific type of Key. Cache configuration could look like:

<caching-scheme-mapping>
<cache-mapping>
<cache-name>Key1
<scheme-name>distributed-scheme
<init-params>
<init-param>
<param-type>string
<param-value>com.some.Key1
</init-param>
</init-params>
</cache-mapping>
<cache-mapping>
<cache-name>Key2
<scheme-name>distributed-scheme
<init-params>
<init-param>
<param-type>string
<param-value>com.some.Key2
</init-param>
</init-params>
</cache-mapping>
</caching-scheme-mapping>

Then maintain a local map that stores cache to Object type mapping as:
XmlElement elem = null;
NamedCache nCache = null;
XmlElement cSMap =
CacheFactory.getConfigurableCacheFactory().getConfig().getElement("caching-scheme-mapping");
Iterator mappings = cSMap.getElements("cache-mapping");
String dataType = null;
while (mappings.hasNext()) {
elem = (XmlElement) mappings.next();
nCache = CacheFactory.getCache(elem.getElement("cache-name").getString());
try {
dataType =
elem.getElement("init-params").getElement("init-param").getElement("param-value").getString();

objToMap.put(nCache.getCacheName(), dataType);
mapToObj.put(dataType, nCache);
} catch (Exception exp) {
// -- Log the Exception and skip adding Indexes
log(exp);
}
}
Once mapping is known, create a Discoverer as:
public Collection getKeys(Key filterKey) {
IDiscoverer disc;
try {
disc = (IDiscoverer) keyDiscoverer.get(filterKey);
if (disc == null) {
// -- Even though the cost of Reflection is high but to it takes
// -- a microsecond to initialize an Object. To reduce the
// -- reoccuring cost use a local Map to store the instance
String cName = filterKey.getClass().getName();
String discName =
cName.substring(cName.lastIndexOf(".") + 1) + "Discoverer";
disc =
(IDiscoverer) (this.getClass().getClassLoader().loadClass("com.some." +
discName)).newInstance();
keyDiscoverer.put(filterKey, disc);
}
return disc.getKeys(filterKey);
} catch (Exception exp) {
log(exp);
}
return null;
}
Once the getKeys () is delegated to right Discoverer, the Discoverer object can work on the key's attributes directly. One example being:

public Collection getKeys(Key key)
throws IllegalAccessException, InvocationTargetException {

if (!(key instanceof Key1)) {
return null;
}

String cName = key.getClass().getName();
NamedCache nCache =
CacheFactory.getCache(cName.substring(cName.lastIndexOf (".") + 1));
Key1 pKey = (Key1) key;
Filter filters[] = new Filter[2];
filters[0] = getFilter("getAttribute1", pKey.getAttribute1());
filters[1] = getFilter("getAttribute2", pKey.getAttribute2());
Filter composite = new AllFilter(filters);
return nCache.keySet(composite);
}
As you see the code uses Java Reflection where ever possible but avoids it in re-occurring calls. The Specialized classes like Key1Discoverer on the other hand works on class attributes directly.
This is a simple but quite efficient approach to manage a collection of data and run Templated Queries on it. Coherence cache configuration dictates Object type to be stored in a cache. Discoverer provides a mechanism to have a specialized way of finding keys and manipulate QBE and single service class using Java reflection allows to scale the key types and its discovery to any number of types. Would be glad to see any suggested improvements...

No comments: