Wednesday, December 30, 2009

I don't know which cache this Object goes to

..This was something that came up during a sluggish talk at a customer's site. A developer had just jokingly remarked.

Problem Statement: How to make Coherence objects destination aware?

One way is to annotate the objects that are to be cached. So lets create some simple annotations:

@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target ({ElementType.TYPE, ElementType.PACKAGE})
public @interface Cache {
String name();
int ttl() default -1;
String pk() default "getId";
}
All this annotation is telling is the name of the cache, what should be the TTL (default being -1) and how to find the primary key for this object? The annotation only complements the coherence-cache-config.xml and does not replace it (Unlike JPA annotations). You still need to define all cache names in the configuration file. Next step is to create a sample domain object annotated with Cache:

@Cache(name = "CacheDestination", ttl=-1, pk="getId")
public class Domain implements Serializable {
private Key id;
private int age;
private String name;

public Domain() {
}

public void setId(Key id) {
this.id = id;
}

public Key getId() {
return Key;
}

public void setAge(int age) {
this.age = age;
}

public int getAge() {
return age;
}

public void setName(String name) {
this.name = name;
}

public String getName() {
return name;
}

@Override
public String toString () {
return id + ":" + age + ":" + name;
}
}
Unfortunately this destination aware class will not do anything unless the class that puts this Object in the cache extracts the destination. Cache annotation also tells which method to call to find it's primary key, in this sample it is getId () (Remember JPA?). For developers who do not want to "know" which cache to put this object to, need an Abstraction layer. Quite a few customers that I have worked with try to create an Abstraction layer on top of Coherence to make "all caching" logic to be centralized in one layer. Coherence APIs are already highly abstracted. NamedCache is probably the most important interface in all of Coherence APIs. NamedCache already extends java.util.Map. So having an abstraction layer that returns an instance of NamedCache is good enough for Map centric applications. Problem is that NamedCache is not just a Map. It also provides quereablility feature on top of this data structure. It also provides support for Events, transactional support and process invocation features that are missing from a vanilla Map contract. Most likely this abstraction layer will miss some of these native features of NamedCache that even JCACHE (JSR107) dictates only a subset of features of. The challenge here is that NamedCache has no knowledge of objects being destination aware or in simple English does not understand the Cache annotation, unless I find Product team drunk enough to put this idea forward to. So we need to build one. Lets create a ReducedMap:
public interface ReducedMap {
public Object put (Object value) throws NoSuchMethodException,
IllegalAccessException, InvocationTargetException,
OperationNotSupportedException;
}
and implement it in a custom Map using Java delegation pattern to pass cache invocations to Coherence.
public class AnnoMap extends AbstractMap implements ReducedMap {
private volatile NamedCache nCache;

public AnnoMap() {
}

public int size() {
return (nCache == null) ? 0 : nCache.size();
}

public boolean isEmpty() {
return (nCache == null) ? false : nCache.isEmpty();
}

public Set entrySet() {
return (nCache == null) ? Collections.EMPTY_SET : nCache.entrySet();
}


public boolean containsKey(Object key) {
if (nCache == null) {
Class clz = key.getClass();
Cache c = (Cache)clz.getAnnotation(Cache.class);
if (c != null) {
String name = (c.name() == null) ? clz.getName() : c.name();
nCache = CacheFactory.getCache(name);
}
}
return (nCache == null) ? false : nCache.containsKey(key);
}

public Object put(Object value) throws NoSuchMethodException,
IllegalAccessException, InvocationTargetException,
OperationNotSupportedException {
Class clz = value.getClass();
Cache c = (Cache)clz.getAnnotation(Cache.class);
if (c != null) {
String name = (c.name() == null) ? clz.getName() : c.name();
nCache = CacheFactory.getCache(name);
Method m = value.getClass().getMethod(c.pk(), new Class[0]);
Object key = m.invoke(value, null);
return nCache.put(key, value, c.ttl());
}
throw new OperationNotSupportedException("Class not annotated");
}


public Object get(Object key) {
if (nCache == null) {
Class clz = key.getClass();
Cache c = (Cache)clz.getAnnotation(Cache.class);
if (c != null) {
String name = (c.name() == null) ? clz.getName() : c.name();
nCache = CacheFactory.getCache(name);
}
}
return (nCache == null) ? null : nCache.get(key);
}

public Object put(Object key, Object value) {
Class clz = value.getClass();
Cache c = (Cache)clz.getAnnotation(Cache.class);
if (c == null)
throw new UnsupportedOperationException("Value not annotated");
String name = (c.name() == null) ? clz.getName() : c.name();
nCache = CacheFactory.getCache(name);
return nCache.put(key, value);
}

}
Now as you can see AnnoMap is very limited in its capability but if you are taking this route probably this Map is the place to enhance. Lets run a quick test:
public static void main(String[] args) throws NoSuchMethodException,
IllegalAccessException, InvocationTargetException,
OperationNotSupportedException {
Domain d = new Domain ();
// -- KeyObject will be similar to Domain object with the same Cache
// -- destination defined in the Cache annotation.

Key key = new KeyObject (...);
d.setId(k);
d.setXXX(..);
AnnoMap map = new AnnoMap ();
map.put(d);

System.out.println(map.get (key));
}
Enjoy!

Saturday, December 26, 2009

Thursday, December 17, 2009

बहुत दिनों बाद एक शेर और

इस तरफ सूखी सी नदिया, उस तरफ सैलाब पानी का,
बतला दे ये खुदा मुझे अब, मैं तैरुं तो तैरुं किस तरफ?
इस तरफ टूटी सी नैया, उस तरफ पता न मांझी का,
अब खामोश मत रह, बता मैं बैठूं तो बैठूं किस तरफ?
खड़ा धंसती जमीं पर अब दोराहे पे अकेला,
बतला दे ये खुदा मुझे अब, मैं जाऊं तो जाऊं किस तरफ?

Tuesday, November 24, 2009

Thanks Congress for giving BJP what they wanted... An Issue

Burdens of past should not drag the wishes of future. Ram Temple is a similar issue that has become a playground to gain political mileage. Its in the news again and of course thanks to Congress for giving BJP an issue, once again. You would be politically naive to believe BJP leadership had no knowledge of plans of bringing the disputed structure down while in power in the state where it happened. Level of involvement of its leaders could be debatable but their ignorance is something that can not be believed. It can not be believed in a similar way as if the structure was a center of any relevance in the middle of a Temple town, the birth place of Ram. The disputed structure was less of a religious importance to Muslims in India than being a relic of the history. It neither represented Islamic culture nor an icon of introduction of Islam in India other than a thumping sign of Victory of a Muslim King over the local Hindu rulers. The place also called Masjid-i-Janmasthan was a place where till late 19th century Hindus and Muslims prayed together peacefully. Today more than a few centuries later we have a constitution to respect and abide by that is key to the survival of Indian democracy and is also the life line of our individual identities. BJP who championed the cause of reconstruction of Ram temple, after losing a series of recent political battles and thoroughly rejected by people sees this issue as a life source of its own existence. This is what BJP wanted and so does the Samajwadi Party who got rejected by its own traditional Muslim electorate in the recent elections. BJP is not for Hindus, so not SP for Muslims and Congress is no champion of secularism either. So it is now up to the common mass to scream and let them know that one of the most pious places of Hindu religion can't be used to spill blood of any fellow Indians irrespective of their religious beliefs or its bloody past.

Sunday, November 15, 2009

The clouds from above


मैं क्या कहूँ इन बादलों को देख कर यूँ अब?
लगें नीचे से नम आँखें और ऊपर से मेरे सपने।
सुनहरे सुर्ख फूलों कि ये लम्बी नर्म सी चादर,
बिछी इक छाओं सी ऊपर चाहे ये हमे ढंकने।
या शायद है ये इक नदिया बहे जो यूँ तूफानी सी,
और पानी हैं मेरे सपने जो बादल बन सदा बरसें।

Friday, November 13, 2009

My Name with ditaa

Sunday, November 01, 2009

Functional Programming with Coherence

If you come across the following, don't get surprised. Its called Functional programming.

object FP {
def doSomething (callback: () => Unit) {
callback();
}
def letsPrint() {
println ("Ha ha!");
}
def main (args: Array[String]) {
doSomething (letsPrint)
}
}
Functions in Scala are treated as Objects that can be passed around. The concept of Object has still not changed - Its a place holder of application state. So in Functional Programming Model, functions are states and if it is legal then why not manage them in a state repository like Oracle Coherence?

Problem Statement: Executing new Entry Processors without having to deploy them in the cluster and continue to achieve 100% up time?

In a simpler English it means how can I push new EntryProcessor(s) and execute them without having to bounce the cluster to deploy the new class. The solution lies in the concept of Functional Programming - Passing functions to the Executor but this time slightly in a different way. Lets again, begin with a cache configuration:
<!DOCTYPE cache-config SYSTEM "cache-config.dtd">
<cache-config>
<caching-scheme-mapping>
<cache-mapping>
<cache-name>EPFeeder</cache-name>
<scheme-name>feeder-scheme</scheme-name>
</cache-mapping>
<cache-mapping>
<cache-name>EPCache</cache-name>
<scheme-name>ep-scheme</scheme-name>
</cache-mapping>
</caching-scheme-mapping>

<caching-schemes>
<class-scheme>
<scheme-name>ep-scheme</scheme-name>
<class-name>DynaEPCache</class-name>
<init-params>
<init-param>
<param-type>string</param-type>
<param-value>{cache-name}</param-value>
</init-param>
<init-param>
<param-type>string</param-type>
<param-value>coherence-cache-config.xml</param-value>
</init-param>
</init-params>
</class-scheme>
<distributed-scheme>
<scheme-name>feeder-scheme</scheme-name>
<service-name>EPFeederScheme</service-name>
<backing-map-scheme>
<read-write-backing-map-scheme>
<internal-cache-scheme>
<local-scheme>
<high-units>100MB</high-units>
<listener>
<class-scheme>
<class-name>EPListener</class-name>
</class-scheme>
</listener>
</local-scheme>
</internal-cache-scheme>
</read-write-backing-map-scheme>
</backing-map-scheme>
<autostart>true</autostart>
</distributed-scheme>

<proxy-scheme>
...
</proxy-scheme>

<invocation-scheme>
<scheme-name>InvocationService</scheme-name>
<service-name>InvocationService</service-name>
<thread-count>2</thread-count>
<autostart>true</autostart>
</invocation-scheme>
</caching-schemes>
</cache-config>

EPFeeder Cache is where we store the Functions (Implementations of process() method of the EntryProcessor) against the EP's class name. EPCache is a demo cache where your data resides and you would be executing new EPs against. Next Step is to see how the client code will look like:
public class DynaCUtilTest extends TestCase {
....
private String getEPImpl() {

StringBuffer sBuffer = new StringBuffer();
sBuffer.append("public Object process (Entry entry) {");
sBuffer.append ("System.out.println(\"In process\");");
sBuffer.append ("System.out.println(\"Key:\" + entry.getKey());");
sBuffer.append ("System.out.println(\"Value:\" + entry.getValue());");
sBuffer.append("return null;");
sBuffer.append("}");

return sBuffer.toString();
}

public void testCreateEP()
String impl = getEPImpl();
String clzName = "EPClass_v1";
NamedCache eCache = CacheFactory.getCache("EPFeeder");
eCache.put(clzName, impl);

NamedCache nCache = CacheFactory.getCache("EPCache");
nCache.invoke("A", new EPClass_v1());
}
}
getEPImpl () is the implementation that we would replace the process() method with as we feed new EP classes, in this case the first version of it named EPClass_v1. So what happens next?
When a new implementation is put in the EPFeeder Cache, A Backing Map Listener picks up this event and creates a new class (EntryProcessor) on all the cluster members dynamically using an Invocation Service. This step achieves 100% up time for Coherence. The Backing Map Listener (EPListener) looks something like this:

public class EPListener extends MultiplexingMapListener {

public EPListener() {
}

protected void onMapEvent(MapEvent mapEvent) {
String key = ...;
String impl = ...;
InvocationService iS =
(InvocationService)CacheFactory.getService("InvocationService");
Invocable inv = new EPCreator (key, impl);
// -- Create a new Class on all nodes
Set set = CacheFactory.getCluster ().getMemberSet();
iS.query(inv, set);
}

}
The core of this listener is the magic EPCreator Invocable but before we look at the EPCreator let see an EPInterface:
import com.tangosol.util.InvocableMap;

public interface EPInterface extends InvocableMap.EntryProcessor {
}
The most critical piece of this puzzle is the Invocable and how it does its magic. The EPCreator executes the following in it's run() method. Lets put it in its own Util class (DynaCUtil):
    public static Class createEP(String clzName, String impl) {
ClassPool pool = ClassPool.getDefault();
pool.importPackage("com.tangosol.util.InvocableMap.EntryProcessor");
pool.importPackage("com.tangosol.util.InvocableMap.Entry");
Class clz = null;
CtClass eClass = null;
boolean shouldCreate = false;
try {
eClass = pool.get(clzName);
} catch (NotFoundException e) {
shouldCreate = true;
}
if (shouldCreate) {
eClass = pool.makeClass(clzName);
eClass.setInterfaces(new CtClass[] {
pool.makeClass("EPInterface") });
try {
eClass.addConstructor(CtNewConstructor.defaultConstructor(eClass));
} catch (CannotCompileException e) {
e.printStackTrace();
}
try {
eClass.addMethod(CtNewMethod.make(impl, eClass));
StringBuffer sBuffer = new StringBuffer ();
sBuffer.append ("public java.util.Map processAll(java.util.Set set) {");
sBuffer.append ("System.out.println(\"In processAll\");");
sBuffer.append ("return java.util.Collections.EMPTY_MAP;");
sBuffer.append ("}");
eClass.addMethod (CtNewMethod.make (sBuffer.toString(), eClass));

} catch (CannotCompileException e) {
e.printStackTrace();
}
try {
clz = eClass.toClass();
} catch (CannotCompileException e) {
e.printStackTrace();
}
}

What the heck was it? The Invocable uses JavaAssist to create a new EntryProcessor on the fly. Now the last question is as Coherence is a self-healing system where new nodes can join and leave anytime, how to make sure new EP Classes are available to the new nodes? And the answer is a Custom NamedCache which also is the last piece in the puzzle. The class would look something like the following:

public class DynaEPCache extends WrapperNamedCache {

public Object invoke(Object oKey, InvocableMap.EntryProcessor agent) {
String name = agent.getClass().getName();
createEP (name, (String) CacheFactory.getCache("EPFeeder").get (name));
return super.invoke(oKey, agent);
}

public Map invokeAll(Collection collKeys, EntryProcessor agent) {
....
}

private void createEP (String name, String impl) {
if (impl == null) {
throw new RuntimeException ("EntryProcessor not created yet!");
}
DynaCUtil.createEP(name, impl);
}
}

A much more advanced implementation is sitting on my laptop that a pieces I will soon upload to http://sites.google.com/site/miscellaneouscomponents/Home. In the meantime just Enjoy!