The clouds from above

मैं क्या कहूँ इन बादलों को देख कर यूँ अब?
लगें नीचे से नम आँखें और ऊपर से मेरे सपने।
सुनहरे सुर्ख फूलों कि ये लम्बी नर्म सी चादर,
बिछी इक छाओं सी ऊपर चाहे ये हमे ढंकने।
या शायद है ये इक नदिया बहे जो यूँ तूफानी सी,
और पानी हैं मेरे सपने जो बादल बन सदा बरसें।

My Name with ditaa

Functional Programming with Coherence

If you come across the following, don't get surprised. Its called Functional programming.

object FP {
  def doSomething (callback: () => Unit) {
    callback();
  }
  def letsPrint() {
    println ("Ha ha!");
  }
  def main (args: Array[String]) {
    doSomething (letsPrint)
  }
}

Functions in Scala are treated as Objects that can be passed around. The concept of Object has still not changed - Its a place holder of application state. So in Functional Programming Model, functions are states and if it is legal then why not manage them in a state repository like Oracle Coherence?

Problem Statement: Executing new Entry Processors without having to deploy them in the cluster and continue to achieve 100% up time?

In a simpler English it means how can I push new EntryProcessor(s) and execute them without having to bounce the cluster to deploy the new class. The solution lies in the concept of Functional Programming - Passing functions to the Executor but this time slightly in a different way. Lets again, begin with a cache configuration:

<!DOCTYPE cache-config SYSTEM "cache-config.dtd">
<cache-config>
    <caching-scheme-mapping>
        <cache-mapping>
            <cache-name>EPFeeder</cache-name>
            <scheme-name>feeder-scheme</scheme-name>
        </cache-mapping>        
        <cache-mapping>
            <cache-name>EPCache</cache-name>
            <scheme-name>ep-scheme</scheme-name>
        </cache-mapping>        
    </caching-scheme-mapping>
    
    <caching-schemes>
        <class-scheme>
            <scheme-name>ep-scheme</scheme-name>
            <class-name>DynaEPCache</class-name>
            <init-params>
                <init-param>
                    <param-type>string</param-type>
                    <param-value>{cache-name}</param-value>
                </init-param>
                <init-param>
                    <param-type>string</param-type>
                    <param-value>coherence-cache-config.xml</param-value>
                </init-param>
            </init-params>
        </class-scheme>        
        <distributed-scheme>
            <scheme-name>feeder-scheme</scheme-name>
            <service-name>EPFeederScheme</service-name>
            <backing-map-scheme>
                <read-write-backing-map-scheme>
                    <internal-cache-scheme>
                        <local-scheme>
                            <high-units>100MB</high-units>
                            <listener>
                                <class-scheme>
                                    <class-name>EPListener</class-name>
                                </class-scheme>
                            </listener>
                        </local-scheme>
                    </internal-cache-scheme>
                </read-write-backing-map-scheme>
            </backing-map-scheme>
            <autostart>true</autostart>
        </distributed-scheme>        

        <proxy-scheme>
          ...
        </proxy-scheme>

        <invocation-scheme>
            <scheme-name>InvocationService</scheme-name>
            <service-name>InvocationService</service-name>
            <thread-count>2</thread-count>
            <autostart>true</autostart>
        </invocation-scheme>        
    </caching-schemes>
</cache-config>

EPFeeder Cache is where we store the Functions (Implementations of process() method of the EntryProcessor) against the EP's class name. EPCache is a demo cache where your data resides and you would be executing new EPs against. Next Step is to see how the client code will look like:

public class DynaCUtilTest extends TestCase {
    ....
    private String getEPImpl() {

        StringBuffer sBuffer = new StringBuffer();
        sBuffer.append("public Object process (Entry entry) {");
        sBuffer.append ("System.out.println(\"In process\");");
        sBuffer.append ("System.out.println(\"Key:\" + entry.getKey());");
        sBuffer.append ("System.out.println(\"Value:\" + entry.getValue());");
        sBuffer.append("return null;");
        sBuffer.append("}");

        return sBuffer.toString();
    }

    public void testCreateEP()
        String impl = getEPImpl();
        String clzName = "EPClass_v1";
        NamedCache eCache = CacheFactory.getCache("EPFeeder");
        eCache.put(clzName, impl);

        NamedCache nCache = CacheFactory.getCache("EPCache");
        nCache.invoke("A", new EPClass_v1());
    }    
}

getEPImpl () is the implementation that we would replace the process() method with as we feed new EP classes, in this case the first version of it named EPClass_v1. So what happens next?
When a new implementation is put in the EPFeeder Cache, A Backing Map Listener picks up this event and creates a new class (EntryProcessor) on all the cluster members dynamically using an Invocation Service. This step achieves 100% up time for Coherence. The Backing Map Listener (EPListener) looks something like this:

public class EPListener extends MultiplexingMapListener {

    public EPListener() {
    }

    protected void onMapEvent(MapEvent mapEvent) {
        String key = ...;
        String impl = ...;
        InvocationService iS = 
            (InvocationService)CacheFactory.getService("InvocationService");
        Invocable inv = new EPCreator (key, impl);
        // -- Create a new Class on all nodes
        Set set = CacheFactory.getCluster ().getMemberSet();
        iS.query(inv, set);        
    }

}

The core of this listener is the magic EPCreator Invocable but before we look at the EPCreator let see an EPInterface:

import com.tangosol.util.InvocableMap;

public interface EPInterface extends InvocableMap.EntryProcessor {
}

The most critical piece of this puzzle is the Invocable and how it does its magic. The EPCreator executes the following in it's run() method. Lets put it in its own Util class (DynaCUtil):

    public static Class createEP(String clzName, String impl) {
         ClassPool pool = ClassPool.getDefault();
        pool.importPackage("com.tangosol.util.InvocableMap.EntryProcessor");
        pool.importPackage("com.tangosol.util.InvocableMap.Entry");
        Class clz = null;
        CtClass eClass = null;
        boolean shouldCreate = false;
        try {
            eClass = pool.get(clzName);
        } catch (NotFoundException e) {
            shouldCreate = true;
        }
        if (shouldCreate) {
            eClass = pool.makeClass(clzName);
                eClass.setInterfaces(new CtClass[] { 
                pool.makeClass("EPInterface") });
            try {
                eClass.addConstructor(CtNewConstructor.defaultConstructor(eClass));
            } catch (CannotCompileException e) {
                e.printStackTrace();
            }
            try {
                eClass.addMethod(CtNewMethod.make(impl, eClass));
                StringBuffer sBuffer = new StringBuffer ();
                sBuffer.append ("public java.util.Map processAll(java.util.Set set) {");
                sBuffer.append ("System.out.println(\"In processAll\");");
                sBuffer.append ("return java.util.Collections.EMPTY_MAP;");
                sBuffer.append ("}");
                eClass.addMethod (CtNewMethod.make (sBuffer.toString(), eClass));
                
            } catch (CannotCompileException e) {
                e.printStackTrace();
            }
            try {
                clz = eClass.toClass();
            } catch (CannotCompileException e) {
                e.printStackTrace();
            }
        }

What the heck was it? The Invocable uses JavaAssist to create a new EntryProcessor on the fly. Now the last question is as Coherence is a self-healing system where new nodes can join and leave anytime, how to make sure new EP Classes are available to the new nodes? And the answer is a Custom NamedCache which also is the last piece in the puzzle. The class would look something like the following:

public class DynaEPCache extends WrapperNamedCache {

    public Object invoke(Object oKey, InvocableMap.EntryProcessor agent) {
        String name = agent.getClass().getName();
        createEP (name, (String) CacheFactory.getCache("EPFeeder").get (name));
        return super.invoke(oKey, agent);
    }

    public Map invokeAll(Collection collKeys, EntryProcessor agent) {
        ....
    }

    private void createEP (String name, String impl) {
        if (impl == null) {
            throw new RuntimeException ("EntryProcessor not created yet!");
        }
        DynaCUtil.createEP(name, impl);
    }
}

A much more advanced implementation is sitting on my laptop that a pieces I will soon upload to http://sites.google.com/site/miscellaneouscomponents/Home. In the meantime just Enjoy!

Why should I pay a single penny for Windows 7 upgrade from Vista?

I bought a Sony Vaio with Vista Home premium edition about a year ago. It looked good with new Apple-type interface and I brought it home. It did not take me long to find this was the worst OS Microsoft has ever produced. The most irks I got to find out that I could not VPN to my work network as it failed to connect. With no Vista support offered by my Company I was stuck and had to continue with my other very bulky Dell with XP. This weekend I had a chance to look at Windows(7) at a local BestBuy. It did not feel even a little different from Vista. Now Microsoft is asking for $129 upgrade fee for it. Why? When did I ever use my Vista properly at the first place? You sell me a buggy, practically non-working OS and then asking for a fee to get it fixed, if they have? Isn't this keeping Vista users hostage? Unless I shelve another $129 on top of a costly purchase that we already had made they do not have any other options. For me Windows(7) has already failed in the first week of its launch.

Why measuring exact size in memory could be a futile exercise?

Coherence being an In-memory data grid, it is important to provision the hardware right. Many factors play different roles - Total RAM on the box, avoiding paging, providing linear scalability without stepping on Out of Memory errors and so on. Now the problem is how to measure how many additional nodes (Cache Servers) one would need and in result how many new boxes when we have to scale out? Also, if Indexes are created how to measure the additional space required and how to do it right?
There are two ways to measure things - First, like measuring Gold and Second like measuring Onions. Onions are always approximate. Coherence data sizing is like measuring Onions and its not like you can not measure it like Gold - accurate and precise but in most cases it is not needed. Why? Because the dynamic/auto provisioning nature of the cluster and cheaper memory by the day. It is much easier to approximate the size and add new nodes or boxes in the cluster than to be a Mathematician and calculate the size in bytes. If you are an Operations person you need quick and almost correct formulas. If you are a Coherence enthusiast you might already know it - On a 32-bit machine 1.2GB of RAM is needed to run a JVM with 1GB heap. Off of 1GB heap having only 375MB of space for primary data storage in distributed data scheme with one backup count. Keeping 30% of scratch space left per JVM to keep the GC profile in check and so on. What about Indexes? That's easy too.. Account for 30% overhead for each Index added. So watch for how many Indexes are added as it is easy to cross the data size itself. Are these numbers accurate? Nope. They are not meant to be either. Are they simple? Yes and close to correct. After all when it comes to provisioning a system like Coherence its okay to just measure it like Onions.

A love letter from Nigeria, again!

Just received this letter from that Dead Uncle who I never knew that had migrated to Nigeria I did not know about. A few uncles like these and I will be the richest person on this Nigerian Earth!

I am Mr David Lewis. a Foreign Transfer Manager working with ZENITH BANK of Nigeria.I just started working with ZENITH. and I came across your unpaid fund File stamped hold due to you have not come for the claim.

The most annoying thing is that they won't tell you the truth that on no account will they ever release the fund to you,instead they allow you spend money unnecessarily, or allow the government confiscate your fund, I do not intend to work here all the days of my life, I can release this fund to you if you can certify me of my security.

I needed to do this because you need to know the statues of your Funds and cause for the delay,Please this is like a Mafia setting in Nigeria, you may not understand it because you are not a Nigerian.. The only thing needed to release this fund is the Change Of Ownership which will be tendered to this bank Zenith to prove to them that you have come for the claim of your fund left in your name and the INTERNAL REVENUE S ERVICE(IRS)for clearance of the transferred amount in your account or in any means you will like to receive your fund.

Once the Change Of Ownership is obtained from the Federal High Court here in Nigeria funds will immediately reflect in your bank within 10 Minutes,the document is all that is needed to complete this transaction.

I have the Deposit Certificate for your own proof and the Next Of Kin application form to fill out.

Note that the actual funds is valued at $25 MILLION USD and the president made a compensation fund release for all unpaid beneficiary valued at $15 million usd.
Listed below are the mafia's and banks behind the non release of your funds that i managed to sneak out for your kind persual.

1) Prof. Charles soludo
2) Chief Joseph Sanusi
3) Dr. R. Rasheed
4) Barrister Awele Ugorji
5) Mr Roland Ngwa
6) Barrister Ucheuzo Williams
6) Mr. Ernest Chukwudi Obi
7) Dr. Patrick Aziza
Deputy Governor - Policy / Board Member
8) Mr. Tunde Lemo
Deputy Governor - Financial Sector Surveillance / Board Member
9) Mrs. W. D. A. Mshelia
Deputy Governor - Corporate Services / B oard Members
10) Mrs. Okonjo Iweala

Do get in touch with me immediately with my direct number to conclude this final transaction immediately,and also send to me your convenient tel/fax numbers for easy communications

Regards,
Mr . David Lewis.

This much snow in Denver in October