चलो एक और सुनो

कहाँ पूछूं खुद से परेशां क्यूँ हैं हम?
धुंधला लगे है आईना जो देखूं खुद को।

Colocating data in Coherence and NoSQL

Key/Value pair based distributed systems that partition data based on some hashing of keys pose a challenge of unnecessarily incurring additional network cost in storing and reading of strongly associated data with additional challenges of atomicity of operations across this data set. Fortunately both of Oracle's best of breed big distributed data source Coherence and NoSQL support co-locating associated entries and provide a rich set of APIs to create atomic operations. In both the systems association is dictated by the Map' keys, only differently.

Association in Coherence

Two entries will be co-located in the same partition if

1. The key of the associated entry implements a Coherence KeyAssociation interface (Or define a KeyAssociator and configure it in the cache config).
2. If these two entries are in two different NamedCaches then they must share the same cache service.
3. When the cache service calculates the partition Id for entries, it uses the associated key if the cache key implements the KeyAssociation interface. 
4. Coherence provides a set of APIs typically called TransactionLite framework to access associated entries from the BackingMap as a single unit of transaction. You can refer to the blog for an example.
5. Because Coherence keys can be complex objects it is much more flexible in terms of storing complex data structure.

Association in NoSQL

NoSQL has much simpler design but it is limited in a sense that it requires keys to be Strings.

1. NoSQL keys are structured as multiple major and minor keys.
2. If we are storing People in NoSQL and want everyone with the same last name to be in the same partition then, keys can be structured as:
1. /familylastname/-/myfirstname
2. /familylastname/-/wifesfirstname
3. /familylastname/-/kidsfirstname
3. Though these are three Map entries as they all share the same major key /familylastname will all end up in the same partition.
4. NoSQL then provides APIs like multiDelete() and multiGet() to delete or retrieve multiple keys as one atomic operation.

Working with Coherence and NoSQL together  

Oracle's Elastic Charging Engine is one such product that uses Coherence and NoSQL together with Coherence as a rating engine and NoSQL as its fast persistence store for the processed RatedEvents. That said aside, such integration is fast becoming very common and following pieces of codes demonstrate some of its use cases.

Code Samples

Lets start with an interface:

public interface NoSQLKeyAssociation extends KeyAssociation {
    String getRootKey();
}

public abstract class PartitionAwareNoSQLCacheStore extends AbstractCacheStore
 {
    private KVStore kvStore;

    public PartitionAwareNoSQLCacheStore(String kvServers) {
        KVStoreConfig kvStoreConfig = new KVStoreConfig("addressStore", kvServers);
        kvStoreConfig.setConsistency(Consistency.NONE_REQUIRED);
        kvStore = KVStoreFactory.getStore(kvStoreConfig);
    }

    @Override
    public void store(Object key, Object value) {
        storeAll(Collections.singletonMap(key, value));
    }

    @Override
    public void storeAll(Map map) {
        Map<NoSQLKeyAssociation, Object> transformedMap = transformer(map);

        Set<Map.Entry<NoSQLKeyAssociation,Object>> set;
        set = transformedMap.entrySet();
        ArrayList majorComponents;
        ArrayList minorComponents;

        for (Map.Entry entry : set) {
            majorComponents = new ArrayList();
            minorComponents = new ArrayList();
            NoSQLKeyAssociation associationKey = entry.getKey();
            majorComponents.add((String)associationKey.getAssociatedKey());
            minorComponents.add(associationKey.getRootKey());

            Key key = Key.createKey(majorComponents, minorComponents);
            byte [] bytes = ExternalizableHelper.toByteArray(entry.getValue())
            kvStore.put(key, Value.createValue(bytes));
        }

    }

    public abstract Map transformer(Map entries);
    
}

The method transformer( ) allows for the Cache Store's concrete class to either transform the Map entries in some form or return the Map as is if the keys are already of type NoSQLKeyAssociation. If not this method gives an opportunity to convert the key to a type of NoSQLKeyAssociation, for one.

In example where we need to store a Customer and all his Addresses a key similar to the following can be used, both to store these values in Coherence' NamedCache (CUSTOMER and ADDRESS caches) and also to NoSQL and in each system co-located together.

public class CustomerAddressAssociationKey implements NoSQLKeyAssociation {
    String customerId;
    String addressType;

    public CustomerAddressAssociationKey(String customerId, String addressType) {
        this.customerId = customerId;
        this.addressType = addressType;
    }

    @Override
    public Object getAssociatedKey() {
        return customerId;
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;

        CustomerAddressAssociationKey that = (CustomerAddressAssociationKey) o;

        if (!addressType.equals(that.addressType)) return false;
        if (!customerId.equals(that.customerId)) return false;

        return true;
    }

    @Override
    public int hashCode() {
        int result = customerId.hashCode();
        result = 31 * result + addressType.hashCode();
        return result;
    }

    @Override
    public String getRootKey() {
        return addressType;
    }
}

If two addresses HomeAddress (addressType = HOME) and WorkAddress (addressType = WORK) are created for a Customer with customerId = 1 then they will be stored as following:

In Coherence, Customer and his two addresses will be co-located as:
CUSTOMER Cache:

• <1, Customer>

ADDRESS Cache:

• <<customeraddressassociationkey<HOME>, HomeAddress>
• <<customeraddressassociationkey<WORK>, WorkAddress>

In NoSQL Customer's both address co-located as:
/1/-/HOME, byte[] for HomeAddress
/1/-/WORK, byte[] for WorkAddress

Enjoy!

Speech on assault on rave parties in India

Thanks for giving me time to share my thoughts. India is a free nation. It is a free democracy and it is a free society. As a citizen of India either by birth or by anyway other, we get certain constitutional rights. Rights that are derived from core human and our centuries old traditional values. And one of those rights is to choose how we live.



That said, unlike many western societies our social security is still families. Families are at the core of our fabric. Our democracy derives its energy from it. Mahatma's dreams of Panchayati Raj is a clear indication that in India democracy flows bottom up. If we need support - financial, psychological or emotional we revert to our families, our parents our siblings, our Pind, our community. And these societies over the years form what I call soft laws including certain behavioral guidelines. These guidelines are not written by a few on a piece of paper but are passed down to them by generations over centuries. Its the cost of social security they provide. Then there sits a big Gorilla called Indian constitution that supersedes all such laws and allows anyone who feels threatened by such social laws to seek protection under it. Such social laws can not be used as a justification to perpetrate violence or assault on people and law must take strong action against whoever does that.
I abhor such people and act, but I do support the very intent of such protest against the so called Rave parties. Some of the news that I have read about are nothing but disgusting. For pete's sake some of them were found with packets of condoms lying around with free flowing alcohol and nakedness. Would as a Dad, as a Parent as a family I approve my kid to go to such parties? Of course not and so am sure a bulk of this audience would not too. As a father I would do everything to register my protest against such events if I see that happen that I feel threatens my family and my values. I would not take a gun and shoot them up or break in or slap them around but I will make sure my and many such voices are strongly heard. Unlike many misconceptions Parents even in many western countries feel the same anguish and anger when they hear their kid is drinking, taking drugs or involved in sexual partnerships. Its a parental instinct and I strongly support my rights to do so.
When we were young 18, 19 or 21 we believed behavioral guidelines can not be thrust upon us and this is something that a Generation dictates. Let me say to you in '70s we saw a hippie culture. Where is that now? Are they relevant anymore? A lifestyle evolved in one generation has been disapproved by subsequent ones. So over a few generations the guidelines put forth by societies broadly remains by and large the same. And a sustainable change requires it being slow and not contextual. At the end it is us who make changes or safeguard our beliefs.

Thank you