Why measuring exact size in memory could be a futile exercise?

Coherence being an In-memory data grid, it is important to provision the hardware right. Many factors play different roles - Total RAM on the box, avoiding paging, providing linear scalability without stepping on Out of Memory errors and so on. Now the problem is how to measure how many additional nodes (Cache Servers) one would need and in result how many new boxes when we have to scale out? Also, if Indexes are created how to measure the additional space required and how to do it right?
There are two ways to measure things - First, like measuring Gold and Second like measuring Onions. Onions are always approximate. Coherence data sizing is like measuring Onions and its not like you can not measure it like Gold - accurate and precise but in most cases it is not needed. Why? Because the dynamic/auto provisioning nature of the cluster and cheaper memory by the day. It is much easier to approximate the size and add new nodes or boxes in the cluster than to be a Mathematician and calculate the size in bytes. If you are an Operations person you need quick and almost correct formulas. If you are a Coherence enthusiast you might already know it - On a 32-bit machine 1.2GB of RAM is needed to run a JVM with 1GB heap. Off of 1GB heap having only 375MB of space for primary data storage in distributed data scheme with one backup count. Keeping 30% of scratch space left per JVM to keep the GC profile in check and so on. What about Indexes? That's easy too.. Account for 30% overhead for each Index added. So watch for how many Indexes are added as it is easy to cross the data size itself. Are these numbers accurate? Nope. They are not meant to be either. Are they simple? Yes and close to correct. After all when it comes to provisioning a system like Coherence its okay to just measure it like Onions.

A love letter from Nigeria, again!

Just received this letter from that Dead Uncle who I never knew that had migrated to Nigeria I did not know about. A few uncles like these and I will be the richest person on this Nigerian Earth!

I am Mr David Lewis. a Foreign Transfer Manager working with ZENITH BANK of Nigeria.I just started working with ZENITH. and I came across your unpaid fund File stamped hold due to you have not come for the claim.

The most annoying thing is that they won't tell you the truth that on no account will they ever release the fund to you,instead they allow you spend money unnecessarily, or allow the government confiscate your fund, I do not intend to work here all the days of my life, I can release this fund to you if you can certify me of my security.

I needed to do this because you need to know the statues of your Funds and cause for the delay,Please this is like a Mafia setting in Nigeria, you may not understand it because you are not a Nigerian.. The only thing needed to release this fund is the Change Of Ownership which will be tendered to this bank Zenith to prove to them that you have come for the claim of your fund left in your name and the INTERNAL REVENUE S ERVICE(IRS)for clearance of the transferred amount in your account or in any means you will like to receive your fund.

Once the Change Of Ownership is obtained from the Federal High Court here in Nigeria funds will immediately reflect in your bank within 10 Minutes,the document is all that is needed to complete this transaction.

I have the Deposit Certificate for your own proof and the Next Of Kin application form to fill out.

Note that the actual funds is valued at $25 MILLION USD and the president made a compensation fund release for all unpaid beneficiary valued at $15 million usd.
Listed below are the mafia's and banks behind the non release of your funds that i managed to sneak out for your kind persual.

1) Prof. Charles soludo
2) Chief Joseph Sanusi
3) Dr. R. Rasheed
4) Barrister Awele Ugorji
5) Mr Roland Ngwa
6) Barrister Ucheuzo Williams
6) Mr. Ernest Chukwudi Obi
7) Dr. Patrick Aziza
Deputy Governor - Policy / Board Member
8) Mr. Tunde Lemo
Deputy Governor - Financial Sector Surveillance / Board Member
9) Mrs. W. D. A. Mshelia
Deputy Governor - Corporate Services / B oard Members
10) Mrs. Okonjo Iweala

Do get in touch with me immediately with my direct number to conclude this final transaction immediately,and also send to me your convenient tel/fax numbers for easy communications

Regards,
Mr . David Lewis.

This much snow in Denver in October

Level(3) not accessible

This sign in Denver International Airport's Elevator always reminds me of Level(3) a network company.

BJP must have done something real bad to be rejected like this

So it looks like when it comes to elections, the story BJP is hearing is the same again and again. There must be something new brewing in electorate's minds these days. Aren't these the same guys who got carried away with religious sentiments just a few years back? So what has changed since? We still go to Churches, Gurudwaras, Temple and Mosques. Aren't we Hindus or Muslims any more? So why are we rejecting BJP and beating it like this? There is a reason and the reason is BJP can't seem to see the new realities of India and if they can see it they can't seem to find a leadership who can promise and deliver new frontiers for the nation. Otherwise Mumbai where one of the worst security lapse occurred and caught the state and federal (Congress) governments totally off-guard still would not have rejected the nationalist BJP/Shiv Sena combined. Congress with its new astute and young leadership has seems stuck a chord with this new India. They are struggling to solve problems but still seems sincere to resolve them. Its the truth in their tone which now is making it a "party with a difference". BJP has failed and it has failed in numerous ways. LK Advani has become Kapil Dev of politics. Sitting at the leadership role for as long as possible, not delivering and not allowing new blood to take over either. Kapil was sensible as he knew how much to extend and when to throw in the towel. Advani seems struggling with it. RSS who knows how to disassociate themselves from BJP are only pretending to be a separate entity. They are not. If RSS had an answer BJP would not have been in this quagmire. The disciplined party workers of BJP were never so disciplined after all. In fighting has taken over their regional units and this list goes on and on. At the end by now BJP should see they have been outright rejected in a shape they stand today. When it comes to Congress the truth is we only consider to vote them out when we get disenchanted with its leaders not because we love the other option. And if they continue to do what they are doing I see no reason why even in future elections there will be any new challenges for them.

Coherence - Back to Basics

One question that keeps cropping up is when to configure an application as a cluster member and when over *Extend? The answer has always been pretty simple - If application stability cannot be guaranteed then it cannot be considered as a cluster member. There is more to it, btw. Following questions you must ask before opting for any configuration strategy:

• If an application is written in either C++ or .NET then there is no other choice but to configure it as an *Extend client.
• If the interface to Coherence is Java (including JNI) then choices are a little deeper.

Lets talk about it in more details (The content is not a dictation to its readers but an assistance to put them on a right track).

When to configure as a cluster member?

1. Applications deployed inside a container (Like an Application server) are typically configured as storage disabled cluster members. The reason is application servers themselves being "server-side" components are considered fairly stable.
   
2. Applications demanding "extreme" performance are configured as cluster members.
   
3. *Extend introduces an additional network hop and if this additional couple of milliseconds are not acceptable then configure it as a cluster member.
4. If proxy-reconnect is an issue.

When to *Extend?
I am a big proponent of *Extend configuration because it provides us a framework that we are already so used to. With new enhancements in Coherence v3.5 and later in Proxy services and serialization mechanism, overhead on proxy has reduced considerably and this opens up a some nice architectural options.

1. If the application is a desktop application where reliability cannot be guaranteed.
2. If network over which the application is accessing Coherence resources is slow or unreliable.
3. If application is written in .NET or C++

Besides these there are some advanced considerations:

1. If application is already very TCP/IP centric. New services can be developed on top of Proxy as is demonstrated here, here and here.
   
2. If an application arbiter is needed or something like a Gatekeeper.
   
3. With *Extend proxy layer can be scaled independent of other storage nodes. *Extend configuration allows three layers of scalability - First application clients, second the proxy layer itself and the third the core storage members or non-proxy cluster members. Storage member layer is about data scalability and proxy layer is about request scalability.
4. In line with the popular Adapter pattern typically found in BPEL designs. Application specific adapters can be plugged into Coherence using Proxy's TCP/IP service making application clients even more agnostic of Coherence infrastructure beyond the Map implementation.
5. If a new component is added that introduces new features that are not available out of the box from Coherence, it is much safer to deploy it on proxy nodes. Always remember we do not have any access to either the underline protocol (TCMP) or the core coherence component layer. Any feature that simulate either of the two must be performed by proxy.
6. Anyone still remembers SOA? Even though SOA does not dictate the transport layer but it is much cleaner to have well defined services running and accessing coherence resources over *Extend. If services are deployed inside a container then revisit (1), (2) and (3).
   

Enjoy!

A Biker's spectrum