Nagle or not Nagle?

What is the Nagle algorithm in reference to TCP/IP?

Nagle's algorithm is a way of avoiding network congestion by limiting to at most one the number of "tinygrams" (that is, packets that are less than full size) an application can have outstanding at once. It does this by concatenating small segments until it has a full segment or it receives an acknowledgment of the outstanding small segment. The algorithm was proposed by John Nagle in RFC896. Its latest incarnation, which differs slightly from Nagle's original formulation, is described in RFC1122.

The Nagle algorithm can sometimes interact with the TCP's "delayed ACK" mechanism, another congestion avoidance strategy, in a way that increases network congestion and delay. For this reason, it is sometimes disabled, but this is appropriate only in very special circumstances. See the book Effective TCP/IP Programming or Rich Stevens' TCP/IP Illustrated, for complete details.

** Definition taken from searchnetworkingchannel.techtarget.com

This has been a problem with Windows OS (and probably with other OS as well) that TCP connection waits to send next packets unless an acknowledgment is received. One of the Clients reported a drastically reduced performance in Coherence's multi-cluster data replication strategy. When nothing was found even after reviewing the architecture, code reviewing a number of times and involvement of upper management, we had to bring in a Top Coherence expert and he tracked the problem in 15 minutes. It was Nagle.

At last found sometime to blog

Last two weeks have been non-busingly busy. First there were some anxiety to attend the first Coherence SIG in NY. Last time I met Cameron before this SIG was at a client's site and I had screwed it up then (Not Me really but Hertz never lost) in driving directions. It was also nice to meet Alex, Rob, Brian, Peter O., Parker, Craig, Phil, Peter U., Tom and Patrick. And then it was nice seeing some of my clients as well and discussing their current challenges along with introducing Boris, Vikas and Anusha to the Coherence team members that they interact by emails and phone but had never met in person. It was also nice to listen to Brian Oliver's talk on Asynchronous Order preserving WAN Replication strategy. Nice to see someone bringing components that I had been aware of and used before, together in a more logical and robust way. And also hear Steve outline his strategy in successfully managing 'N' number of teams and establishing a centralized Architecture team.
Then I found there are many takers of my weekend project (jdbc driver for Coherence) that Rob and Brian were kind enough to allow to be submitted to Coherence's Incubator project. In fact two of my other colleagues joined hands with me as well and one of who's has a much mature implementation than mine. While I still think my broad approach is better ;). But it is hard to find ample time to do something that requires off-the-route creativity, and with work burden piling up again I am not sure when I would and should donate anytime towards it. On top of this one of the client reported problems with an earlier implementation and even though understanding their frustrations it is almost impossible to devote anytime to debug it after any active engagement ends. Probably the problem is due to Nagle (TCP connection waiting to send next packets) but need more testing. Oracle has a support structure that is very good in tackling Customer issues and works better than contacting Consultants and Product Engineers directly. I have tried my best and engaged everyone I could have to support them and is good to work with some intelligent developers who are quick to tests things out. Big Guys are already on top of it so I need not worry about it. And then there are some very exciting and technologically challenging projects coming up. I am working simultaneously with at least 5 separate teams to do architecture reviews and coding. When too many things go on in parallel I easily become a stateless machine - do not remember anything. Even simple implementations take time as Context Switching becomes a challenge.
And with watching your investments at a mercy of people who you do not know and have little trust on it has not been very easy. And with so many things to do - Working with Customers, Pending Reading materials, Lagging ideas, traveling every week on a 4 hour+ flights twice and aching body it was nice to steal few minutes to blog.

Election participation - Go Vote

With American elections just around the corner, it reminds me of elections in India. Average percentage of voting in India is around 40-45%. Out of the rest 55 a huge majority does not go vote because they are lazy, because they do not think their vote matters, because they think their candidate will win anyway and other miscellaneous reasons like health, weather and fear of anti-social elements just to find that a candidate has won who he/she did not want. So people living in a Democratic society it is our responsibility to go out and vote. Whoever you like but go out there and vote. My parents have always encouraged us to participate in elections. Not in rhetoric but in ink. I remember whenever I was home during elections in a place I was registered to vote I and my family always went and voted. This is a way to show off your freedom to the rest of Un-democratic world that do not enjoy what we do. And remember if you did not vote then you lose your moral right to whine about the laws, bills and candidates that hurt you. Its your country and its future is your responsibility. Shape it with your vote.

Is surge working or Terrorists already won?

A re-run of Bill Maher show today just shook me. After seven years of 9-11 Osama Bin Laden and his lieutenant is still alive. Work on World Trade Center is still not complete. American economy is seeing unprecedented downturn. The leader of the terrorists who attacked us might be laughing today. And what do we listen in the debate? "My friends I know where he is. I know how to take him out".. and no one asks that it has been a Republican government for last eight years, If someone does know how to take him out why was it not done? More than a Trillion dollars burnt on personal egos and revenge that could have been spent on Health care, Energy and other things. We see 4000+ lives lost with the main culprit still at large, that might have only taken a few hundred million well spent dollars to take him out and destroy his network. We see a Police officer in his uniform bashing his fellow American and planting doubt of him being a Muslim and words like "Kill Him" in Sarah Palin's political rallies and no one denouncing it.. "Who is Barack? - A terrorist!" and Republicans think this is the only way to win? May be Barack does not understand may be he does not, but at least he is clear what he intends to do.. "My friends I know how to do it" is still not making it clear how will it be done. The respect I had for McCain is badly hurt by what Sarah Palin and his other apparatus is doing. I guess they both promised to run a respectable campaign. But when the time is to get together and united we are demeaning personally our own fellow citizen. What has gone wrong with us? Did Terrorists hit us on our character and values?

JDBC driver for Coherence

Okay this might be a crazy idea but as Coherence is a data source it was worth writing a JDBC driver for it. Of course spending time to write a complete SQL engine would be too much without any real motivation. But a simple proof of concept was on the cards. So here you go:

First thing is the client:

public static void main (String [] args) {
   // -- Create a URL: hostname and port being of the proxy
   String url = "jdbc:coh:@localhost:9099";
   String sql = "SELECT * FROM Positions";
        
   try {
      // -- Load the Coherence Database Driver
      Class.forName("com.ezduck.driver.GridDriver");
      Connection conn = DriverManager.getConnection(url);
      Statement stmt = conn.createStatement();
      stmt.executeQuery(sql);
   } catch (ClassNotFoundException e) {
      e.printStackTrace();
   } catch (SQLException e) {
      e.printStackTrace();
   }
}

Now the Driver:

public class GridDriver implements Driver {

    static {
        try {
            DriverManager.registerDriver(new GridDriver ());
        } catch (SQLException e) {
            // TODO
        }
    }
    // -- "jdbc:coh:@localhost:9099"
    public GridDriver() {
    }

    public Connection connect(String url, Properties info) {
        int hostIndex = url.indexOf("@") + 1;
        int portIndex = url.lastIndexOf(":");
        if (hostIndex < 0 || portIndex < 0) {
            throw new IllegalStateException ("Driver URL incorrect");
        }
        String hostName = url.substring(hostIndex, portIndex);
        String port = url.substring(portIndex + 1);
        return new GridConnection (hostName, port);
    }
    ... 
}

Hmm Now the Connection:

public class GridConnection implements Connection {
    
    public GridConnection(String hostName, String port) {
        ConfigurableCacheFactory factory = CacheFactory.getConfigurableCacheFactory();
        XmlElement cc = new SimpleElement("cache-config");
        
        XmlElement sMap = cc.addElement("caching-scheme-mapping");
        XmlElement cMap  = sMap.addElement ("cache-mapping");
        cMap.addElement("cache-name").setString("*");
        cMap.addElement("scheme-name").setString("extend-dist");
         
        XmlElement rcs = 
            cc.addElement("caching-schemes").addElement("remote-cache-scheme");
        rcs.addElement("scheme-name").setString ("extend-dist");
        rcs.addElement("service-name").setString("ExtendTcpCacheService");
        XmlElement xE = 
            rcs.addElement("initiator-config").addElement("tcp-initiator")
                .addElement("remote-addresses");
        XmlElement rAs = xE.addElement("socket-address");
        XmlElement add = rAs.addElement("address");
        add.setString(hostName);
        XmlElement portId = rAs.addElement("port");
        portId.setString(port);

        // -- Now set the new XmlElement   
        factory.setConfig(cc);
        // -- And of course you need to restart the Cache Service thread
        factory.ensureService("ExtendTcpCacheService").stop();
        factory.ensureService("ExtendTcpCacheService").start();                        
    }

    public Statement createStatement() {
        return new GridStatement();
    }
    
    .......
}

Then the Statement:

public class GridStatement implements Statement {
    
    public GridStatement() {
    }

    public ResultSet executeQuery(String sql) {
        String command = sql.substring(0, sql.indexOf (" "));
        try {
            // -- There must be a way to recognize the SQL Command
            ICommand cObject = (ICommand) Beans.instantiate(
                this.getClass().getClassLoader(), 
                this.getClass().getPackage().getName() + "." + 
                                command.toUpperCase().trim());
            Set set = (Set) cObject.execute (cObject, sql);
            // -- A utility to parse the SQL 
            Map map = SQLUtil.parseSelect(sql);
            String cacheName = (String) map.get ("tables");
            // -- And last the ResultSet
            ResultSet rS = new GridResultSet (set, cacheName);
            
        } catch (ClassNotFoundException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
        return null;
    }
    .......
}

And before the ResultSet, lets see a half cooked SQL SELECT:

public class SELECT implements ICommand {
    String token[];

    // -- Select * From Dist-A where name = 'A';

    public SELECT() {
    }

    public Object execute(Object obj, String command) {
        Map map = SQLUtil.parseSelect(command);
        NamedCache nCache = CacheFactory.getCache((String)map.get("tables"));
        String clauses = (String)map.get("clauses");
        // -- Lets demonstrate the Select * before going crazy
        if (clauses == null) {
            return nCache.entrySet(AlwaysFilter.INSTANCE);
        }
        return null;
    }
}

Last but not least - The ResultSet:

public class GridResultSet implements ResultSet {

    private Iterator iter;
    private Set set;
    private String cacheName;
    private InvocableMap.Entry entry;

    public GridResultSet(Set set, String cacheName) {
        this.set = set;
        this.cacheName = cacheName;
        iter = set.iterator();
    }

    public boolean next() {
        entry = (InvocableMap.Entry) iter.next();
        return iter.hasNext();
    }
    ......
}

So here you go - another weekend project for Coherence!

Invitation to New York Coherence SIG

This is an invitation to all of you! who wants to meet Industry's finest and Customers who know what to do and how best to do when market is in slumps and each penny matters. So sign yourself up - Coherence NYCSIG and I will see you there!

PS: When I said finest - I meant the ones I am going to meet ;)