Anonymous | Login
Project:
RSS
  
News | My View | View Issues | Roadmap | Summary

View Revisions: Issue #34534 All Revisions ] Back to Issue ]
Summary 0034534: StatementFinalizer causes some overhead on each statement
Revision 2016-11-15 15:45 by alostale
Steps To Reproduce Easier to benchmark preaparing some testing code in an Openbravo running with xmx limited to 600MB.

1. Memory overhead

Execute this code:
    Connection con = OBDal.getInstance().getConnection();
    try {
      long i = 0L;
      while (true) {
        if (++i % 10000 == 0) {
          System.out.println(i);
        }
        PreparedStatement st = con.prepareStatement("select count(*) from ad_system");
        st.close();
      }
    } catch (SQLException e) {
      e.printStackTrace();
    }


...
2720000
2730000
1f923ef8 77346 [http-8080-3] ERROR org.openbravo.base.exception.OBException - Exception thrown Java heap 
space
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:3181)
    at java.util.ArrayList.grow(ArrayList.java:261)
    at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:235)
    at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:227)
    at java.util.ArrayList.add(ArrayList.java:458)
    at org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer.createStatement(StatementFinalizer.java:43)
    
at org.apache.tomcat.jdbc.pool.interceptor.AbstractCreateStatementInterceptor.invoke(AbstractCreateStatementInterceptor.java:69)



After creating ~2.7M statements OOM occurs while trying to grow the size of the ArrayList that tracks the statement in the transaction.


2. Time overhead

Profile Openbravo and execute this code:
    Connection con = OBDal.getInstance().getConnection();
    try {
      long i = 0L;
      while (i < 20000000) {
        if (++i % 10000 == 0) {
          System.out.println(i);

          // connection is returned to the pool, in this moment statements are tried
          // to be closed again and the list is cleaned up
          OBDal.getInstance().commitAndClose();
          con = OBDal.getInstance().getConnection();
        }
        PreparedStatement st = con.prepareStatement("select count(*) from ad_system");
        st.close();
      }
    } catch (SQLException e) {
      e.printStackTrace();
    }

OBDal.getInstance().getConnection() consumes 26s (22% of the total time). From those 26s, 24s (90%) is spent in StatementFinalizer closing again the already closed statement.


== Removing this finalizer ==

If finalizer is removed we get:

1. The first code snippet can be run indefinitely, as the used heap remains stable.
2. Time for the second one is reduced from 121s to 89s. Mainly in the overhead for the OBDal.commitAndClose, reduced from 26.9s to 2.6s; and in the prepareStatemt (overhead to maintain the list), reduced from 80s to 70s. (see attached graph).
Revision 2016-11-15 15:39 by alostale
Steps To Reproduce Easier to benchmark preaparing some testing code in an Openbravo running with xmx limited to 600MB.

1. Memory overhead

Execute this code:
    Connection con = OBDal.getInstance().getConnection();
    try {
      long i = 0L;
      while (true) {
        if (++i % 10000 == 0) {
          System.out.println(i);
        }
        PreparedStatement st = con.prepareStatement("select count(*) from ad_system");
        st.close();
      }
    } catch (SQLException e) {
      e.printStackTrace();
    }


...
2720000
2730000
1f923ef8 77346 [http-8080-3] ERROR org.openbravo.base.exception.OBException - Exception thrown Java heap 
space
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:3181)
    at java.util.ArrayList.grow(ArrayList.java:261)
    at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:235)
    at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:227)
    at java.util.ArrayList.add(ArrayList.java:458)
    at org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer.createStatement(StatementFinalizer.java:43)
    
at org.apache.tomcat.jdbc.pool.interceptor.AbstractCreateStatementInterceptor.invoke(AbstractCreateStatementInterceptor.java:69)



After creating ~2.7M statements OOM occurs while trying to grow the size of the ArrayList that tracks the statement in the transaction.


2. Time overhead

Profile Openbravo and execute this code:
    Connection con = OBDal.getInstance().getConnection();
    try {
      long i = 0L;
      while (i < 20000000) {
        if (++i % 10000 == 0) {
          System.out.println(i);
          OBDal.getInstance().commitAndClose();
          con = OBDal.getInstance().getConnection();
        }
        PreparedStatement st = con.prepareStatement("select count(*) from ad_system");
        st.close();
      }
    } catch (SQLException e) {
      e.printStackTrace();
    }

OBDal.getInstance().getConnection() consumes 26s (22% of the total time). From those 26s, 24s (90%) is spent in StatementFinalizer closing again the already closed statement.


== Removing this finalizer ==

If finalizer is removed we get:

1. The first code snippet can be run indefinitely, as the used heap remains stable.
2. Time for the second one is reduced from 121s to 89s. Mainly in the overhead for the OBDal.commitAndClose, reduced from 26.9s to 2.6s; and in the prepareStatemt (overhead to maintain the list), reduced from 80s to 70s. (see attached graph).
Revision 2016-11-15 15:31 by alostale
Steps To Reproduce Easier to benchmark preaparing some testing code in an Openbravo running with xmx limited to 600MB.

1. Memory overhead

Execute this code:
    Connection con = OBDal.getInstance().getConnection();
    try {
      long i = 0L;
      while (true) {
        if (++i % 10000 == 0) {
          System.out.println(i);
        }
        PreparedStatement st = con.prepareStatement("select count(*) from ad_system");
        st.close();
      }
    } catch (SQLException e) {
      e.printStackTrace();
    }


...
2720000
2730000
1f923ef8 77346 [http-8080-3] ERROR org.openbravo.base.exception.OBException - Exception thrown Java heap 
space
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:3181)
    at java.util.ArrayList.grow(ArrayList.java:261)
    at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:235)
    at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:227)
    at java.util.ArrayList.add(ArrayList.java:458)
    at org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer.createStatement(StatementFinalizer.java:43)
    
at org.apache.tomcat.jdbc.pool.interceptor.AbstractCreateStatementInterceptor.invoke(AbstractCreateStatementInterceptor.java:69)



After creating ~2.7M statements OOM occurs while trying to grow the size of the ArrayList that tracks the statement in the transaction.


2. Time overhead

Profile Openbravo and execute this code:
    Connection con = OBDal.getInstance().getConnection();
    try {
      long i = 0L;
      while (i < 20000000) {
        if (++i % 10000 == 0) {
          System.out.println(i);
          OBDal.getInstance().commitAndClose();
          con = OBDal.getInstance().getConnection();
        }
        PreparedStatement st = con.prepareStatement("select count(*) from ad_system");
        st.close();
      }
    } catch (SQLException e) {
      e.printStackTrace();
    }


Removing this finalizer we get:

1. The first code snippet can be run indefinitely, as the used heap remains stable.
Revision 2016-11-15 15:24 by alostale
Steps To Reproduce Easier to benchmark...

1. Memory overhead

Run Openbravo with xmx limited to 600MB.

Execute this code:
    Connection con = OBDal.getInstance().getConnection();
    try {
      long i = 0L;
      while (true) {
        if (++i % 10000 == 0) {
          System.out.println(i);
        }
        PreparedStatement st = con.prepareStatement("select count(*) from ad_system");
        st.close();
      }
    } catch (SQLException e) {
      e.printStackTrace();
    }
Revision 2016-11-15 15:24 by alostale
Steps To Reproduce Easier to benchmark...

1. Memory overhead

Run Openbravo with xmx limited to 600MB.

Execute this code:
    Connection con = OBDal.getInstance().getConnection();
    try {
      long i = 0L;
      while (true) {
        if (++i % 10000 == 0) {
          System.out.println(i);
        }
        PreparedStatement st = con.prepareStatement("select count(*) from ad_system");
        st.close();
      }
    } catch (SQLException e) {
      e.printStackTrace();
    }


Copyright © 2000 - 2009 MantisBT Group
Powered by Mantis Bugtracker