Project:
| View Revisions: Issue #34534 | [ Back to Issue ] | ||
| Summary | 0034534: StatementFinalizer causes some overhead on each statement | ||
| Revision | 2016-11-15 15:45 by alostale | ||
| Steps To Reproduce | Easier to benchmark preaparing some testing code in an Openbravo running with xmx limited to 600MB. 1. Memory overhead Execute this code:
Connection con = OBDal.getInstance().getConnection();
try {
long i = 0L;
while (true) {
if (++i % 10000 == 0) {
System.out.println(i);
}
PreparedStatement st = con.prepareStatement("select count(*) from ad_system");
st.close();
}
} catch (SQLException e) {
e.printStackTrace();
}
...
2720000
2730000
1f923ef8 77346 [http-8080-3] ERROR org.openbravo.base.exception.OBException - Exception thrown Java heap
space
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3181)
at java.util.ArrayList.grow(ArrayList.java:261)
at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:235)
at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:227)
at java.util.ArrayList.add(ArrayList.java:458)
at org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer.createStatement(StatementFinalizer.java:43)
at org.apache.tomcat.jdbc.pool.interceptor.AbstractCreateStatementInterceptor.invoke(AbstractCreateStatementInterceptor.java:69)
After creating ~2.7M statements OOM occurs while trying to grow the size of the ArrayList that tracks the statement in the transaction. 2. Time overhead Profile Openbravo and execute this code:
Connection con = OBDal.getInstance().getConnection();
try {
long i = 0L;
while (i < 20000000) {
if (++i % 10000 == 0) {
System.out.println(i);
// connection is returned to the pool, in this moment statements are tried
// to be closed again and the list is cleaned up
OBDal.getInstance().commitAndClose();
con = OBDal.getInstance().getConnection();
}
PreparedStatement st = con.prepareStatement("select count(*) from ad_system");
st.close();
}
} catch (SQLException e) {
e.printStackTrace();
}
OBDal.getInstance().getConnection() consumes 26s (22% of the total time). From those 26s, 24s (90%) is spent in StatementFinalizer closing again the already closed statement. == Removing this finalizer == If finalizer is removed we get: 1. The first code snippet can be run indefinitely, as the used heap remains stable. 2. Time for the second one is reduced from 121s to 89s. Mainly in the overhead for the OBDal.commitAndClose, reduced from 26.9s to 2.6s; and in the prepareStatemt (overhead to maintain the list), reduced from 80s to 70s. (see attached graph). |
||
| Revision | 2016-11-15 15:39 by alostale | ||
| Steps To Reproduce | Easier to benchmark preaparing some testing code in an Openbravo running with xmx limited to 600MB. 1. Memory overhead Execute this code:
Connection con = OBDal.getInstance().getConnection();
try {
long i = 0L;
while (true) {
if (++i % 10000 == 0) {
System.out.println(i);
}
PreparedStatement st = con.prepareStatement("select count(*) from ad_system");
st.close();
}
} catch (SQLException e) {
e.printStackTrace();
}
...
2720000
2730000
1f923ef8 77346 [http-8080-3] ERROR org.openbravo.base.exception.OBException - Exception thrown Java heap
space
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3181)
at java.util.ArrayList.grow(ArrayList.java:261)
at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:235)
at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:227)
at java.util.ArrayList.add(ArrayList.java:458)
at org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer.createStatement(StatementFinalizer.java:43)
at org.apache.tomcat.jdbc.pool.interceptor.AbstractCreateStatementInterceptor.invoke(AbstractCreateStatementInterceptor.java:69)
After creating ~2.7M statements OOM occurs while trying to grow the size of the ArrayList that tracks the statement in the transaction. 2. Time overhead Profile Openbravo and execute this code:
Connection con = OBDal.getInstance().getConnection();
try {
long i = 0L;
while (i < 20000000) {
if (++i % 10000 == 0) {
System.out.println(i);
OBDal.getInstance().commitAndClose();
con = OBDal.getInstance().getConnection();
}
PreparedStatement st = con.prepareStatement("select count(*) from ad_system");
st.close();
}
} catch (SQLException e) {
e.printStackTrace();
}
OBDal.getInstance().getConnection() consumes 26s (22% of the total time). From those 26s, 24s (90%) is spent in StatementFinalizer closing again the already closed statement. == Removing this finalizer == If finalizer is removed we get: 1. The first code snippet can be run indefinitely, as the used heap remains stable. 2. Time for the second one is reduced from 121s to 89s. Mainly in the overhead for the OBDal.commitAndClose, reduced from 26.9s to 2.6s; and in the prepareStatemt (overhead to maintain the list), reduced from 80s to 70s. (see attached graph). |
||
| Revision | 2016-11-15 15:31 by alostale | ||
| Steps To Reproduce | Easier to benchmark preaparing some testing code in an Openbravo running with xmx limited to 600MB. 1. Memory overhead Execute this code:
Connection con = OBDal.getInstance().getConnection();
try {
long i = 0L;
while (true) {
if (++i % 10000 == 0) {
System.out.println(i);
}
PreparedStatement st = con.prepareStatement("select count(*) from ad_system");
st.close();
}
} catch (SQLException e) {
e.printStackTrace();
}
...
2720000
2730000
1f923ef8 77346 [http-8080-3] ERROR org.openbravo.base.exception.OBException - Exception thrown Java heap
space
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3181)
at java.util.ArrayList.grow(ArrayList.java:261)
at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:235)
at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:227)
at java.util.ArrayList.add(ArrayList.java:458)
at org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer.createStatement(StatementFinalizer.java:43)
at org.apache.tomcat.jdbc.pool.interceptor.AbstractCreateStatementInterceptor.invoke(AbstractCreateStatementInterceptor.java:69)
After creating ~2.7M statements OOM occurs while trying to grow the size of the ArrayList that tracks the statement in the transaction. 2. Time overhead Profile Openbravo and execute this code:
Connection con = OBDal.getInstance().getConnection();
try {
long i = 0L;
while (i < 20000000) {
if (++i % 10000 == 0) {
System.out.println(i);
OBDal.getInstance().commitAndClose();
con = OBDal.getInstance().getConnection();
}
PreparedStatement st = con.prepareStatement("select count(*) from ad_system");
st.close();
}
} catch (SQLException e) {
e.printStackTrace();
}
Removing this finalizer we get: 1. The first code snippet can be run indefinitely, as the used heap remains stable. |
||
| Revision | 2016-11-15 15:24 by alostale | ||
| Steps To Reproduce | Easier to benchmark... 1. Memory overhead Run Openbravo with xmx limited to 600MB. Execute this code:
Connection con = OBDal.getInstance().getConnection();
try {
long i = 0L;
while (true) {
if (++i % 10000 == 0) {
System.out.println(i);
}
PreparedStatement st = con.prepareStatement("select count(*) from ad_system");
st.close();
}
} catch (SQLException e) {
e.printStackTrace();
}
|
||
| Revision | 2016-11-15 15:24 by alostale | ||
| Steps To Reproduce | Easier to benchmark... 1. Memory overhead Run Openbravo with xmx limited to 600MB. Execute this code: Connection con = OBDal.getInstance().getConnection(); try { long i = 0L; while (true) { if (++i % 10000 == 0) { System.out.println(i); } PreparedStatement st = con.prepareStatement("select count(*) from ad_system"); st.close(); } } catch (SQLException e) { e.printStackTrace(); } |
||
| Copyright © 2000 - 2009 MantisBT Group |
 |