[midPoint] Running out of Java Heap Space
Radovan Semancik
radovan.semancik at evolveum.com
Fri Aug 26 10:40:09 CEST 2016
Yes, it is. But the memory consumption of the LDAP connector should be
proportional to schema size and number of connector instances. It should
NOT be proportional to the number of LDAP entries.
The memory consumption should normally increase at the beginning of the
import. But after first few entries are imported the memory consumption
should stay more or less constant. If the heap size continually
increases during import it is most likely midpoint bug, connector bug or
Apache Directory API bug.
--
Radovan Semancik
Software Architect
evolveum.com
On 08/25/2016 04:28 PM, Ivan Noris wrote:
> I have a feeling that LDAP connector might be a little resource
> consuming for some resources with big schemas etc.
> Ivan
>
> ------------------------------------------------------------------------
>
> *From: *"Jason Everling" <jeverling at bshp.edu>
> *To: *"midPoint General Discussion" <midpoint at lists.evolveum.com>
> *Sent: *Thursday, August 25, 2016 5:15:33 PM
> *Subject: *Re: [midPoint] Running out of Java Heap Space
>
> I had the same issue when I imported around 20k accounts from one
> of our databases in the beginning, although, we have more than one
> java application running on this server, 64bit jdk,
>
> -Xms512m -Xmx4096m -XX:PermSize=512m -XX:MaxPermSize=1024m
>
> JASON
>
> On Thu, Aug 25, 2016 at 1:36 AM, Ivan Noris
> <Ivan.Noris at evolveum.com <mailto:Ivan.Noris at evolveum.com>> wrote:
>
> Hi Matt,
> please try to increase -Xmx to at least 2048m and see if it helps.
>
> Ivan
>
> ------------------------------------------------------------------------
>
> *From: *"Matt Mencel" <mr-mencel at wiu.edu
> <mailto:mr-mencel at wiu.edu>>
> *To: *"midPoint General Discussion"
> <midpoint at lists.evolveum.com
> <mailto:midpoint at lists.evolveum.com>>
> *Sent: *Thursday, August 25, 2016 12:01:33 AM
> *Subject: *[midPoint] Running out of Java Heap Space
>
>
> Is there any further recommendations for setting the Java
> memory requirements? My JAVA_OPTS is set to this.
>
> JAVA_OPTS="$JAVA_OPTS $JSSE_OPTS
> -Dmidpoint.home=/opt/midpoint
> -Djavax.net.ssl.trustStore=/opt/midpoint/keystore.jceks
> -Djavax.net.ssl.trustStoreType=jceks -XX:MaxPermSize=1024m
> -Xss1m -Xmx1024m"
>
> I'm running into the error below after an LDAP import task
> suspends due to an error. I have yet to resolve the
> error, see my earlier post about how to ignore an LDAP
> attribute on import. I've got plenty of memory on the VM,
> but I may need to tune my Java even further.
>
> Thanks,
> Matt
>
>
> 2016-08-24 13:29:56,835 [] [midPointScheduler_Worker-4]
> INFO
> (com.evolveum.midpoint.task.quartzimpl.execution.JobExecutor):
> Task encountered temporary error, suspending it. Task =
> Task(id:1472058712002-0-1, name:LDAP Import,
> oid:bfeccb74-5aa2-40f2-a107-18e120b765fc)
> 2016-08-24 13:29:56,836 [] [midPointScheduler_Worker-4]
> INFO
> (com.evolveum.midpoint.task.quartzimpl.TaskManagerQuartzImpl):
> Suspending tasks [Task(id:1472058712002-0-1, name:LDAP
> Import, oid:bfeccb74-5aa2-40f2-a107-18e120b765fc)]; do not
> stop tasks.
> 2016-08-24 13:51:21,162 [] [NioProcessor-1] WARN
> (org.apache.directory.ldap.client.api.LdapNetworkConnection):
> Java heap space
> java.lang.OutOfMemoryError: Java heap space
> 2016-08-24 13:52:46,208 [] [NioProcessor-1] WARN
> (org.apache.directory.ldap.client.api.LdapNetworkConnection):
> Java heap space
> java.lang.OutOfMemoryError: Java heap space
> 2016-08-24 14:01:27,961 [] [ClusterManagerThread] ERROR
> (com.evolveum.midpoint.task.quartzimpl.cluster.ClusterManager):
> Unexpected exception in ClusterManager thread; continuing
> execution., reason: Java heap space (class
> java.lang.OutOfMemoryError)
> 2016-08-24 14:02:59,710 [] [midPointScheduler_Worker-5]
> ERROR (org.quartz.core.JobRunShell): Job
> DEFAULT.00000000-0000-0000-0000-000000000006 threw an
> unhandled Exception:
> java.lang.OutOfMemoryError: Java heap space
> 2016-08-24 14:03:01,507 [] [NioProcessor-2] WARN
> (org.apache.directory.ldap.client.api.LdapNetworkConnection):
> Java heap space
> java.lang.OutOfMemoryError: Java heap space
> 2016-08-24 14:03:03,490 [] [NioProcessor-2] WARN
> (org.apache.directory.ldap.client.api.LdapNetworkConnection):
> Java heap space
> java.lang.OutOfMemoryError: Java heap space
> 2016-08-24 14:03:03,490 [] [midPointScheduler_Worker-5]
> ERROR (org.quartz.core.ErrorLogger): Job
> (DEFAULT.00000000-0000-0000-0000-000000000006 threw an
> exception.
> org.quartz.SchedulerException: Job threw an unhandled
> exception.
> at
> org.quartz.core.JobRunShell.run(JobRunShell.java:224)
> ~[quartz-2.1.3.jar:na]
> at
> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
> [quartz-2.1.3.jar:na]
> Caused by: java.lang.OutOfMemoryError: Java heap space
> 2016-08-24 14:29:17,649 [] [Thread-9] ERROR
> (org.activiti.engine.impl.jobexecutor.AcquireJobsRunnableImpl):
> exception during job acquisition: Java heap space
> java.lang.OutOfMemoryError: Java heap space
> at
> sun.reflect.GeneratedConstructorAccessor5.newInstance(Unknown
> Source) ~[na:na]
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> ~[na:1.8.0_101]
> at
> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> ~[na:1.8.0_101]
> at
> com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
> ~[mysql-connector-java-5.1.39-bin.jar:5.1.39]
> at
> com.mysql.jdbc.ResultSetImpl.getInstance(ResultSetImpl.java:319)
> ~[mysql-connector-java-5.1.39-bin.jar:5.1.39]
> at
> com.mysql.jdbc.MysqlIO.buildResultSetWithUpdates(MysqlIO.java:3187)
> ~[mysql-connector-java-5.1.39-bin.jar:5.1.39]
> at
> com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:3087)
> ~[mysql-connector-java-5.1.39-bin.jar:5.1.39]
> at
> com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:2336)
> ~[mysql-connector-java-5.1.39-bin.jar:5.1.39]
> at
> com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2729)
> ~[mysql-connector-java-5.1.39-bin.jar:5.1.39]
> at
> com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2545)
> ~[mysql-connector-java-5.1.39-bin.jar:5.1.39]
> at
> com.mysql.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:4842)
> ~[mysql-connector-java-5.1.39-bin.jar:5.1.39]
> at
> sun.reflect.GeneratedMethodAccessor254.invoke(Unknown
> Source) ~[na:na]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.8.0_101]
> at
> java.lang.reflect.Method.invoke(Method.java:498)
> ~[na:1.8.0_101]
> at
> org.apache.ibatis.datasource.pooled.PooledConnection.invoke(PooledConnection.java:245)
> ~[mybatis-3.3.0.jar:3.3.0]
> at com.sun.proxy.$Proxy106.setAutoCommit(Unknown
> Source) ~[na:na]
> at
> org.apache.ibatis.transaction.jdbc.JdbcTransaction.setDesiredAutoCommit(JdbcTransaction.java:104)
> ~[mybatis-3.3.0.jar:3.3.0]
> at
> org.apache.ibatis.transaction.jdbc.JdbcTransaction.openConnection(JdbcTransaction.java:144)
> ~[mybatis-3.3.0.jar:3.3.0]
> at
> org.apache.ibatis.transaction.jdbc.JdbcTransaction.getConnection(JdbcTransaction.java:62)
> ~[mybatis-3.3.0.jar:3.3.0]
> at
> org.apache.ibatis.executor.BaseExecutor.getConnection(BaseExecutor.java:315)
> ~[mybatis-3.3.0.jar:3.3.0]
> at
> org.apache.ibatis.executor.SimpleExecutor.prepareStatement(SimpleExecutor.java:75)
> ~[mybatis-3.3.0.jar:3.3.0]
> at
> org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:61)
> ~[mybatis-3.3.0.jar:3.3.0]
> at
> org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:303)
> ~[mybatis-3.3.0.jar:3.3.0]
> at
> org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:154)
> ~[mybatis-3.3.0.jar:3.3.0]
> at
> org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:102)
> ~[mybatis-3.3.0.jar:3.3.0]
> at
> org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:82)
> ~[mybatis-3.3.0.jar:3.3.0]
> at
> org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:120)
> ~[mybatis-3.3.0.jar:3.3.0]
> at
> org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:113)
> ~[mybatis-3.3.0.jar:3.3.0]
> at
> org.activiti.engine.impl.db.DbSqlSession.selectListWithRawParameter(DbSqlSession.java:438)
> ~[activiti-engine-5.19.0.2.jar:5.19.0.2]
> at
> org.activiti.engine.impl.db.DbSqlSession.selectList(DbSqlSession.java:429)
> ~[activiti-engine-5.19.0.2.jar:5.19.0.2]
> at
> org.activiti.engine.impl.db.DbSqlSession.selectList(DbSqlSession.java:424)
> ~[activiti-engine-5.19.0.2.jar:5.19.0.2]
> at
> org.activiti.engine.impl.db.DbSqlSession.selectList(DbSqlSession.java:411)
> ~[activiti-engine-5.19.0.2.jar:5.19.0.2]
>
>
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com
> <mailto:midPoint at lists.evolveum.com>
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
>
>
>
> --
> Ivan Noris
> Senior Identity Engineer
> evolveum.com <http://evolveum.com>
>
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com <mailto:midPoint at lists.evolveum.com>
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
>
>
>
>
> CONFIDENTIALITY NOTICE:
> This e-mail together with any attachments is proprietary and
> confidential; intended for only the recipient(s) named above and
> may contain information that is privileged. You should not retain,
> copy or use this e-mail or any attachments for any purpose, or
> disclose all or any part of the contents to any person. Any views
> or opinions expressed in this e-mail are those of the author and
> do not represent those of the Baptist School of Health
> Professions. If you have received this e-mail in error, or are not
> the named recipient(s), you are hereby notified that any review,
> dissemination, distribution or copying of this communication is
> prohibited by the sender and to do so might constitute a violation
> of the Electronic Communications Privacy Act, 18 U.S.C. section
> 2510-2521. Please immediately notify the sender and delete this
> e-mail and any attachments from your computer.
>
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
>
>
>
> --
> Ivan Noris
> Senior Identity Engineer
> evolveum.com
>
>
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com
> http://lists.evolveum.com/mailman/listinfo/midpoint
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20160826/ccf50b30/attachment.htm>
More information about the midPoint
mailing list