[midPoint] Running out of Java Heap Space

Pálos Gustáv gustav.palos at evolveum.com
Fri Aug 26 10:56:37 CEST 2016


you can also decrease schema size in resource over this:
<schema>
<generationConstraints>
<generateObjectClass>ri:inetOrgPerson</generateObjectClass>
<generateObjectClass>ri:groupOfNames</generateObjectClass>
</generationConstraints>
</schema>

when ri:inetOrgPerson and ri:groupOfNames are your object classes what you
are using in schemaHandling section.

Gustav

2016-08-26 10:40 GMT+02:00 Radovan Semancik <radovan.semancik at evolveum.com>:

> Yes, it is. But the memory consumption of the LDAP connector should be
> proportional to schema size and number of connector instances. It should
> NOT be proportional to the number of LDAP entries.
>
> The memory consumption should normally increase at the beginning of the
> import. But after first few entries are imported the memory consumption
> should stay more or less constant. If the heap size continually increases
> during import it is most likely midpoint bug, connector bug or Apache
> Directory API bug.
>
> --
> Radovan Semancik
> Software Architectevolveum.com
>
>
>
> On 08/25/2016 04:28 PM, Ivan Noris wrote:
>
> I have a feeling that LDAP connector might be a little resource consuming
> for some resources with big schemas etc.
> Ivan
>
> ------------------------------
>
> *From: *"Jason Everling" <jeverling at bshp.edu> <jeverling at bshp.edu>
> *To: *"midPoint General Discussion" <midpoint at lists.evolveum.com>
> <midpoint at lists.evolveum.com>
> *Sent: *Thursday, August 25, 2016 5:15:33 PM
> *Subject: *Re: [midPoint] Running out of Java Heap Space
>
> I had the same issue when I imported around 20k accounts from one of our
> databases in the beginning, although, we have more than one java
> application running on this server, 64bit jdk,
>
> -Xms512m -Xmx4096m -XX:PermSize=512m -XX:MaxPermSize=1024m
>
> JASON
>
> On Thu, Aug 25, 2016 at 1:36 AM, Ivan Noris <Ivan.Noris at evolveum.com>
> wrote:
>
>> Hi Matt,
>> please try to increase -Xmx to at least 2048m and see if it helps.
>>
>> Ivan
>>
>> ------------------------------
>>
>> *From: *"Matt Mencel" <mr-mencel at wiu.edu>
>> *To: *"midPoint General Discussion" <midpoint at lists.evolveum.com>
>> *Sent: *Thursday, August 25, 2016 12:01:33 AM
>> *Subject: *[midPoint] Running out of Java Heap Space
>>
>>
>> Is there any further recommendations for setting the Java memory
>> requirements?  My JAVA_OPTS is set to this.
>>
>> JAVA_OPTS="$JAVA_OPTS $JSSE_OPTS -Dmidpoint.home=/opt/midpoint
>> -Djavax.net.ssl.trustStore=/opt/midpoint/keystore.jceks -Djavax.net.ssl.trustStoreType=jceks
>> -XX:MaxPermSize=1024m -Xss1m -Xmx1024m"
>>
>> I'm running into the error below after an LDAP import task suspends due
>> to an error.  I have yet to resolve the error, see my earlier post about
>> how to ignore an LDAP attribute on import.  I've got plenty of memory on
>> the VM, but I may need to tune my Java even further.
>>
>> Thanks,
>> Matt
>>
>>
>> 2016-08-24 13:29:56,835 [] [midPointScheduler_Worker-4] INFO
>> (com.evolveum.midpoint.task.quartzimpl.execution.JobExecutor): Task
>> encountered temporary error, suspending it. Task =
>> Task(id:1472058712002-0-1, name:LDAP Import, oid:bfeccb74-5aa2-40f2-a107-
>> 18e120b765fc)
>> 2016-08-24 13:29:56,836 [] [midPointScheduler_Worker-4] INFO
>> (com.evolveum.midpoint.task.quartzimpl.TaskManagerQuartzImpl):
>> Suspending tasks [Task(id:1472058712002-0-1, name:LDAP Import,
>> oid:bfeccb74-5aa2-40f2-a107-18e120b765fc)]; do not stop tasks.
>> 2016-08-24 13:51:21,162 [] [NioProcessor-1] WARN
>> (org.apache.directory.ldap.client.api.LdapNetworkConnection): Java heap
>> space
>> java.lang.OutOfMemoryError: Java heap space
>> 2016-08-24 13:52:46,208 [] [NioProcessor-1] WARN
>> (org.apache.directory.ldap.client.api.LdapNetworkConnection): Java heap
>> space
>> java.lang.OutOfMemoryError: Java heap space
>> 2016-08-24 14:01:27,961 [] [ClusterManagerThread] ERROR
>> (com.evolveum.midpoint.task.quartzimpl.cluster.ClusterManager):
>> Unexpected exception in ClusterManager thread; continuing execution.,
>> reason: Java heap space (class java.lang.OutOfMemoryError)
>> 2016-08-24 14:02:59,710 [] [midPointScheduler_Worker-5] ERROR
>> (org.quartz.core.JobRunShell): Job DEFAULT.00000000-0000-0000-0000-000000000006
>> threw an unhandled Exception:
>> java.lang.OutOfMemoryError: Java heap space
>> 2016-08-24 14:03:01,507 [] [NioProcessor-2] WARN
>> (org.apache.directory.ldap.client.api.LdapNetworkConnection): Java heap
>> space
>> java.lang.OutOfMemoryError: Java heap space
>> 2016-08-24 14:03:03,490 [] [NioProcessor-2] WARN
>> (org.apache.directory.ldap.client.api.LdapNetworkConnection): Java heap
>> space
>> java.lang.OutOfMemoryError: Java heap space
>> 2016-08-24 14:03:03,490 [] [midPointScheduler_Worker-5] ERROR
>> (org.quartz.core.ErrorLogger): Job (DEFAULT.00000000-0000-0000-0000-000000000006
>> threw an exception.
>> org.quartz.SchedulerException: Job threw an unhandled exception.
>>         at org.quartz.core.JobRunShell.run(JobRunShell.java:224)
>> ~[quartz-2.1.3.jar:na]
>>         at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
>> [quartz-2.1.3.jar:na]
>> Caused by: java.lang.OutOfMemoryError: Java heap space
>> 2016-08-24 14:29:17,649 [] [Thread-9] ERROR (org.activiti.engine.impl.
>> jobexecutor.AcquireJobsRunnableImpl): exception during job acquisition:
>> Java heap space
>> java.lang.OutOfMemoryError: Java heap space
>>         at sun.reflect.GeneratedConstructorAccessor5.newInstance(Unknown
>> Source) ~[na:na]
>>         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
>> DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_101]
>>         at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>> ~[na:1.8.0_101]
>>         at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
>> ~[mysql-connector-java-5.1.39-bin.jar:5.1.39]
>>         at com.mysql.jdbc.ResultSetImpl.getInstance(ResultSetImpl.java:319)
>> ~[mysql-connector-java-5.1.39-bin.jar:5.1.39]
>>         at com.mysql.jdbc.MysqlIO.buildResultSetWithUpdates(MysqlIO.java:3187)
>> ~[mysql-connector-java-5.1.39-bin.jar:5.1.39]
>>         at com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:3087)
>> ~[mysql-connector-java-5.1.39-bin.jar:5.1.39]
>>         at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:2336)
>> ~[mysql-connector-java-5.1.39-bin.jar:5.1.39]
>>         at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2729)
>> ~[mysql-connector-java-5.1.39-bin.jar:5.1.39]
>>         at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2545)
>> ~[mysql-connector-java-5.1.39-bin.jar:5.1.39]
>>         at com.mysql.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:4842)
>> ~[mysql-connector-java-5.1.39-bin.jar:5.1.39]
>>         at sun.reflect.GeneratedMethodAccessor254.invoke(Unknown Source)
>> ~[na:na]
>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(
>> DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_101]
>>         at java.lang.reflect.Method.invoke(Method.java:498)
>> ~[na:1.8.0_101]
>>         at org.apache.ibatis.datasource.pooled.PooledConnection.
>> invoke(PooledConnection.java:245) ~[mybatis-3.3.0.jar:3.3.0]
>>         at com.sun.proxy.$Proxy106.setAutoCommit(Unknown Source) ~[na:na]
>>         at org.apache.ibatis.transaction.jdbc.JdbcTransaction.
>> setDesiredAutoCommit(JdbcTransaction.java:104) ~[mybatis-3.3.0.jar:3.3.0]
>>         at org.apache.ibatis.transaction.jdbc.JdbcTransaction.
>> openConnection(JdbcTransaction.java:144) ~[mybatis-3.3.0.jar:3.3.0]
>>         at org.apache.ibatis.transaction.jdbc.JdbcTransaction.
>> getConnection(JdbcTransaction.java:62) ~[mybatis-3.3.0.jar:3.3.0]
>>         at org.apache.ibatis.executor.BaseExecutor.getConnection(BaseExecutor.java:315)
>> ~[mybatis-3.3.0.jar:3.3.0]
>>         at org.apache.ibatis.executor.SimpleExecutor.prepareStatement(SimpleExecutor.java:75)
>> ~[mybatis-3.3.0.jar:3.3.0]
>>         at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:61)
>> ~[mybatis-3.3.0.jar:3.3.0]
>>         at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:303)
>> ~[mybatis-3.3.0.jar:3.3.0]
>>         at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:154)
>> ~[mybatis-3.3.0.jar:3.3.0]
>>         at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:102)
>> ~[mybatis-3.3.0.jar:3.3.0]
>>         at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:82)
>> ~[mybatis-3.3.0.jar:3.3.0]
>>         at org.apache.ibatis.session.defaults.DefaultSqlSession.
>> selectList(DefaultSqlSession.java:120) ~[mybatis-3.3.0.jar:3.3.0]
>>         at org.apache.ibatis.session.defaults.DefaultSqlSession.
>> selectList(DefaultSqlSession.java:113) ~[mybatis-3.3.0.jar:3.3.0]
>>         at org.activiti.engine.impl.db.DbSqlSession.
>> selectListWithRawParameter(DbSqlSession.java:438)
>> ~[activiti-engine-5.19.0.2.jar:5.19.0.2]
>>         at org.activiti.engine.impl.db.DbSqlSession.selectList(DbSqlSession.java:429)
>> ~[activiti-engine-5.19.0.2.jar:5.19.0.2]
>>         at org.activiti.engine.impl.db.DbSqlSession.selectList(DbSqlSession.java:424)
>> ~[activiti-engine-5.19.0.2.jar:5.19.0.2]
>>         at org.activiti.engine.impl.db.DbSqlSession.selectList(DbSqlSession.java:411)
>> ~[activiti-engine-5.19.0.2.jar:5.19.0.2]
>>
>>
>> _______________________________________________
>> midPoint mailing list
>> midPoint at lists.evolveum.com
>> http://lists.evolveum.com/mailman/listinfo/midpoint
>>
>>
>>
>>
>> --
>> Ivan Noris
>> Senior Identity Engineer
>> evolveum.com
>>
>> _______________________________________________
>> midPoint mailing list
>> midPoint at lists.evolveum.com
>> http://lists.evolveum.com/mailman/listinfo/midpoint
>>
>>
>
>
>
> CONFIDENTIALITY NOTICE:
> This e-mail together with any attachments is proprietary and confidential;
> intended for only the recipient(s) named above and may contain information
> that is privileged. You should not retain, copy or use this e-mail or any
> attachments for any purpose, or disclose all or any part of the contents to
> any person. Any views or opinions expressed in this e-mail are those of the
> author and do not represent those of the Baptist School of Health
> Professions. If you have received this e-mail in error, or are not the
> named recipient(s), you are hereby notified that any review, dissemination,
> distribution or copying of this communication is prohibited by the sender
> and to do so might constitute a violation of the Electronic Communications
> Privacy Act, 18 U.S.C. section 2510-2521. Please immediately notify the
> sender and delete this e-mail and any attachments from your computer.
>
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
>
>
>
> --
> Ivan Noris
> Senior Identity Engineer
> evolveum.com
>
>
> _______________________________________________
> midPoint mailing listmidPoint at lists.evolveum.comhttp://lists.evolveum.com/mailman/listinfo/midpoint
>
>
>
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
>


-- 
Gustáv Pálos
Identity Engineer
evolveum.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20160826/5d73b55e/attachment.htm>


More information about the midPoint mailing list