[midPoint] memory leak issue

Mon Nov 5 20:46:44 CET 2018

Hi, I strongly recommend building your own 3.7-support mp on very latest
branch. Dont forget new DB script on upgrade path (with indexes). Just
today, there was another OOM fix released. We are still working with
Evolveum on this and continually improving memory performance. I dont know
about your particular SQL error, try purging audit log by smaller amounts
by playing with auditRecords cleanuppolicy in systemconfig.

Also make sure your search-iterative tasks use aggregateOutput=false param
to save memory.
<scext:executeScript xmlns:scext="
http://midpoint.evolveum.com/xml/ns/public/model/scripting/extension-3">
<s:search>
....
....
*<s:aggregateOutput>false</s:aggregateOutput>*
</s:search>
</scext:executeScript>

*Martin Lízner*
chief solution architect

gsm: [+420] 737 745 571
e‑mail: martin.lizner at ami.cz

*AMI Praha a.s.*
Pláničkova 11, 162 00 Praha 6

tel.: [+420] 274 783 239 | web: www.ami.cz <http://dtp.ami.cz/www.ami.cz>

[image: AMI Praha a.s.]

Textem tohoto e‑mailu podepisující neslibuje uzavřít ani neuzavírá
za společnost AMI Praha a.s.
jakoukoliv smlouvu. Každá smlouva, pokud bude uzavřena, musí mít výhradně
písemnou formu.

Tento e‑mail je určen výhradně pro potřeby jeho adresáta/ů a může obsahovat
důvěrné nebo osobní
informace. Nejste‑li zamýšleným příjemcem, je zakázáno jakékoliv
zveřejňování, zprostředkování
nebo jiné použití těchto informací. Pokud jste obdrželi e‑mail neoprávněně,
informujte o tom prosím
odesílatele a vymažte neprodleně všechny kopie tohoto e‑mailu včetně
všech jeho příloh. Nakládáním
s neoprávněně získanými informacemi se vystavujete riziku právního postihu.

čt 1. 11. 2018 v 22:24 odesílatel Nicolas Rossi <nrossi at identicum.com>
napsal:

> Hi Martin, One of our customers is running midPoint 3.7.1 and they started
> having several memory issues. It has 8GB RAM and is configured to use
> almost all of this (-Xms6144M -Xmx6144). Do you know if there is a known
> issue with the memory on this version ?
>
> I also found this error each time I run the Cleanup task:
>
> 2018-11-01 17:18:25,119 [] [midPointScheduler_Worker-3] ERROR
> (com.evolveum.midpoint.model.impl.cleanup.CleanUpTaskHandler): Audit
> cleanup: error executing work
>
> com.evolveum.midpoint.util.exception.SystemException: error executing work
>
> at
> com.evolveum.midpoint.repo.sql.helpers.BaseHelper.handleGeneralRuntimeException(BaseHelper.java:178)
>
> at
> com.evolveum.midpoint.repo.sql.SqlAuditServiceImpl.batchDeletionAttempt(SqlAuditServiceImpl.java:544)
>
> at
> com.evolveum.midpoint.repo.sql.SqlAuditServiceImpl.cleanupAuditMaxAge(SqlAuditServiceImpl.java:420)
>
> at
> com.evolveum.midpoint.repo.sql.SqlAuditServiceImpl.cleanupAudit(SqlAuditServiceImpl.java:376)
>
> at
> com.evolveum.midpoint.init.AuditServiceProxy.cleanupAudit(AuditServiceProxy.java:113)
>
> at
> com.evolveum.midpoint.model.impl.cleanup.CleanUpTaskHandler.run(CleanUpTaskHandler.java:116)
>
> at
> com.evolveum.midpoint.task.quartzimpl.execution.JobExecutor.executeHandler(JobExecutor.java:639)
>
> at
> com.evolveum.midpoint.task.quartzimpl.execution.JobExecutor.executeRecurrentTask(JobExecutor.java:522)
>
> at
> com.evolveum.midpoint.task.quartzimpl.execution.JobExecutor.execute(JobExecutor.java:180)
>
> at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
>
> at
> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:588)
>
> Caused by: org.hibernate.exception.GenericJDBCException: error executing
> work
>
> at
> org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:54)
>
> at
> org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:126)
>
> at
> org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:112)
>
> at
> org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.coordinateWork(JdbcCoordinatorImpl.java:318)
>
> at org.hibernate.internal.SessionImpl.doWork(SessionImpl.java:2095)
>
> at org.hibernate.internal.SessionImpl.doWork(SessionImpl.java:2080)
>
> at
> com.evolveum.midpoint.repo.sql.SqlAuditServiceImpl.createTemporaryTable(SqlAuditServiceImpl.java:624)
>
> at
> com.evolveum.midpoint.repo.sql.SqlAuditServiceImpl.batchDeletionAttempt(SqlAuditServiceImpl.java:507)
>
> ... 9 common frames omitted
>
> Caused by: java.sql.SQLException: (conn:1256) Statement violates GTID
> consistency: CREATE TEMPORARY TABLE and DROP TEMPORARY TABLE can only be
> executed outside transactional context.  These statements are also not
> allowed in a function or trigger because functions and triggers are also
> considered to be multi-statement transactions.
>
> Query is : create temporary table if not exists HT_m_audit_event (id
> bigint not null)
>
> at
> org.mariadb.jdbc.internal.util.ExceptionMapper.get(ExceptionMapper.java:145)
>
> at
> org.mariadb.jdbc.internal.util.ExceptionMapper.getException(ExceptionMapper.java:101)
>
> at
> org.mariadb.jdbc.internal.util.ExceptionMapper.throwAndLogException(ExceptionMapper.java:77)
>
> at
> org.mariadb.jdbc.MariaDbStatement.executeQueryEpilog(MariaDbStatement.java:224)
>
> at
> org.mariadb.jdbc.MariaDbStatement.executeInternal(MariaDbStatement.java:258)
>
> at org.mariadb.jdbc.MariaDbStatement.execute(MariaDbStatement.java:271)
>
> at
> com.mchange.v2.c3p0.impl.NewProxyStatement.execute(NewProxyStatement.java:909)
>
> at
> com.evolveum.midpoint.repo.sql.SqlAuditServiceImpl.lambda$createTemporaryTable$2(SqlAuditServiceImpl.java:643)
>
> at org.hibernate.jdbc.WorkExecutor.executeWork(WorkExecutor.java:54)
>
> at org.hibernate.internal.SessionImpl$2.accept(SessionImpl.java:2076)
>
> at org.hibernate.internal.SessionImpl$2.accept(SessionImpl.java:2073)
>
> at
> org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.coordinateWork(JdbcCoordinatorImpl.java:313)
>
> ... 13 common frames omitted
>
> Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Statement
> violates GTID consistency: CREATE TEMPORARY TABLE and DROP TEMPORARY TABLE
> can only be executed outside transactional context.  These statements are
> also not allowed in a function or trigger because functions and triggers
> are also considered to be multi-statement transactions.
>
> Query is : create temporary table if not exists HT_m_audit_event (id
> bigint not null)
>
> at
> org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:1114)
>
> at
> org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:137)
>
> at
> org.mariadb.jdbc.MariaDbStatement.executeInternal(MariaDbStatement.java:249)
>
> ... 20 common frames omitted
>
> 2018-11-01 17:18:25,359 [] [http-nio-8080-exec-8] ERROR
> (com.evolveum.midpoint.web.util.MidPointProfilingServletFilter):
> Encountered exception: javax.servlet.ServletException: Filter execution
> threw an exception
>
> javax.servlet.ServletException: Filter execution threw an exception
>
>
> Ing Nicolás Rossi
> Identicum S.A.
> Jorge Newbery 3226
> Oficina: +54 (11) 4552-3050
> Móvil: +54 (911) 6041-3920
> www.identicum.com
>
>
> On Tue, Jan 2, 2018 at 11:50 AM Martin Lízner - AMI Praha a.s. <
> martin.lizner at ami.cz> wrote:
>
>> Hi, Im having OOM problems on 3.7. Which version are you on? There is
>> Jira for it already and I think it has high priority:
>> https://jira.evolveum.com/browse/MID-4349
>>
>> M.
>>
>> Martin Lízner
>> solution architect
>>
>> gsm: [+420] 737 745 571
>> e-mail: martin.lizner at ami.cz
>>
>>
>> AMI Praha a.s.
>> Pláničkova 11
>> 162 00 Praha 6
>> tel.: [+420] 274 783 239
>> web: www.ami.cz
>>
>>
>>
>> [image: AMI Praha a.s.] <http://www.skyidentity.com/>
>>
>> Textem tohoto e-mailu podepisující neslibuje uzavřít ani neuzavírá za
>> společnost AMI Praha a.s.
>> jakoukoliv smlouvu. Každá smlouva, pokud bude uzavřena, musí mít výhradně
>> písemnou formu.
>>
>>
>> 2017-12-28 21:26 GMT+01:00 Juan Manuel Catá <juan.cata at despegar.com>:
>>
>>> Greetings
>>>
>>> Im writing in reference to a memory leak issue with my instance of
>>> MidPoint.
>>>
>>> When I try to run MidPoint on my VM (4 cpus, 8gb RAM, 40gb HD) I got
>>> alerts in relation to "critical memory usage". In relation to this, I found
>>> that memory is not managed correctly, and is never released until the
>>> kernel kill the MidPoint process; another problem that i found is that I'm
>>> running out of availables Inodes. ¿Could this be related to some
>>> missconfiguration on my MidPoint instance? Have you noticed these kind of
>>> issues before?
>>>
>>> Any kind of help will be appreciated
>>>
>>> Regards
>>>
>>> --
>>> *Juan Manuel Catá*
>>> Application Security
>>> Juana Manso 999 - piso 2° - C.A.B.A.
>>>
>>>
>>> _______________________________________________
>>> midPoint mailing list
>>> midPoint at lists.evolveum.com
>>> http://lists.evolveum.com/mailman/listinfo/midpoint
>>>
>>>
>> _______________________________________________
>> midPoint mailing list
>> midPoint at lists.evolveum.com
>> http://lists.evolveum.com/mailman/listinfo/midpoint
>>
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20181105/a145fe49/attachment.htm>