[midPoint] High CPU Utilization After Import Attempt - UI Unresponsive
Jason Everling
jeverling at bshp.edu
Fri Aug 26 19:20:12 CEST 2016
Yeah, so if you go to 'Configuration' -> 'Shadow Details' and choose your
resource, you will see all accounts.
How many vCPUs does your machine have? You can also try and see if giving
it more worker threads would help, for our resources that have every single
person I utilize it and it does help speed up the recon tasks,
https://wiki.evolveum.com/display/midPoint/Parallelized+background+tasks
Sample from one of my task xmls,
<name>Reconcile: Some Database</name>
<extension xmlns:mext="
http://midpoint.evolveum.com/xml/ns/public/model/extension-3">
<mext:kind>account</mext:kind>
<mext:workerThreads>4</mext:workerThreads>
</extension>
JASON
On Fri, Aug 26, 2016 at 11:33 AM, Mencel, Matt <mr-mencel at wiu.edu> wrote:
> Yep, see below...
>
> <synchronization>
> <objectSynchronization>
> <kind>account</kind>
> <intent>person</intent>
> <focusType>c:UserType</focusType>
> <enabled>true</enabled>
> <!-- Only Sync uTech from LDAP -->
> <condition>
> <script>
> <code>
> tmp = basic.getAttributeValue(shadow, '
> http://midpoint.evolveum.com/xml/ns/public/resource/instance-3',
> 'department'); return (tmp == 'University Technology' || tmp == 'AIMS')
> </code>
> </script>
> </condition>
> <correlation>
> <q:equal>
> <q:path>c:name</q:path>
> <expression>
> <path>$account/attributes/icfs:name</path>
> </expression>
> </q:equal>
> </correlation>
> <reaction>
> <situation>linked</situation>
> <action>
> <handlerUri>http://midpoint.
> evolveum.com/xml/ns/public/model/action-3#modifyUser</handlerUri>
> </action>
> </reaction>
> <reaction>
> <situation>deleted</situation>
> <action>
> <handlerUri>http://midpoint.
> evolveum.com/xml/ns/public/model/action-3#unlinkAccount</handlerUri>
> </action>
> </reaction>
> <reaction>
> <situation>unlinked</situation>
> <action>
> <handlerUri>http://midpoint.
> evolveum.com/xml/ns/public/model/action-3#linkAccount</handlerUri>
> </action>
> </reaction>
> <reaction>
> <situation>unmatched</situation>
> <action>
> <handlerUri>http://midpoint.
> evolveum.com/xml/ns/public/model/action-3#addUser</handlerUri>
> </action>
> </reaction>
> </objectSynchronization>
> </synchronization>
>
> On Fri, Aug 26, 2016 at 11:30 AM, Jason Everling <jeverling at bshp.edu>
> wrote:
>
>> So, I noticed this, did you use a condition in the objectSynchronization
>> part? If so, it will still create 'shadows' for every item. I have a
>> condition to only sync certain groups but if I go to the debug pages and
>> view the shadows for the resource I can see all my AD groups. I am assuming
>> this should be correct so that it would not create a new account on the
>> resource that already exist but did not know about.
>>
>>
>>
>> JASON
>>
>> On Fri, Aug 26, 2016 at 10:39 AM, Pavol Mederly <mederly at evolveum.com>
>> wrote:
>>
>>> Hello Matt,
>>>
>>> I have no direct experience with deploying midPoint with large number of
>>> users, but let me give you a few hints:
>>>
>>> 1) If GUI is dead, maybe REST would allow you to check the task's state;
>>> at http://localhost:8080/midpoint/ws/rest/tasks/<oid>, e.g.
>>> http://localhost:8080/midpoint/ws/rest/tasks/00000000-0000-0
>>> 000-0000-000000000005
>>>
>>> 2) If REST is unresponsive as well, or if there's need to know more
>>> about what's going on, I sometimes use jvisualvm - to see e.g. heap
>>> utilization, thread state, even collecting thread dump, etc. Maybe you
>>> could use it to see if there's a memleak somewhere (as Radovan said).
>>>
>>> Its output looks like this:
>>>
>>>
>>> Best regards,
>>>
>>> Pavol Mederly
>>> Software developerevolveum.com
>>>
>>> On 26.08.2016 17:25, Mencel, Matt wrote:
>>>
>>> New entries in the idm.log appeared about 40 minutes after it stopped
>>> logging. My -Xmx setting in JAVA_OPTS is set to 4096.
>>>
>>> Matt
>>>
>>> 2016-08-26 10:06:51,393 [] [C3P0PooledConnectionPoolManag
>>> er[identityToken->2tu6q89i1j961p54hnz75|180fb7dc]-AdminTaskTimer] WARN
>>> (com.mchange.v2.async.ThreadPoolAsynchronousRunner):
>>> com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDe
>>> tector at 2f41b4ad -- APPARENT DEADLOCK!!! Creating emergency threads for
>>> unassigned pending tasks!
>>> 2016-08-26 10:12:30,979 [] [NioProcessor-1] WARN
>>> (org.apache.directory.ldap.client.api.LdapNetworkConnection): Java heap
>>> space
>>> java.lang.OutOfMemoryError: Java heap space
>>> at org.apache.directory.api.util.Strings.utf8ToString(Strings.java:724)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.util.Strings.toLowerCase(Strings.java:1890)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.model.entry.DefaultAttribute.s
>>> etUpId(DefaultAttribute.java:637) ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.model.entry.DefaultAttribute.s
>>> etUpId(DefaultAttribute.java:575) ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.model.entry.DefaultAttribute.<
>>> init>(DefaultAttribute.java:192) ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.codec.decorators.SearchResultE
>>> ntryDecorator.addAttribute(SearchResultEntryDecorator.java:112)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.codec.actions.searchResultEntr
>>> y.AddAttributeType.action(AddAttributeType.java:87)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.codec.actions.searchResultEntr
>>> y.AddAttributeType.action(AddAttributeType.java:48)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.asn1.ber.grammar.AbstractGrammar.ex
>>> ecuteAction(AbstractGrammar.java:136) ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.asn1.ber.Asn1Decoder.treatTLVDoneState(Asn1Decoder.java:600)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.asn1.ber.Asn1Decoder.decode(Asn1Decoder.java:736)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.codec.protocol.mina.LdapProtoc
>>> olDecoder.decode(LdapProtocolDecoder.java:145)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.codec.protocol.mina.LdapProtoc
>>> olDecoder.decode(LdapProtocolDecoder.java:94)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.mina.filter.codec.ProtocolCodecFilter.messageRece
>>> ived(ProtocolCodecFilter.java:230) ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNe
>>> xtMessageReceived(DefaultIoFilterChain.java:542)
>>> [mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.DefaultIoFilterChain.access
>>> $1300(DefaultIoFilterChain.java:48) [mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryI
>>> mpl$1.messageReceived(DefaultIoFilterChain.java:943)
>>> [mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:109)
>>> [mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNe
>>> xtMessageReceived(DefaultIoFilterChain.java:542)
>>> [mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMe
>>> ssageReceived(DefaultIoFilterChain.java:535) [mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.read
>>> (AbstractPollingIoProcessor.java:697) [mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.proc
>>> ess(AbstractPollingIoProcessor.java:651) [mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.proc
>>> ess(AbstractPollingIoProcessor.java:640) [mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.acce
>>> ss$600(AbstractPollingIoProcessor.java:68) [mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.polling.AbstractPollingIoProcessor$Proc
>>> essor.run(AbstractPollingIoProcessor.java:1097)
>>> [mina-core-2.0.13.jar:na]
>>> at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
>>> [mina-core-2.0.13.jar:na]
>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> [na:1.8.0_101]
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> [na:1.8.0_101]
>>> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
>>> 2016-08-26 10:12:30,980 [] [NioProcessor-1] WARN
>>> (org.apache.directory.ldap.client.api.LdapNetworkConnection): Java heap
>>> space
>>> java.lang.OutOfMemoryError: Java heap space
>>> at org.apache.directory.api.util.Strings.utf8ToString(Strings.java:724)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.util.Strings.toLowerCase(Strings.java:1890)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.model.entry.DefaultAttribute.s
>>> etUpId(DefaultAttribute.java:637) ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.model.entry.DefaultAttribute.s
>>> etUpId(DefaultAttribute.java:575) ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.model.entry.DefaultAttribute.<
>>> init>(DefaultAttribute.java:192) ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.codec.decorators.SearchResultE
>>> ntryDecorator.addAttribute(SearchResultEntryDecorator.java:112)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.codec.actions.searchResultEntr
>>> y.AddAttributeType.action(AddAttributeType.java:87)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.codec.actions.searchResultEntr
>>> y.AddAttributeType.action(AddAttributeType.java:48)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.asn1.ber.grammar.AbstractGrammar.ex
>>> ecuteAction(AbstractGrammar.java:136) ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.asn1.ber.Asn1Decoder.treatTLVDoneState(Asn1Decoder.java:600)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.asn1.ber.Asn1Decoder.decode(Asn1Decoder.java:736)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.codec.protocol.mina.LdapProtoc
>>> olDecoder.decode(LdapProtocolDecoder.java:145)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.codec.protocol.mina.LdapProtoc
>>> olDecoder.decode(LdapProtocolDecoder.java:94)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.mina.filter.codec.ProtocolCodecFilter.messageRece
>>> ived(ProtocolCodecFilter.java:230) ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNe
>>> xtMessageReceived(DefaultIoFilterChain.java:542)
>>> [mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.DefaultIoFilterChain.access
>>> $1300(DefaultIoFilterChain.java:48) [mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryI
>>> mpl$1.messageReceived(DefaultIoFilterChain.java:943)
>>> ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:109)
>>> ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNe
>>> xtMessageReceived(DefaultIoFilterChain.java:542)
>>> [mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMe
>>> ssageReceived(DefaultIoFilterChain.java:535) [mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.read
>>> (AbstractPollingIoProcessor.java:697) [mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.proc
>>> ess(AbstractPollingIoProcessor.java:651) [mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.proc
>>> ess(AbstractPollingIoProcessor.java:640) [mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.acce
>>> ss$600(AbstractPollingIoProcessor.java:68) [mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.polling.AbstractPollingIoProcessor$Proc
>>> essor.run(AbstractPollingIoProcessor.java:1097)
>>> [mina-core-2.0.13.jar:na]
>>> at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
>>> [mina-core-2.0.13.jar:na]
>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> [na:1.8.0_101]
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> [na:1.8.0_101]
>>> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
>>> 2016-08-26 10:12:30,981 [] [NioProcessor-2] WARN
>>> (org.apache.directory.ldap.client.api.LdapNetworkConnection): Java heap
>>> space
>>> java.lang.OutOfMemoryError: Java heap space
>>> at java.nio.charset.CharsetEncoder.<init>(CharsetEncoder.java:233)
>>> ~[na:1.8.0_101]
>>> at sun.nio.cs.UTF_8$Encoder.<init>(UTF_8.java:558)
>>> ~[na:1.8.0_101]
>>> at sun.nio.cs.UTF_8$Encoder.<init>(UTF_8.java:554)
>>> ~[na:1.8.0_101]
>>> at sun.nio.cs.UTF_8.newEncoder(UTF_8.java:72) ~[na:1.8.0_101]
>>> at java.lang.StringCoding.encode(StringCoding.java:348)
>>> ~[na:1.8.0_101]
>>> at java.lang.String.getBytes(String.java:941) ~[na:1.8.0_101]
>>> at org.apache.directory.api.util.Strings.getBytesUtf8(Strings.java:1578)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.model.entry.StringValue.<init>(StringValue.java:93)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.model.entry.DefaultAttribute.c
>>> reateStringValue(DefaultAttribute.java:93)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.model.entry.DefaultAttribute.add(DefaultAttribute.java:1064)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.codec.decorators.SearchResultE
>>> ntryDecorator.addAttributeValue(SearchResultEntryDecorator.java:127)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.codec.actions.searchResultEntr
>>> y.StoreSearchResultAttributeValue.action(StoreSearchResultAttributeValue.java:100)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.codec.actions.searchResultEntr
>>> y.StoreSearchResultAttributeValue.action(StoreSearchResultAttributeValue.java:44)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.asn1.ber.grammar.AbstractGrammar.ex
>>> ecuteAction(AbstractGrammar.java:136) ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.asn1.ber.Asn1Decoder.treatTLVDoneState(Asn1Decoder.java:600)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.asn1.ber.Asn1Decoder.decode(Asn1Decoder.java:736)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.codec.protocol.mina.LdapProtoc
>>> olDecoder.decode(LdapProtocolDecoder.java:145)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.codec.protocol.mina.LdapProtoc
>>> olDecoder.decode(LdapProtocolDecoder.java:94)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.mina.filter.codec.ProtocolCodecFilter.messageRece
>>> ived(ProtocolCodecFilter.java:230) ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNe
>>> xtMessageReceived(DefaultIoFilterChain.java:542)
>>> ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.DefaultIoFilterChain.access
>>> $1300(DefaultIoFilterChain.java:48) ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryI
>>> mpl$1.messageReceived(DefaultIoFilterChain.java:943)
>>> ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:109)
>>> ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNe
>>> xtMessageReceived(DefaultIoFilterChain.java:542)
>>> ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMe
>>> ssageReceived(DefaultIoFilterChain.java:535) ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.read
>>> (AbstractPollingIoProcessor.java:697) ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.proc
>>> ess(AbstractPollingIoProcessor.java:651) ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.proc
>>> ess(AbstractPollingIoProcessor.java:640) ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.acce
>>> ss$600(AbstractPollingIoProcessor.java:68) ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.polling.AbstractPollingIoProcessor$Proc
>>> essor.run(AbstractPollingIoProcessor.java:1097)
>>> ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
>>> ~[mina-core-2.0.13.jar:na]
>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> ~[na:1.8.0_101]
>>> 2016-08-26 10:12:30,982 [] [NioProcessor-2] WARN
>>> (org.apache.directory.ldap.client.api.LdapNetworkConnection): Java heap
>>> space
>>> java.lang.OutOfMemoryError: Java heap space
>>> at java.nio.charset.CharsetEncoder.<init>(CharsetEncoder.java:233)
>>> ~[na:1.8.0_101]
>>> at sun.nio.cs.UTF_8$Encoder.<init>(UTF_8.java:558)
>>> ~[na:1.8.0_101]
>>> at sun.nio.cs.UTF_8$Encoder.<init>(UTF_8.java:554)
>>> ~[na:1.8.0_101]
>>> at sun.nio.cs.UTF_8.newEncoder(UTF_8.java:72) ~[na:1.8.0_101]
>>> at java.lang.StringCoding.encode(StringCoding.java:348)
>>> ~[na:1.8.0_101]
>>> at java.lang.String.getBytes(String.java:941) ~[na:1.8.0_101]
>>> at org.apache.directory.api.util.Strings.getBytesUtf8(Strings.java:1578)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.model.entry.StringValue.<init>(StringValue.java:93)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.model.entry.DefaultAttribute.c
>>> reateStringValue(DefaultAttribute.java:93)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.model.entry.DefaultAttribute.add(DefaultAttribute.java:1064)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.codec.decorators.SearchResultE
>>> ntryDecorator.addAttributeValue(SearchResultEntryDecorator.java:127)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.codec.actions.searchResultEntr
>>> y.StoreSearchResultAttributeValue.action(StoreSearchResultAttributeValue.java:100)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.codec.actions.searchResultEntr
>>> y.StoreSearchResultAttributeValue.action(StoreSearchResultAttributeValue.java:44)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.asn1.ber.grammar.AbstractGrammar.ex
>>> ecuteAction(AbstractGrammar.java:136) ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.asn1.ber.Asn1Decoder.treatTLVDoneState(Asn1Decoder.java:600)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.asn1.ber.Asn1Decoder.decode(Asn1Decoder.java:736)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.codec.protocol.mina.LdapProtoc
>>> olDecoder.decode(LdapProtocolDecoder.java:145)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.directory.api.ldap.codec.protocol.mina.LdapProtoc
>>> olDecoder.decode(LdapProtocolDecoder.java:94)
>>> ~[api-all-1.0.0-RC1.jar:1.0.0-RC1]
>>> at org.apache.mina.filter.codec.ProtocolCodecFilter.messageRece
>>> ived(ProtocolCodecFilter.java:230) ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNe
>>> xtMessageReceived(DefaultIoFilterChain.java:542)
>>> ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.DefaultIoFilterChain.access
>>> $1300(DefaultIoFilterChain.java:48) ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryI
>>> mpl$1.messageReceived(DefaultIoFilterChain.java:943)
>>> ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:109)
>>> ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNe
>>> xtMessageReceived(DefaultIoFilterChain.java:542)
>>> ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMe
>>> ssageReceived(DefaultIoFilterChain.java:535) ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.read
>>> (AbstractPollingIoProcessor.java:697) ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.proc
>>> ess(AbstractPollingIoProcessor.java:651) ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.proc
>>> ess(AbstractPollingIoProcessor.java:640) ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.polling.AbstractPollingIoProcessor.acce
>>> ss$600(AbstractPollingIoProcessor.java:68) ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.core.polling.AbstractPollingIoProcessor$Proc
>>> essor.run(AbstractPollingIoProcessor.java:1097)
>>> ~[mina-core-2.0.13.jar:na]
>>> at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
>>> ~[mina-core-2.0.13.jar:na]
>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> ~[na:1.8.0_101]
>>> 2016-08-26 10:13:22,931 [] [C3P0PooledConnectionPoolManag
>>> er[identityToken->2tu6q89i1j961p54hnz75|180fb7dc]-AdminTaskTimer] WARN
>>> (com.mchange.v2.async.ThreadPoolAsynchronousRunner):
>>> com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDe
>>> tector at 2f41b4ad -- APPARENT DEADLOCK!!! Complete Status:
>>> Managed Threads: 3
>>> Active Threads: 3
>>> Active Tasks:
>>> com.mchange.v2.resourcepool.Ba
>>> sicResourcePool$AsyncTestIdleResourceTask at 5b637151
>>> on thread: C3P0PooledConnectionPoolManage
>>> r[identityToken->2tu6q89i1j961p54hnz75|180fb7dc]-HelperThread-#2
>>> com.mchange.v2.resourcepool.Ba
>>> sicResourcePool$AsyncTestIdleResourceTask at 79a5524c
>>> on thread: C3P0PooledConnectionPoolManage
>>> r[identityToken->2tu6q89i1j961p54hnz75|180fb7dc]-HelperThread-#0
>>> com.mchange.v2.resourcepool.Ba
>>> sicResourcePool$AsyncTestIdleResourceTask at a6cc06e
>>> on thread: C3P0PooledConnectionPoolManage
>>> r[identityToken->2tu6q89i1j961p54hnz75|180fb7dc]-HelperThread-#1
>>> Pending Tasks:
>>> com.mchange.v2.resourcepool.Ba
>>> sicResourcePool$AsyncTestIdleResourceTask at 716cf0fa
>>> com.mchange.v2.resourcepool.Ba
>>> sicResourcePool$AsyncTestIdleResourceTask at 3145fe7f
>>> com.mchange.v2.resourcepool.Ba
>>> sicResourcePool$AsyncTestIdleResourceTask at 62a1fd29
>>> com.mchange.v2.resourcepool.Ba
>>> sicResourcePool$AsyncTestIdleResourceTask at e223ea3
>>> com.mchange.v2.resourcepool.Ba
>>> sicResourcePool$AsyncTestIdleResourceTask at 185ff4bd
>>> com.mchange.v2.resourcepool.Ba
>>> sicResourcePool$AsyncTestIdleResourceTask at 6f757cf
>>> com.mchange.v2.resourcepool.Ba
>>> sicResourcePool$AsyncTestIdleResourceTask at 6b52ba8d
>>> com.mchange.v2.resourcepool.Ba
>>> sicResourcePool$AsyncTestIdleResourceTask at 749a4662
>>> com.mchange.v2.resourcepool.Ba
>>> sicResourcePool$AsyncTestIdleResourceTask at 1307ae50
>>> com.mchange.v2.resourcepool.Ba
>>> sicResourcePool$AsyncTestIdleResourceTask at 2f72cad0
>>> com.mchange.v2.resourcepool.Ba
>>> sicResourcePool$AsyncTestIdleResourceTask at 510a9b61
>>> com.mchange.v2.resourcepool.Ba
>>> sicResourcePool$AsyncTestIdleResourceTask at 77057e74
>>> com.mchange.v2.resourcepool.Ba
>>> sicResourcePool$AsyncTestIdleResourceTask at 595e2bea
>>> com.mchange.v2.resourcepool.Ba
>>> sicResourcePool$AsyncTestIdleResourceTask at 60c34e5b
>>> com.mchange.v2.resourcepool.Ba
>>> sicResourcePool$AsyncTestIdleResourceTask at 659710b1
>>> Pool thread stack traces:
>>> Thread[C3P0PooledConnectionPoolManager[identityToken->2tu6q8
>>> 9i1j961p54hnz75|180fb7dc]-HelperThread-#2,5,main]
>>> com.mchange.v2.async.ThreadPoo
>>> lAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRun
>>> ner.java:674)
>>> Thread[C3P0PooledConnectionPoolManager[identityToken->2tu6q8
>>> 9i1j961p54hnz75|180fb7dc]-HelperThread-#1,5,main]
>>> com.mchange.v2.async.ThreadPoo
>>> lAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRun
>>> ner.java:674)
>>> Thread[C3P0PooledConnectionPoolManager[identityToken->2tu6q8
>>> 9i1j961p54hnz75|180fb7dc]-HelperThread-#0,5,main]
>>> com.mchange.v2.async.ThreadPoo
>>> lAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRun
>>> ner.java:672)
>>>
>>>
>>> 2016-08-26 10:13:51,355 [] [Thread-9] ERROR
>>> (org.activiti.engine.impl.jobexecutor.AcquireJobsRunnableImpl):
>>> exception during job acquisition: Java heap space
>>> java.lang.OutOfMemoryError: Java heap space
>>> at java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1019)
>>> ~[na:1.8.0_101]
>>> at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
>>> ~[na:1.8.0_101]
>>> at sun.util.resources.ParallelListResourceBundle.loadLookupTabl
>>> esIfNecessary(ParallelListResourceBundle.java:169) ~[na:1.8.0_101]
>>> at sun.util.resources.ParallelListResourceBundle.handleKeySet(P
>>> arallelListResourceBundle.java:134) ~[na:1.8.0_101]
>>> at sun.util.resources.ParallelListResourceBundle.keySet(ParallelListResourceBundle.java:143)
>>> ~[na:1.8.0_101]
>>> at sun.util.resources.ParallelListResourceBundle.containsKey(Pa
>>> rallelListResourceBundle.java:129) ~[na:1.8.0_101]
>>> at sun.util.resources.ParallelListResourceBundle$KeySet.contain
>>> s(ParallelListResourceBundle.java:208) ~[na:1.8.0_101]
>>> at sun.util.resources.ParallelListResourceBundle.containsKey(Pa
>>> rallelListResourceBundle.java:129) ~[na:1.8.0_101]
>>> at sun.util.resources.ParallelListResourceBundle$KeySet.contain
>>> s(ParallelListResourceBundle.java:208) ~[na:1.8.0_101]
>>> at sun.util.resources.ParallelListResourceBundle.containsKey(Pa
>>> rallelListResourceBundle.java:129) ~[na:1.8.0_101]
>>> at java.text.DateFormatSymbols.initializeData(DateFormatSymbols.java:702)
>>> ~[na:1.8.0_101]
>>> at java.text.DateFormatSymbols.<init>(DateFormatSymbols.java:146)
>>> ~[na:1.8.0_101]
>>> at sun.util.locale.provider.DateFormatSymbolsProviderImpl.getIn
>>> stance(DateFormatSymbolsProviderImpl.java:85) ~[na:1.8.0_101]
>>> at java.text.DateFormatSymbols.getProviderInstance(DateFormatSymbols.java:359)
>>> ~[na:1.8.0_101]
>>> at java.text.DateFormatSymbols.getInstanceRef(DateFormatSymbols.java:349)
>>> ~[na:1.8.0_101]
>>> at java.text.SimpleDateFormat.<init>(SimpleDateFormat.java:603)
>>> ~[na:1.8.0_101]
>>> at com.mysql.jdbc.PreparedStatement.setTimestampInternal(PreparedStatement.java:4224)
>>> ~[mysql-connector-java-5.1.39-bin.jar:5.1.39]
>>> at com.mysql.jdbc.PreparedStatement.setTimestamp(PreparedStatement.java:4183)
>>> ~[mysql-connector-java-5.1.39-bin.jar:5.1.39]
>>> at org.apache.ibatis.type.DateTypeHandler.setNonNullParameter(DateTypeHandler.java:33)
>>> ~[mybatis-3.3.0.jar:3.3.0]
>>> at org.apache.ibatis.type.DateTypeHandler.setNonNullParameter(DateTypeHandler.java:28)
>>> ~[mybatis-3.3.0.jar:3.3.0]
>>> at org.apache.ibatis.type.BaseTypeHandler.setParameter(BaseTypeHandler.java:53)
>>> ~[mybatis-3.3.0.jar:3.3.0]
>>> at org.apache.ibatis.type.UnknownTypeHandler.setNonNullParameter(UnknownTypeHandler.java:45)
>>> ~[mybatis-3.3.0.jar:3.3.0]
>>> at org.apache.ibatis.type.BaseTypeHandler.setParameter(BaseTypeHandler.java:53)
>>> ~[mybatis-3.3.0.jar:3.3.0]
>>> at org.apache.ibatis.scripting.defaults.DefaultParameterHandler
>>> .setParameters(DefaultParameterHandler.java:87)
>>> ~[mybatis-3.3.0.jar:3.3.0]
>>> at org.apache.ibatis.executor.statement.PreparedStatementHandle
>>> r.parameterize(PreparedStatementHandler.java:85)
>>> ~[mybatis-3.3.0.jar:3.3.0]
>>> at org.apache.ibatis.executor.statement.RoutingStatementHandler
>>> .parameterize(RoutingStatementHandler.java:63)
>>> ~[mybatis-3.3.0.jar:3.3.0]
>>> at org.apache.ibatis.executor.SimpleExecutor.prepareStatement(SimpleExecutor.java:77)
>>> ~[mybatis-3.3.0.jar:3.3.0]
>>> at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:61)
>>> ~[mybatis-3.3.0.jar:3.3.0]
>>> at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:303)
>>> ~[mybatis-3.3.0.jar:3.3.0]
>>> at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:154)
>>> ~[mybatis-3.3.0.jar:3.3.0]
>>> at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:102)
>>> ~[mybatis-3.3.0.jar:3.3.0]
>>> at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:82)
>>> ~[mybatis-3.3.0.jar:3.3.0]
>>>
>>> On Fri, Aug 26, 2016 at 10:05 AM, Mencel, Matt <mr-mencel at wiu.edu>
>>> wrote:
>>>
>>>> I just attempted my first import from LDAP. I have a condition that
>>>> limits the import to a single department of about 80 accounts, but the
>>>> import still processes all the others.
>>>>
>>>> The import processed nearly 30000 accounts. At just over 9000 the web
>>>> UI became unresponsive. It's been 30 minutes and I still can't do anything
>>>> in the UI.
>>>>
>>>> My tomcat java process is still eating up the CPU, but nothing has
>>>> logged in the idm.log for the last 30 minutes. Is there some other place I
>>>> can look to see what this process is doing?
>>>>
>>>> Here's the processes from htop...
>>>>
>>>> 111390 tomcat_mi 20 0 9163M 5052M 15752 S 591. 42.1 2h08:45
>>>> /usr/bin/java -Djava.util.logging.config.fil
>>>> e=/opt/tomcat_midpoint/conf/logging.properties
>>>> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
>>>> -Djdk.tls.ephe
>>>> 111404 tomcat_mi 20 0 9163M 5052M 15752 R 98.3 42.1 19:45.81
>>>> /usr/bin/java -Djava.util.logging.config.fil
>>>> e=/opt/tomcat_midpoint/conf/logging.properties
>>>> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
>>>> -Djdk.tls.ephe
>>>> 111407 tomcat_mi 20 0 9163M 5052M 15752 R 98.3 42.1 19:42.20
>>>> /usr/bin/java -Djava.util.logging.config.fil
>>>> e=/opt/tomcat_midpoint/conf/logging.properties
>>>> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
>>>> -Djdk.tls.ephe
>>>> 111405 tomcat_mi 20 0 9163M 5052M 15752 R 99.0 42.1 19:42.24
>>>> /usr/bin/java -Djava.util.logging.config.fil
>>>> e=/opt/tomcat_midpoint/conf/logging.properties
>>>> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
>>>> -Djdk.tls.ephe
>>>> 111406 tomcat_mi 20 0 9163M 5052M 15752 R 98.3 42.1 19:47.07
>>>> /usr/bin/java -Djava.util.logging.config.fil
>>>> e=/opt/tomcat_midpoint/conf/logging.properties
>>>> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
>>>> -Djdk.tls.ephe
>>>> 111402 tomcat_mi 20 0 9163M 5052M 15752 R 97.7 42.1 19:44.50
>>>> /usr/bin/java -Djava.util.logging.config.fil
>>>> e=/opt/tomcat_midpoint/conf/logging.properties
>>>> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
>>>> -Djdk.tls.ephe
>>>> 111403 tomcat_mi 20 0 9163M 5052M 15752 R 98.3 42.1 19:43.86
>>>> /usr/bin/java -Djava.util.logging.config.fil
>>>> e=/opt/tomcat_midpoint/conf/logging.properties
>>>> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
>>>> -Djdk.tls.ephe
>>>> 111408 tomcat_mi 20 0 9163M 5052M 15752 S 1.3 42.1 0:53.12
>>>> /usr/bin/java -Djava.util.logging.config.fil
>>>> e=/opt/tomcat_midpoint/conf/logging.properties
>>>> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
>>>> -Djdk.tls.ephe
>>>>
>>>> Details on one of the processes....
>>>> # ps aux | grep 11390
>>>>
>>>> tomcat_+ 111390 204 42.1 9383488 5173404 ? Ssl 08:57 133:18
>>>> /usr/bin/java -Djava.util.logging.config.fil
>>>> e=/opt/tomcat_midpoint/conf/logging.properties
>>>> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
>>>> -Djdk.tls.ephemeralDHKeySize=2048 -Dmidpoint.home=/opt/midpoint
>>>> -Djavax.net.ssl.trustStore=/opt/midpoint/keystore.jceks
>>>> -Djavax.net.ssl.trustStoreType=jceks -Xms512m -XX:MaxPermSize=1024m
>>>> -Xss1m -Xmx4096m -classpath /opt/tomcat_midpoint/bin/boots
>>>> trap.jar:/opt/tomcat_midpoint/bin/tomcat-juli.jar
>>>> -Dcatalina.base=/opt/tomcat_midpoint -Dcatalina.home=/opt/tomcat_midpoint
>>>> -Djava.io.tmpdir=/opt/tomcat_midpoint/temp
>>>> org.apache.catalina.startup.Bootstrap start
>>>>
>>>> Are those threads still just busy processing the records?
>>>>
>>>> Thanks,
>>>> Matt
>>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> midPoint mailing listmidPoint at lists.evolveum.comhttp://lists.evolveum.com/mailman/listinfo/midpoint
>>>
>>>
>>>
>>> _______________________________________________
>>> midPoint mailing list
>>> midPoint at lists.evolveum.com
>>> http://lists.evolveum.com/mailman/listinfo/midpoint
>>>
>>>
>>
>>
>>
>> CONFIDENTIALITY NOTICE:
>> This e-mail together with any attachments is proprietary and
>> confidential; intended for only the recipient(s) named above and may
>> contain information that is privileged. You should not retain, copy or use
>> this e-mail or any attachments for any purpose, or disclose all or any part
>> of the contents to any person. Any views or opinions expressed in this
>> e-mail are those of the author and do not represent those of the Baptist
>> School of Health Professions. If you have received this e-mail in error, or
>> are not the named recipient(s), you are hereby notified that any review,
>> dissemination, distribution or copying of this communication is prohibited
>> by the sender and to do so might constitute a violation of the Electronic
>> Communications Privacy Act, 18 U.S.C. section 2510-2521. Please immediately
>> notify the sender and delete this e-mail and any attachments from your
>> computer.
>>
>> _______________________________________________
>> midPoint mailing list
>> midPoint at lists.evolveum.com
>> http://lists.evolveum.com/mailman/listinfo/midpoint
>>
>>
>
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
>
--
CONFIDENTIALITY NOTICE:
This e-mail together with any attachments is proprietary and confidential;
intended for only the recipient(s) named above and may contain information
that is privileged. You should not retain, copy or use this e-mail or any
attachments for any purpose, or disclose all or any part of the contents to
any person. Any views or opinions expressed in this e-mail are those of the
author and do not represent those of the Baptist School of Health
Professions. If you have received this e-mail in error, or are not the
named recipient(s), you are hereby notified that any review, dissemination,
distribution or copying of this communication is prohibited by the sender
and to do so might constitute a violation of the Electronic Communications
Privacy Act, 18 U.S.C. section 2510-2521. Please immediately notify the
sender and delete this e-mail and any attachments from your computer.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20160826/9e8130b0/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: kficpbebihkpdopd.png
Type: image/png
Size: 50643 bytes
Desc: not available
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20160826/9e8130b0/attachment.png>
More information about the midPoint
mailing list