[midPoint-git] [Evolveum/midpoint] 08b949: Fix caching configuration for multithreaded tasks

mederly noreply at github.com
Tue Jun 11 12:26:16 CEST 2019 

  Branch: refs/heads/feature/gui-wrapper
  Home:   https://github.com/Evolveum/midpoint
  Commit: 08b9498880351a7d6c9a7014682c55099e8263e0
      https://github.com/Evolveum/midpoint/commit/08b9498880351a7d6c9a7014682c55099e8263e0
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-05-24 (Fri, 24 May 2019)

  Changed paths:
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/cache/CacheConfigurationManager.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/RepositoryCache.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractSearchIterativeResultHandler.java
    M repo/repo-sql-impl-test/src/test/java/com/evolveum/midpoint/repo/sql/util/SimpleTaskAdapter.java
    M repo/repo-sql-impl/src/main/java/com/evolveum/midpoint/repo/sql/perf/SqlPerformanceMonitorImpl.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/Task.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskQuartzImpl.java
    M testing/story/src/test/java/com/evolveum/midpoint/testing/story/ldap/TestLdapAssociationPerformance.java
    A testing/story/src/test/resources/ldap/assoc-perf/system-configuration-no-role-cache.xml
    A testing/story/src/test/resources/ldap/assoc-perf/task-recompute-4.xml

  Log Message:
  -----------
  Fix caching configuration for multithreaded tasks

Task-specific caching profiles in worker threads were not applied.
These threads were running with system-wide (global) caching
configuration.


  Commit: bf9ddac76d3aac68f9fb157e3315d93a7c496e75
      https://github.com/Evolveum/midpoint/commit/bf9ddac76d3aac68f9fb157e3315d93a7c496e75
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-05-24 (Fri, 24 May 2019)

  Changed paths:
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/ChangeNotificationDispatcherImpl.java

  Log Message:
  -----------
  Improve synchronization of provisioning listeners

1. There were some places where synchronization was missing
2. We also added diagnostics of listeners leaks


  Commit: a0eae3f73ada6fe83ca238311e494c61dfc62ff2
      https://github.com/Evolveum/midpoint/commit/a0eae3f73ada6fe83ca238311e494c61dfc62ff2
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-05-25 (Sat, 25 May 2019)

  Changed paths:
    M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/sync/AbstractSynchronizationStoryTest.java
    M model/model-test/src/main/java/com/evolveum/midpoint/model/test/AbstractModelIntegrationTest.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/TaskManager.java

  Log Message:
  -----------
  Fix partitioned synchronization tests

There were some race conditions related to retrieving tasks in tests.


  Commit: 0dd1a7699f0836f64c55814edb29418b7ad417a9
      https://github.com/Evolveum/midpoint/commit/0dd1a7699f0836f64c55814edb29418b7ad417a9
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-05-25 (Sat, 25 May 2019)

  Changed paths:
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/RepositoryCache.java

  Log Message:
  -----------
  Fix global query cache corruption

Global query cache value was not cloned before returning to client
after cache hit.


  Commit: e3b92b8a41d131fd31e53da0a87bc72c1f5c5591
      https://github.com/Evolveum/midpoint/commit/e3b92b8a41d131fd31e53da0a87bc72c1f5c5591
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-05-25 (Sat, 25 May 2019)

  Changed paths:
    M infra/prism-api/src/main/java/com/evolveum/midpoint/prism/util/CloneUtil.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/ActionsExecutedInformation.java
    A infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/CachePerformanceInformationUtil.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/EnvironmentalPerformanceInformation.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/IterativeTaskInformation.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/ProvisioningStatisticsData.java
    A infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/RepositoryPerformanceInformationUtil.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/StatisticsUtil.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/SynchronizationInformation.java
    R infra/schema/src/main/java/com/evolveum/midpoint/schema/util/CachePerformanceInformationUtil.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/util/TaskTypeUtil.java
    M infra/util/src/main/java/com/evolveum/midpoint/util/caching/CachePerformanceCollector.java
    M infra/util/src/main/java/com/evolveum/midpoint/util/caching/CacheUtil.java
    M model/model-test/src/main/java/com/evolveum/midpoint/model/test/AbstractModelIntegrationTest.java
    R repo/repo-api/src/main/java/com/evolveum/midpoint/repo/api/perf/PerformanceInformationUtil.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractSearchIterativeTaskHandler.java
    M repo/repo-test-util/src/main/java/com/evolveum/midpoint/test/AbstractIntegrationTest.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/LightweightTaskHandler.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/RunningTask.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/RunningTaskQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/HandlerExecutor.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/JobExecutor.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/statistics/Statistics.java
    M testing/story/src/test/java/com/evolveum/midpoint/testing/story/ldap/TestLdapAssociationPerformance.java
    M testing/story/src/test/java/com/evolveum/midpoint/testing/story/ldap/TestLdapReconPerformance.java
    M testing/story/src/test/resources/ldap/assoc-perf/task-recompute-1.xml
    M testing/story/src/test/resources/ldap/assoc-perf/task-recompute-4.xml
    A testing/story/src/test/resources/ldap/assoc-perf/task-recompute-multinode-multithreaded.xml
    A testing/story/src/test/resources/ldap/assoc-perf/task-recompute-multinode.xml
    M testing/story/src/test/resources/ldap/assoc-perf/task-recompute-no-role-and-shadow-cache.xml

  Log Message:
  -----------
  Fix statistics for multinode-multithreaded tasks

Also, added displayOperationStatistics to be used e.g. in tests
or for logging.


  Commit: 99330369bbfe5e2be15371122d8c82f74a9af792
      https://github.com/Evolveum/midpoint/commit/99330369bbfe5e2be15371122d8c82f74a9af792
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-05-25 (Sat, 25 May 2019)

  Changed paths:
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/Clockwork.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/ChangeNotificationDispatcherImpl.java

  Log Message:
  -----------
  Fix leaking of change listeners (MID-5355)

When AssociationSearchExpressionEvaluatorCache was entered twice,
two invalidators for it were created and registered (by mistake).
Upon exiting the cache, only one of them was unregistered, because
the reference to the other one was already lost at that moment.

(cherry-picked from f1433a6f)


  Commit: 6a415e089ef55814368f858a7d8562fb46f6946e
      https://github.com/Evolveum/midpoint/commit/6a415e089ef55814368f858a7d8562fb46f6946e
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-06-10 (Mon, 10 Jun 2019)

  Changed paths:
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/cache/CacheConfigurationManager.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/ActionsExecutedInformation.java
    A infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/CachePerformanceInformationUtil.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/EnvironmentalPerformanceInformation.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/IterativeTaskInformation.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/ProvisioningStatisticsData.java
    A infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/RepositoryPerformanceInformationUtil.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/StatisticsUtil.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/SynchronizationInformation.java
    R infra/schema/src/main/java/com/evolveum/midpoint/schema/util/CachePerformanceInformationUtil.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/util/TaskTypeUtil.java
    M infra/util/src/main/java/com/evolveum/midpoint/util/caching/CachePerformanceCollector.java
    M infra/util/src/main/java/com/evolveum/midpoint/util/caching/CacheUtil.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/Clockwork.java
    M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/sync/AbstractSynchronizationStoryTest.java
    M model/model-test/src/main/java/com/evolveum/midpoint/model/test/AbstractModelIntegrationTest.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/ChangeNotificationDispatcherImpl.java
    R repo/repo-api/src/main/java/com/evolveum/midpoint/repo/api/perf/PerformanceInformationUtil.java
    M repo/repo-cache/src/main/java/com/evolveum/midpoint/repo/cache/RepositoryCache.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractSearchIterativeResultHandler.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractSearchIterativeTaskHandler.java
    M repo/repo-sql-impl-test/src/test/java/com/evolveum/midpoint/repo/sql/util/SimpleTaskAdapter.java
    M repo/repo-sql-impl/src/main/java/com/evolveum/midpoint/repo/sql/perf/SqlPerformanceMonitorImpl.java
    M repo/repo-test-util/src/main/java/com/evolveum/midpoint/test/AbstractIntegrationTest.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/LightweightTaskHandler.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/RunningTask.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/Task.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/TaskManager.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/RunningTaskQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/HandlerExecutor.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/JobExecutor.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/statistics/Statistics.java
    M testing/story/src/test/java/com/evolveum/midpoint/testing/story/ldap/TestLdapAssociationPerformance.java
    M testing/story/src/test/java/com/evolveum/midpoint/testing/story/ldap/TestLdapReconPerformance.java
    A testing/story/src/test/resources/ldap/assoc-perf/system-configuration-no-role-cache.xml
    M testing/story/src/test/resources/ldap/assoc-perf/task-recompute-1.xml
    A testing/story/src/test/resources/ldap/assoc-perf/task-recompute-4.xml
    A testing/story/src/test/resources/ldap/assoc-perf/task-recompute-multinode-multithreaded.xml
    A testing/story/src/test/resources/ldap/assoc-perf/task-recompute-multinode.xml
    M testing/story/src/test/resources/ldap/assoc-perf/task-recompute-no-role-and-shadow-cache.xml

  Log Message:
  -----------
  Merge branch 'feature/optimizing-repo-access' into feature/gui-wrapper

# Conflicts:
#	infra/prism-api/src/main/java/com/evolveum/midpoint/prism/util/CloneUtil.java
#	model/model-test/src/main/java/com/evolveum/midpoint/model/test/AbstractModelIntegrationTest.java
#	repo/repo-sql-impl-test/src/test/java/com/evolveum/midpoint/repo/sql/util/SimpleTaskAdapter.java
#	repo/task-api/src/main/java/com/evolveum/midpoint/task/api/Task.java


  Commit: 7c7fbfd390dfd8a4c6f4a9bbcf94be46d8781abe
      https://github.com/Evolveum/midpoint/commit/7c7fbfd390dfd8a4c6f4a9bbcf94be46d8781abe
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-06-10 (Mon, 10 Jun 2019)

  Changed paths:
    M model/model-impl/src/test/java/com/evolveum/midpoint/model/impl/AbstractInternalModelIntegrationTest.java
    M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/AbstractConfiguredModelIntegrationTest.java
    M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/password/AbstractPasswordTest.java
    M model/model-test/src/main/java/com/evolveum/midpoint/model/test/AbstractModelIntegrationTest.java
    M testing/story/pom.xml
    A testing/story/src/test/java/com/evolveum/midpoint/testing/story/perf/Generate.java
    A testing/story/src/test/java/com/evolveum/midpoint/testing/story/perf/TestImport.java
    A testing/story/src/test/resources/perf/import/org-basic.xml
    A testing/story/src/test/resources/perf/import/resource-source.xml
    A testing/story/src/test/resources/perf/import/source.sql
    A testing/story/src/test/resources/perf/import/system-configuration.xml
    A testing/story/src/test/resources/perf/import/task-import.xml
    A testing/story/src/test/resources/perf/import/template-import.xml
    A testing/story/src/test/resources/schema/perf-import.xsd

  Log Message:
  -----------
  Add bulk import performance test (MID-5368)


  Commit: d59290317c4fa4fb0634f43469810356075ea807
      https://github.com/Evolveum/midpoint/commit/d59290317c4fa4fb0634f43469810356075ea807
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-06-10 (Mon, 10 Jun 2019)

  Changed paths:
    M infra/util/src/main/java/com/evolveum/midpoint/util/aspect/MidpointInterceptor.java
    M repo/repo-sql-impl-test/src/main/java/com/evolveum/midpoint/repo/sql/testing/TestSqlRepositoryFactory.java
    M repo/repo-sql-impl-test/src/main/java/com/evolveum/midpoint/repo/sql/testing/TestSqlRepositoryServiceImpl.java
    M repo/repo-test-util/src/main/java/com/evolveum/midpoint/test/DummyAuditService.java
    M testing/story/src/test/java/com/evolveum/midpoint/testing/story/perf/TestImport.java
    M testing/story/src/test/resources/perf/import/system-configuration.xml

  Log Message:
  -----------
  Adapt bulk import test

1. Disabled dummy audit service to avoid memory leak (audit records)
2. Configured regular profiling intervals to observe performance

Temporarily disabled TestSqlRepositoryServiceImpl - but the class
currently does nothing, so nothing changed by this.


  Commit: 530ae76369d0e40cfb9050825c1b4e831dccfbe4
      https://github.com/Evolveum/midpoint/commit/530ae76369d0e40cfb9050825c1b4e831dccfbe4
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-06-11 (Tue, 11 Jun 2019)

  Changed paths:
    M config/sql/_all/oracle-4.0-all.sql
    M config/sql/_all/oracle-upgrade-3.9-4.0.sql
    M model/model-test/src/main/java/com/evolveum/midpoint/model/test/AbstractModelIntegrationTest.java
    M repo/repo-sql-impl-test/src/test/java/com/evolveum/midpoint/repo/sql/ConcurrencyTest.java
    M repo/repo-sql-impl/src/main/java/com/evolveum/midpoint/repo/sql/helpers/BaseHelper.java
    A testing/story/src/test/java/com/evolveum/midpoint/testing/story/TestShadowsPerformance.java
    A testing/story/src/test/resources/shadows/resource-dummy.xml
    A testing/story/src/test/resources/shadows/system-configuration.xml
    A testing/story/src/test/resources/shadows/task-bulk-delete.xml
    A testing/story/src/test/resources/shadows/task-import.xml
    A testing/story/src/test/resources/shadows/task-reconciliation.xml

  Log Message:
  -----------
  Fix ORA-00060 on object deletion (MID-5365)

A couple of foreign key related indices were missing from Oracle schema.
Also, detection of serialization-related issues was faulty.


  Commit: 94fec84654d8f59848658e3adfe3e4031634277f
      https://github.com/Evolveum/midpoint/commit/94fec84654d8f59848658e3adfe3e4031634277f
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-06-11 (Tue, 11 Jun 2019)

  Changed paths:
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/gui/impl/component/icon/IconCssStyle.java
    A gui/admin-gui/src/main/java/com/evolveum/midpoint/gui/impl/component/icon/TopRightForColumnIconCssStyle.java
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/component/objectdetails/FocusProjectionsTabPanel.java
    M gui/admin-gui/src/main/resources/static/less/midpoint/icon-style.less

  Log Message:
  -----------
  Merge remote-tracking branch 'origin/feature/gui-wrapper' into feature/gui-wrapper


Compare: https://github.com/Evolveum/midpoint/compare/b9ea0923aee0...94fec84654d8

More information about the midPoint-svn mailing list