[midPoint-git] [Evolveum/midpoint] fcff84: Change Set<Task> to List<Task> to lower contention

mederly noreply at github.com
Fri Feb 15 16:35:27 CET 2019 

  Branch: refs/heads/tmp/fixes-for-vilo-new
  Home:   https://github.com/Evolveum/midpoint
  Commit: fcff849fc3a1b6845ebbe5e40d3b19e543d4ee5b
      https://github.com/Evolveum/midpoint/commit/fcff849fc3a1b6845ebbe5e40d3b19e543d4ee5b
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-02-12 (Tue, 12 Feb 2019)

  Changed paths:
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/Task.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/TaskManager.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskManagerQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/ExecutionManager.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/JobExecutor.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/LocalNodeManager.java

  Log Message:
  -----------
  Change Set<Task> to List<Task> to lower contention

Lightweight asynchronous tasks are shared objects that hold their
data in thread-unsafe prism objects. This is to be fixed somehow.

Using HashSet<Task> structures makes this problem even worse because
hashCode/equals are called repetitively in various threads that
access those sets. So here we replace sets with lists to treat
at least some of the concurrency issues (MID-5136).


  Commit: e28ce2ac3b0d625d395dc689bf4844305979b7c7
      https://github.com/Evolveum/midpoint/commit/e28ce2ac3b0d625d395dc689bf4844305979b7c7
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-02-14 (Thu, 14 Feb 2019)

  Changed paths:
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/page/admin/configuration/PageAdminConfiguration.java
    M infra/prism-api/src/main/java/com/evolveum/midpoint/prism/xml/XmlTypeConverter.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/StatisticsCollector.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccCertTimedActionTriggerHandler.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationCampaignCreationTaskHandler.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationCampaignReiterationTriggerHandler.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationCloseStageApproachingTriggerHandler.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationCloseStageTriggerHandler.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationClosingTaskHandler.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationRemediationTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/ModelConstants.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/cleanup/CleanUpTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/cleanup/ShadowRefreshTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/controller/ExecuteDeltasTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/controller/ModelOperationTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/importer/ImportAccountsFromResourceTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/importer/ImportObjectsFromFileTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/integrity/ObjectIntegrityCheckResultHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/integrity/ObjectIntegrityCheckTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/integrity/ShadowIntegrityCheckResultHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/integrity/ShadowIntegrityCheckTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/AssignmentEvaluator.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/Clockwork.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/OperationalDataManager.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/scripting/ExecutionContext.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/scripting/IterativeScriptExecutionTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/scripting/ScriptExecutionTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/ExecuteChangesTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/FocusValidityScannerTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/LiveSyncTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/RecomputeTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/ReconciliationTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/SynchronizationContext.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/SynchronizeAccountResultHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/trigger/CompletedTaskCleanupTriggerHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/trigger/RecomputeTriggerHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/trigger/TriggerHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/trigger/TriggerScannerTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/AbstractScannerResultHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/AbstractScannerTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/AuditReindexTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/DeleteTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/ModelImplUtils.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/ReindexResultHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/util/ReindexTaskHandler.java
    M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/manual/TestSemiManualGroupingProposed.java
    M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/util/MockTaskHandler.java
    M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/util/MockTriggerHandler.java
    M model/report-impl/src/main/java/com/evolveum/midpoint/report/impl/ReportCreateTaskHandler.java
    M model/report-impl/src/test/java/com/evolveum/midpoint/report/TestReportJasper.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processes/common/WfTimedActionTriggerHandler.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processes/itemApproval/MidpointUtil.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processors/general/GeneralChangeProcessor.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processors/primary/WfPrepareChildOperationTaskHandler.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processors/primary/WfPrepareRootOperationTaskHandler.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processors/primary/WfPropagateTaskObjectReferenceTaskHandler.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/tasks/WfProcessInstanceShadowTaskHandler.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/tasks/WfTask.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/tasks/WfTaskController.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/tasks/WfTaskUtil.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/ProvisioningServiceImpl.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/ShadowCache.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/task/MultiPropagationResultHandler.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/task/MultiPropagationTaskHandler.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/task/PropagationResultHandler.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/task/PropagationTaskHandler.java
    M provisioning/provisioning-impl/src/test/java/com/evolveum/midpoint/provisioning/impl/mock/SynchronizationServiceMock.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractSearchIterativeResultHandler.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractSearchIterativeTaskHandler.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/TaskHandlerUtil.java
    M repo/repo-sql-impl-test/src/test/java/com/evolveum/midpoint/repo/sql/util/SimpleTaskAdapter.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/LightweightTaskHandler.java
    A repo/task-api/src/main/java/com/evolveum/midpoint/task/api/RunningTask.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/Task.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/TaskHandler.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/TaskManager.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/WorkBucketAwareTaskHandler.java
    A repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/InternalTaskInterface.java
    A repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/RunningTaskQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskManagerQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskQuartzImplUtil.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/ExecutionManager.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/JobExecutor.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/StalledTasksWatcher.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/TaskSynchronizer.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/JdbcPingTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/NoOpTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/PartitioningTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/WaitForSubtasksByPollingTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/WaitForTasksTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/WorkersCreationTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/WorkersRestartTaskHandler.java
    A repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/statistics/Statistics.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/WorkStateManager.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/workers/WorkersManager.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/AbstractTaskManagerTest.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/CleanupTest.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockCycleTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockLongTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockParallelTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockSingleTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockWorkBucketsTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/TestQuartzTaskManagerContract.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/TestWorkDistribution.java
    M repo/task-quartz-impl/src/test/resources/basic/task-100LightweightSubtasks.xml

  Log Message:
  -----------
  Make running tasks thread-safe by synchronization

This is to resolve ConcurrentModificationExceptions (e.g. MID-5111,
MID-5113, MID-4088, MID-3954, MID-5131, MID-5135).

Features:
1) Methods of Task implementation are synchronized on various
objects to ensure thread safety
2) RunningTask as a subtype of Task provides even more safety
by cloning returned objects where appropriate

Although the implementation is not 100% safe yet, it should be well
enough for current state of midPoint code base.


  Commit: 2d777dc23b6ebb2957b196da1244cbc25685c6c7
      https://github.com/Evolveum/midpoint/commit/2d777dc23b6ebb2957b196da1244cbc25685c6c7
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-02-15 (Fri, 15 Feb 2019)

  Changed paths:
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskManagerQuartzImpl.java

  Log Message:
  -----------
  Fix memory leak in tests


Compare: https://github.com/Evolveum/midpoint/compare/bffbd35f03f2...2d777dc23b6e

More information about the midPoint-svn mailing list