[midPoint-git] [Evolveum/midpoint] 0f567d: Improve synchronization situation reporting

[mederly]

  Branch: refs/heads/feature/new-sync-states
  Home:   https://github.com/Evolveum/midpoint
  Commit: 0f567df203a0b52a74968d26ee945ea4b9d46968
      https://github.com/Evolveum/midpoint/commit/0f567df203a0b52a74968d26ee945ea4b9d46968
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2021-02-03 (Wed, 03 Feb 2021)

  Changed paths:
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/StatisticsCollector.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/SynchronizationInformation.java
    A infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/SynchronizationInformationCollector.java
    A infra/schema/src/main/java/com/evolveum/midpoint/schema/util/SyncSituationUtil.java
    M infra/schema/src/main/resources/xml/ns/public/common/common-core-3.xsd
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/cleanup/ShadowRefreshTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/expr/MidpointFunctionsImpl.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/integrity/ObjectIntegrityCheckItemProcessor.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/integrity/ShadowIntegrityCheckItemProcessor.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/ChangeExecutor.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/LensContext.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/SynchronizationContext.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/SynchronizationService.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/SynchronizationServiceImpl.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/tasks/AsyncUpdateTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/tasks/LiveSyncTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/tasks/SyncItemProcessingRequest.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/tasks/Synchronizer.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/tasks/imp/ImportFromResourceTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/tasks/imp/ImportFromResourceTaskPartExecution.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/tasks/recon/ReconciliationTaskFirstPartExecution.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/tasks/recon/ReconciliationTaskSecondPartExecution.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/sync/tasks/recon/ReconciliationTaskThirdPartExecution.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/tasks/scanner/FocusValidityScannerItemProcessor.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/tasks/simple/SimpleIterativeTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/trigger/TriggerScannerItemProcessor.java
    M model/model-impl/src/test/java/com/evolveum/midpoint/model/impl/sync/TestCorrelationConfirmationEvaluator.java
    M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/reporting/TestTaskReporting.java
    M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/sync/TestImportRecon.java
    M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/sync/TestLiveSyncTaskMechanics.java
    M provisioning/provisioning-api/src/main/java/com/evolveum/midpoint/provisioning/api/ResourceObjectShadowChangeDescription.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/ProvisioningServiceImpl.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/task/MultiPropagationItemProcessor.java
    M provisioning/provisioning-impl/src/main/java/com/evolveum/midpoint/provisioning/impl/task/PropagationItemProcessor.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractIterativeItemProcessor.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractIterativeTaskPartExecution.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractSearchIterativeItemProcessor.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractSearchIterativeTaskPartExecution.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractTaskHandler.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/ItemProcessingGatekeeper.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/ItemProcessingRequest.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/ObjectProcessingRequest.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/TaskReportingOptions.java
    M repo/repo-sql-impl-test/src/main/java/com/evolveum/midpoint/repo/sql/testing/LightweightIdentifierGeneratorMock.java
    M repo/repo-test-util/src/main/java/com/evolveum/midpoint/test/asserter/SynchronizationInfoAsserter.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/LightweightIdentifier.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/LightweightIdentifierGenerator.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/Task.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/test/NullTaskImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/InternalTaskInterface.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/LightweightIdentifierGeneratorImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/statistics/Statistics.java
    M testing/story/src/test/java/com/evolveum/midpoint/testing/story/TestThresholds.java
    M testing/story/src/test/java/com/evolveum/midpoint/testing/story/TestThresholdsLiveSyncFull.java
    M testing/story/src/test/java/com/evolveum/midpoint/testing/story/TestThresholdsLiveSyncSimulate.java
    M testing/story/src/test/java/com/evolveum/midpoint/testing/story/TestThresholdsLiveSyncSimulateMultithreaded.java
    M testing/story/src/test/java/com/evolveum/midpoint/testing/story/TestThresholdsReconFull.java
    M testing/story/src/test/java/com/evolveum/midpoint/testing/story/TestThresholdsReconSimulate.java
    M testing/story/src/test/java/com/evolveum/midpoint/testing/story/TestThresholdsReconSimulateMultithreaded.java
    M tools/ninja/pom.xml
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/util/BasicLightweightIdentifierGenerator.java

  Log Message:
  -----------
  Improve synchronization situation reporting

In order to reduce misunderstanding of the synchronization situation
statistics the data were restructured: instead of summary information
(situations before/after) we now provide counters for each transition
between situations.

Moreover, we now distinguish not only "before" and "after" states
(now called onSynchronizationStart/onSynchronizationEnd) but also
the ultimate "before" state: onProcessingStart. It is the state in
shadow before the synchronization service is even invoked. Also,
reasons for skipping synchronization are reported separately.

Third, unrelated synchronization events were avoided, hopefully
improving the precision. This filtering is not finished, however:
It is perhaps too strong, not catching "delete" situations occurring
in remaining shadows reconciliation.

Original reporting is (temporarily) kept in place, just to allow
checking the new vs. old counters.

Other changes:

1) Original synchronization situation counts are now represented with
minOccurs=0 (instead of 1). So getters can get null values instead of 0.

2) Transitions are now reported directly in change executor, instead
of using projCtx.synchronizationSituationResolved.

3) Reporting options are now specific for task part. This was used
to (maybe temporarily) turning off sync statistics for shadows recon.

This is related to MID-6384.