[midPoint-git] [Evolveum/midpoint] 45a7ad: Fix task progress reporting

[mederly]

  Branch: refs/heads/tmp/fixing-progress
  Home:   https://github.com/Evolveum/midpoint
  Commit: 45a7ad457286803176b451b4bb168cd89424773f
      https://github.com/Evolveum/midpoint/commit/45a7ad457286803176b451b4bb168cd89424773f
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2021-03-18 (Thu, 18 Mar 2021)

  Changed paths:
    A infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/IterativeOperationCollector.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/IterativeTaskInformation.java
    A infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/MappingStatisticsCollector.java
    A infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/NotificationStatisticsCollector.java
    A infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/ObjectActionsCollector.java
    A infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/ProvisioningStatisticsCollector.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/StatisticsCollector.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/StructuredProgressCollector.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/StructuredTaskProgress.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/StructuredTaskProgressPrinter.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/statistics/SynchronizationInformationCollector.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/util/task/BucketsProgressInformation.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/util/task/ItemsProgressInformation.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/util/task/TaskPartProgressInformation.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/util/task/TaskProgressInformation.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/util/task/TaskProgressUtil.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationCampaignCreationTaskHandler.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationClosingTaskHandler.java
    M model/certification-impl/src/main/java/com/evolveum/midpoint/certification/impl/AccessCertificationRemediationTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/cleanup/CleanUpTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/lens/projector/mappings/MappingEvaluator.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/scripting/ScriptExecutionTaskHandler.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/scripting/actions/AbstractObjectBasedActionExecutor.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/scripting/actions/BaseActionExecutor.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/scripting/actions/DiscoverConnectorsExecutor.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/scripting/actions/ReencryptExecutor.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/scripting/actions/ValidateExecutor.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/scripting/helpers/OperationsHelper.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/tasks/DeleteTaskHandler.java
    M model/model-intest/src/test/java/com/evolveum/midpoint/model/intest/sync/TestProgressReporting.java
    M model/notifications-impl/src/main/java/com/evolveum/midpoint/notifications/impl/api/transports/MailTransport.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/WorkflowManagerImpl.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractIterativeTaskPartExecution.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractSearchIterativeTaskPartExecution.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/AbstractTaskExecution.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/ItemProcessingGatekeeper.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/ItemProcessingStatistics.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/ProcessingCoordinator.java
    M repo/repo-common/src/main/java/com/evolveum/midpoint/repo/common/task/TaskReportingOptions.java
    M repo/repo-test-util/src/main/java/com/evolveum/midpoint/test/asserter/StructuredTaskProgressAsserter.java
    M repo/repo-test-util/src/main/java/com/evolveum/midpoint/test/asserter/TaskAsserter.java
    A repo/repo-test-util/src/main/java/com/evolveum/midpoint/test/asserter/TaskPartProgressAsserter.java
    M repo/repo-test-util/src/main/java/com/evolveum/midpoint/test/asserter/TaskPartProgressInformationAsserter.java
    M repo/repo-test-util/src/main/java/com/evolveum/midpoint/test/asserter/TaskProgressInformationAsserter.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/LightweightTaskHandler.java
    A repo/task-api/src/main/java/com/evolveum/midpoint/task/api/RunningLightweightTask.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/RunningTask.java
    A repo/task-api/src/main/java/com/evolveum/midpoint/task/api/RunningTaskStatisticsCollector.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/StateReporter.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/StatisticsCollectionStrategy.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/Task.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/TaskUtil.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/WorkBucketAwareTaskHandler.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/test/NullTaskImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/LightweightTaskManager.java
    A repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/RunningLightweightTaskImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/RunningTaskQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/JdbcPingTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/LightweightPartitioningTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/NoOpTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/nodes/NodeCleaner.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/run/BucketAwareHandlerExecution.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/run/HandlerExecutor.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/run/JobExecutor.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/statistics/Statistics.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/tasks/TaskCleaner.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/tasks/TaskInstantiator.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/tasks/TaskPersister.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/segmentation/StringWorkSegmentationStrategy.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockCycleTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockLongTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockParallelTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockSingleTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/MockWorkBucketsTaskHandler.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/TestTaskManagerBasic.java
    M repo/task-quartz-impl/src/test/java/com/evolveum/midpoint/task/quartzimpl/TestWorkDistribution.java

  Log Message:
  -----------
  Fix task progress reporting

Functional changes:
1. The "complete" flag for parts is now set.
2. Expected total is correctly cleared when not applicable.
3. Statistics are now zeroed on initial task start.
4. Open/closed items in structured progress are maintained correctly.

Code changes:
1. Cleaned up statistics-related code in RunningTaskQuartzImpl.
2. Separated RunningLightweightTask from RunningTask (including
implementations).
3. Iterative statistics are now always maintained (no opt-out possible).
4. Writing task statistics to repo is now covered by operation result
in standard way.
5. Cleaned up statistics keeping in ItemProcessingGatekeeper.

Work in progress. Throughput is not measured precisely yet.