[midPoint-git] [Evolveum/midpoint] 7902ec: Add template expansion engine

mederly noreply at github.com
Thu Dec 12 16:42:10 UTC 2019 

  Branch: refs/heads/support-4.0
  Home:   https://github.com/Evolveum/midpoint
  Commit: 7902ec0da95660d403e24898b09bb1f0658d8445
      https://github.com/Evolveum/midpoint/commit/7902ec0da95660d403e24898b09bb1f0658d8445
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-12-12 (Thu, 12 Dec 2019)

  Changed paths:
    M infra/common/src/main/java/com/evolveum/midpoint/common/configuration/api/MidpointConfiguration.java
    M infra/util/src/main/java/com/evolveum/midpoint/util/MiscUtil.java
    R infra/util/src/main/java/com/evolveum/midpoint/util/TemplateUtil.java
    A infra/util/src/main/java/com/evolveum/midpoint/util/template/AbstractChainedResolver.java
    A infra/util/src/main/java/com/evolveum/midpoint/util/template/JavaPropertiesResolver.java
    A infra/util/src/main/java/com/evolveum/midpoint/util/template/MapResolver.java
    A infra/util/src/main/java/com/evolveum/midpoint/util/template/OsEnvironmentResolver.java
    A infra/util/src/main/java/com/evolveum/midpoint/util/template/ReferenceResolver.java
    A infra/util/src/main/java/com/evolveum/midpoint/util/template/TemplateEngine.java
    A infra/util/src/main/java/com/evolveum/midpoint/util/template/TemplateResolution.java
    M infra/util/src/test/java/com/evolveum/midpoint/util/TestMiscUtil.java
    A infra/util/src/test/java/com/evolveum/midpoint/util/TestTemplateEngine.java
    M infra/util/testng-unit.xml
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/controller/ModelDiagController.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/Initializer.java
    A repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/NodeIdComputer.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskManagerConfiguration.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/PartitioningTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/workers/WorkersManager.java

  Log Message:
  -----------
  Add template expansion engine

This is a generic template expansion engine. Here we use it to implement
expressions for cluster node identifiers (-Dmidpoint.nodeIdExpression).

It is the first part of resolution of MID-5904.

(cherry picked from commit ca20dc3d549b5d03c602c0a76253a986e92069a6)


  Commit: 9ccc55049e41fb3ecdbb5d256a654e8c7b0ab71d
      https://github.com/Evolveum/midpoint/commit/9ccc55049e41fb3ecdbb5d256a654e8c7b0ab71d
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-12-12 (Thu, 12 Dec 2019)

  Changed paths:
    M infra/prism-api/src/main/java/com/evolveum/midpoint/prism/xml/XmlTypeConverter.java
    M infra/schema/src/main/resources/xml/ns/public/common/common-core-3.xsd
    M infra/util/src/main/java/com/evolveum/midpoint/util/template/ReferenceResolver.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/cleanup/CleanUpTaskHandler.java
    M model/report-impl/src/main/java/com/evolveum/midpoint/report/impl/ReportManagerImpl.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/TaskManager.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/NodeIdComputer.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskManagerConfiguration.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskManagerQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/cluster/ClusterExecutionHelperImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/cluster/ClusterManager.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/cluster/NodeRegistrar.java

  Log Message:
  -----------
  Add dead nodes handling (MID-5904)

Marking nodes that have not checked-in for 30 seconds as down.
Avoiding REST calls to nodes that are marked as down.
Deleting dead nodes according to specified clean-up policy.
Minor fixes in cleanup task handler and closed task cleanup method.
New XmlTypeConverter.compareMillis method.

(cherry picked from commit 82381c2c4c8c24e501640679dcdeb2afe836bef6)


  Commit: e8fd02d82347d6de373d7e60d7061dc94e72033f
      https://github.com/Evolveum/midpoint/commit/e8fd02d82347d6de373d7e60d7061dc94e72033f
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-12-12 (Thu, 12 Dec 2019)

  Changed paths:
    M build-system/pom.xml
    M gui/admin-gui/pom.xml
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/page/admin/resources/PageResourceVisualization.java
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/security/MidPointApplication.java
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/security/WebApplicationConfiguration.java
    M infra/common/pom.xml
    M infra/common/src/main/java/com/evolveum/midpoint/common/InternalsConfigController.java
    M infra/common/src/main/java/com/evolveum/midpoint/common/LoggingConfigurationManager.java
    M infra/common/src/main/java/com/evolveum/midpoint/common/configuration/api/MidpointConfiguration.java
    M infra/common/src/main/java/com/evolveum/midpoint/common/configuration/api/RuntimeConfiguration.java
    M infra/schema/pom.xml
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/internals/InternalsConfig.java
    M model/model-common/pom.xml
    M model/model-common/src/main/java/com/evolveum/midpoint/model/common/ConstantsManager.java
    M model/model-common/src/test/java/com/evolveum/midpoint/model/common/expression/ExpressionTestUtil.java
    M model/workflow-impl/pom.xml
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/WfConfiguration.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/processors/BaseChangeProcessor.java
    M provisioning/ucf-impl-connid/pom.xml
    M provisioning/ucf-impl-connid/src/main/java/com/evolveum/midpoint/provisioning/ucf/impl/connid/ConnectorFactoryConnIdImpl.java
    M repo/audit-api/pom.xml
    M repo/audit-api/src/main/java/com/evolveum/midpoint/audit/api/AuditServiceFactory.java
    M repo/audit-impl/pom.xml
    M repo/audit-impl/src/main/java/com/evolveum/midpoint/audit/impl/LoggerAuditServiceFactory.java
    M repo/repo-api/pom.xml
    M repo/repo-api/src/main/java/com/evolveum/midpoint/repo/api/RepositoryServiceFactory.java
    M repo/repo-sql-impl-test/pom.xml
    M repo/repo-sql-impl-test/src/main/java/com/evolveum/midpoint/repo/sql/testing/TestSqlRepositoryFactory.java
    M repo/repo-sql-impl/pom.xml
    M repo/repo-sql-impl/src/main/java/com/evolveum/midpoint/repo/sql/SqlAuditServiceFactory.java
    M repo/repo-sql-impl/src/main/java/com/evolveum/midpoint/repo/sql/SqlRepositoryConfiguration.java
    M repo/repo-sql-impl/src/main/java/com/evolveum/midpoint/repo/sql/SqlRepositoryFactory.java
    M repo/repo-test-util/pom.xml
    M repo/repo-test-util/src/main/java/com/evolveum/midpoint/test/DummyAuditServiceFactory.java
    M repo/system-init/pom.xml
    M repo/system-init/src/main/java/com/evolveum/midpoint/init/AuditFactory.java
    M repo/system-init/src/main/java/com/evolveum/midpoint/init/ConfigurablePrismContextFactory.java
    M repo/system-init/src/main/java/com/evolveum/midpoint/init/ConfigurableProtectorFactory.java
    M repo/system-init/src/main/java/com/evolveum/midpoint/init/ProtectorConfiguration.java
    M repo/system-init/src/main/java/com/evolveum/midpoint/init/RepositoryFactory.java
    M repo/system-init/src/main/java/com/evolveum/midpoint/init/StartupConfiguration.java
    M repo/system-init/src/main/java/com/evolveum/midpoint/init/SystemConfigurationSectionImpl.java
    M repo/system-init/src/test/java/com/evolveum/midpoint/init/TestConfigurationLoad.java
    M repo/task-quartz-impl/pom.xml
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/NodeIdComputer.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskManagerConfiguration.java
    M tools/ninja/pom.xml
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/util/InitializationBeanPostprocessor.java

  Log Message:
  -----------
  Migrate to Apache Commons Configuration 2

Currently this does not work because of some POM dependencies.

(cherry picked from commit 017ef1fa20b5e6ef338be42be98e362b204672cf)


  Commit: dba9153fb860a3bb64de7a4aa69ad75c8c8a3cf0
      https://github.com/Evolveum/midpoint/commit/dba9153fb860a3bb64de7a4aa69ad75c8c8a3cf0
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-12-12 (Thu, 12 Dec 2019)

  Changed paths:
    M build-system/pom.xml
    M gui/admin-gui/pom.xml
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/boot/AbstractSpringBootApplication.java
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/boot/MidPointSpringApplication.java
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/security/MidPointApplication.java
    M gui/admin-gui/src/test/java/com/evolveum/midpoint/gui/test/TestMidPointSpringApplication.java
    M infra/common/src/main/java/com/evolveum/midpoint/common/LocalizationServiceImpl.java
    M infra/common/src/main/java/com/evolveum/midpoint/common/configuration/api/MidpointConfiguration.java
    M infra/common/src/test/java/com/evolveum/midpoint/common/LocalizationTest.java
    M infra/util/pom.xml
    A infra/util/src/main/java/com/evolveum/midpoint/util/NetworkUtil.java
    R infra/util/src/main/java/com/evolveum/midpoint/util/template/AbstractChainedResolver.java
    R infra/util/src/main/java/com/evolveum/midpoint/util/template/JavaPropertiesResolver.java
    R infra/util/src/main/java/com/evolveum/midpoint/util/template/MapResolver.java
    R infra/util/src/main/java/com/evolveum/midpoint/util/template/OsEnvironmentResolver.java
    R infra/util/src/main/java/com/evolveum/midpoint/util/template/ReferenceResolver.java
    A infra/util/src/main/java/com/evolveum/midpoint/util/template/StringSubstitutorUtil.java
    R infra/util/src/main/java/com/evolveum/midpoint/util/template/TemplateEngine.java
    R infra/util/src/main/java/com/evolveum/midpoint/util/template/TemplateResolution.java
    A infra/util/src/test/java/com/evolveum/midpoint/util/TestStringSubstitutorUtil.java
    R infra/util/src/test/java/com/evolveum/midpoint/util/TestTemplateEngine.java
    M infra/util/testng-unit.xml
    M model/model-impl/pom.xml
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/ExtensionSchemaRestService.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/controller/ModelDiagController.java
    M model/report-impl/src/main/java/com/evolveum/midpoint/report/impl/ReportJasperCreateTaskHandler.java
    M model/report-impl/src/main/java/com/evolveum/midpoint/report/impl/ReportUtils.java
    M model/workflow-impl/src/main/java/com/evolveum/midpoint/wf/impl/WfConfiguration.java
    M repo/repo-sql-impl/pom.xml
    M repo/repo-sql-impl/src/main/java/com/evolveum/midpoint/repo/sql/SqlAuditServiceFactory.java
    M repo/repo-sql-impl/src/main/java/com/evolveum/midpoint/repo/sql/SqlRepositoryConfiguration.java
    M repo/system-init/src/main/java/com/evolveum/midpoint/init/ApplicationHomeSetup.java
    M repo/system-init/src/main/java/com/evolveum/midpoint/init/AuditFactory.java
    M repo/system-init/src/main/java/com/evolveum/midpoint/init/ConfigurableProtectorFactory.java
    M repo/system-init/src/main/java/com/evolveum/midpoint/init/StartupConfiguration.java
    A repo/system-init/src/main/java/com/evolveum/midpoint/init/interpol/HostnameLookup.java
    A repo/system-init/src/main/java/com/evolveum/midpoint/init/interpol/RandomLookup.java
    M repo/system-init/src/test/java/com/evolveum/midpoint/init/TestConfigurationLoad.java
    M repo/task-quartz-impl/pom.xml
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/NodeIdComputer.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskManagerConfiguration.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/cluster/NodeRegistrar.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/handlers/PartitioningTaskHandler.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/tracing/TracerImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/work/workers/WorkersManager.java
    M tools/midpoint-war-layout/src/main/java/com/evolveum/midpoint/tools/layout/MidPointWarLauncher.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/impl/NinjaContext.java

  Log Message:
  -----------
  Rework node ID generation to Apache Conf 2.6

Fixed pom.xml changes related to Apache Commons Configuration 2.6.
Cleaned up ApplicationHomeSetup and StartupConfiguration code.
Fixed handling of midpoint.home throughout midPoint code.
Removed TemplateEngine. It had a sad short life, really. Replaced with
  StringSubstitutor and ConfigurationInterpolator calls.

This should (finally) resolve MID-5904.

(cherry picked from commit b95e40e4f5338272fc4f716705584bed154e2541)


  Commit: 160e01ba941f3d8a1e6dd98770a5f09fb1b9d573
      https://github.com/Evolveum/midpoint/commit/160e01ba941f3d8a1e6dd98770a5f09fb1b9d573
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-12-12 (Thu, 12 Dec 2019)

  Changed paths:
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskManagerConfiguration.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/cluster/ClusterManager.java

  Log Message:
  -----------
  Mark node as down after 15 minutes by default

...also fixing a bug in aliveness check routine.

(Relates to MID-5904.)

(cherry picked from commit f7d542f503c269a4e78a5a6adabe5bdab10a7cf6)


  Commit: c89956998d1a791a84529aa70b1c1486298f47ff
      https://github.com/Evolveum/midpoint/commit/c89956998d1a791a84529aa70b1c1486298f47ff
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-12-12 (Thu, 12 Dec 2019)

  Changed paths:
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/page/admin/configuration/InternalsLoggedInUsersPanel.java
    M infra/schema/src/main/java/com/evolveum/midpoint/schema/GetOperationOptionsBuilder.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/ClusterCacheListener.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/security/NodeAuthenticationEvaluatorImpl.java
    M model/report-impl/src/main/java/com/evolveum/midpoint/report/impl/ReportManagerImpl.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/ClusterExecutionHelper.java
    A repo/task-api/src/main/java/com/evolveum/midpoint/task/api/ClusterExecutionOptions.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/TaskManager.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskManagerQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/cluster/ClusterExecutionHelperImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/cluster/NodeRegistrar.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/RemoteNodesManager.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/remote/RestConnector.java

  Log Message:
  -----------
  Improve dead and almost-dead nodes handling

1) Remote node authentication did not work correctly if there were
dead or almost-dead nodes with the same hostname/address as a live node
2) Introduced ClusterExecutionOptions in order to set how hard should
we try connecting to remote nodes (only for live nodes, also for
almost-dead nodes, for all nodes). This is to minimize delays and
error messages e.g. on cache invalidation events.

(Relates to MID-5904.)

(cherry picked from commit 66b5479c0ce212ab815b5186c2c1b35c8700cad3)


  Commit: dc33754e743ec773f19138a2952c6325e582152f
      https://github.com/Evolveum/midpoint/commit/dc33754e743ec773f19138a2952c6325e582152f
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-12-12 (Thu, 12 Dec 2019)

  Changed paths:
    M gui/admin-gui/src/main/java/com/evolveum/midpoint/web/page/admin/configuration/InternalsLoggedInUsersPanel.java
    M infra/schema/src/main/resources/xml/ns/public/common/common-core-3.xsd
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/ClusterCacheListener.java
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/security/NodeAuthenticationEvaluatorImpl.java
    M model/report-impl/src/main/java/com/evolveum/midpoint/report/impl/ReportManagerImpl.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/ClusterExecutionOptions.java
    M repo/task-api/src/main/java/com/evolveum/midpoint/task/api/TaskManager.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/TaskManagerQuartzImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/cluster/ClusterExecutionHelperImpl.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/cluster/ClusterManager.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/cluster/NodeRegistrar.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/RemoteNodesManager.java
    M repo/task-quartz-impl/src/main/java/com/evolveum/midpoint/task/quartzimpl/execution/remote/RestConnector.java

  Log Message:
  -----------
  Add "not checking in" node state

This is to differentiate between nodes that are permanently down
and ones that currently are not checking in for a short time
(between 30 and 900 seconds, by default). Also got rid of "almost dead"
terminology, replacing that by "not checking in".

(cherry picked from commit 18b4dccc6d60ef7fc9217d978f23a79dca187143)


  Commit: 5fcd6b0c731a4f4605a87114a228a13a400989a1
      https://github.com/Evolveum/midpoint/commit/5fcd6b0c731a4f4605a87114a228a13a400989a1
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2019-12-12 (Thu, 12 Dec 2019)

  Changed paths:
    M model/model-impl/src/main/java/com/evolveum/midpoint/model/impl/security/NodeAuthenticationEvaluatorImpl.java

  Log Message:
  -----------
  Fix remote node authentication

The case when not-checking-in nodes are there was handled improperly.

(cherry picked from commit afa28faf52025c6eeda06648e9da238327d6c117)


Compare: https://github.com/Evolveum/midpoint/compare/244953aa03fa...5fcd6b0c731a

More information about the midPoint-svn mailing list