[midPoint-git] [Evolveum/midpoint] 15fc48: Added support to aggregate queries to return enums...

mederly noreply at github.com
Tue Oct 14 19:23:24 CEST 2025 

  Branch: refs/heads/support-4.9
  Home:   https://github.com/Evolveum/midpoint
  Commit: 15fc480de4d6b5540c460e934f710ae58325e7c4
      https://github.com/Evolveum/midpoint/commit/15fc480de4d6b5540c460e934f710ae58325e7c4
  Author: Tony Tkáčik <tonydamage at gmail.com>
  Date:   2025-10-13 (Mon, 13 Oct 2025)

  Changed paths:
    M repo/repo-sqale/src/main/java/com/evolveum/midpoint/repo/sqale/AggregateSearchContext.java

  Log Message:
  -----------
  Added support to aggregate queries to return enums and QNames

(cherry picked from commit 20f03d55f61925ef21819a3135502ed1d7afe1ea)


  Commit: c26f8d28b3969104b4aa0979784d87a7c5ee0ed0
      https://github.com/Evolveum/midpoint/commit/c26f8d28b3969104b4aa0979784d87a7c5ee0ed0
  Author: Tony Tkáčik <tonydamage at gmail.com>
  Date:   2025-10-13 (Mon, 13 Oct 2025)

  Changed paths:
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/AbstractRepositorySearchAction.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/CountRepositoryAction.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportConfigurationAction.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportFocusStatisticsAction.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportResourceSchemaAction.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportShadowStatisticsAction.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportShadowStatisticsOptions.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/worker/ExportConfigurationWorker.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/worker/ExportConsumerWorker.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/worker/ExportFocusStatisticsWorker.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/impl/Command.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/util/NinjaUtils.java

  Log Message:
  -----------
  Added export statistics action skeletons to Ninja

(cherry picked from commit df12803d7112acb5592c3966586e237a8b92c246)


  Commit: 5d690356d84ba545adc680c812fd8c9e59977198
      https://github.com/Evolveum/midpoint/commit/5d690356d84ba545adc680c812fd8c9e59977198
  Author: Andrej Zan <andrej.zan at evolveum.com>
  Date:   2025-10-13 (Mon, 13 Oct 2025)

  Changed paths:
    R tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportFocusStatisticsAction.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportShadowStatisticsAction.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/CategoriesDistribution.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/ExportFocusStatisticsAction.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/ExportFocusStatisticsWorker.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/FocusStats.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/FocusTypeCounter.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/FocusTypeStats.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/PrismTransformableRecord.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/PropertyStats.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/PropertyStatsCounter.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/StatsCounter.java
    R tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/worker/ExportFocusStatisticsWorker.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/impl/Command.java

  Log Message:
  -----------
  Implement export of focus statistics.

**What**

Implement calculation and export of various statistics related to focus
objects.

**Why**

This statistics can be very useful for future development of midPoint.

**Notes**

Current implementation does not consider complex structures inside of
focus objects. For example the `activation` is not currently counted to
any statistics, because it is a complex structure.

This version is not yet fully optimized in terms of memory usage.

(cherry picked from commit 6e10a2a7b7a894e5f669420063c957b813fad604)


  Commit: c4352dd9d9ac744017e7fc115393c8084fc99706
      https://github.com/Evolveum/midpoint/commit/c4352dd9d9ac744017e7fc115393c8084fc99706
  Author: Michal Zelencik <michal.zelencik at evolveum.com>
  Date:   2025-10-13 (Mon, 13 Oct 2025)

  Changed paths:
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportOptions.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportRepositoryAction.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/worker/ExportPerObjectWorker.java

  Log Message:
  -----------
  Add per-object export mode with type subdirectories and filenames

- Add ExportPerObjectWorker to write each object into a separate file
- Add --split-files (-sf) flag to ExportOptions

Route export to the appropriate worker in ExportRepositoryAction
- ExportPerObjectWorker when --split-files is enabled
- ExportConsumerWorker otherwise (existing single-file behavior)

Signed-off-by: Michal Zelencik <michal.zelencik at evolveum.com>

(cherry picked from commit b18aabe9f5a064632e82611e8877ec87914cc8d2)


  Commit: 6be5462b83a625c9900241898450333a68bdf5bf
      https://github.com/Evolveum/midpoint/commit/6be5462b83a625c9900241898450333a68bdf5bf
  Author: Michal Zelencik <michal.zelencik at evolveum.com>
  Date:   2025-10-13 (Mon, 13 Oct 2025)

  Changed paths:
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportConfigurationAction.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportRepositoryAction.java
    M tools/ninja/src/main/resources/messages.properties

  Log Message:
  -----------
  Fix: Move Split into files possibility from export repository to export
configuration
Fix: Add messages properties to be able to use --split-files flag

(cherry picked from commit 96e255541d40742de586b33f9c1d7b54a0b4993e)


  Commit: d1ccdc3f212a4b5107f6f09683165f8e61f926c0
      https://github.com/Evolveum/midpoint/commit/d1ccdc3f212a4b5107f6f09683165f8e61f926c0
  Author: Michal Zelencik <michal.zelencik at evolveum.com>
  Date:   2025-10-13 (Mon, 13 Oct 2025)

  Changed paths:
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportConfigurationAction.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/worker/ExportConfigurationSeparateWorker.java
    R tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/worker/ExportPerObjectWorker.java

  Log Message:
  -----------
  Remake configuration exporter to inherit from
ExportConfigurationSeparateWorker to include getExcludeItemsPaths()
and editObject() methods.

Signed-off-by: Michal Zelencik <michal.zelencik at evolveum.com>

(cherry picked from commit ef5e44fc7766d50b7e219de8d67ddeab4e948625)


  Commit: de8e21e10a7aa0d7e31c865e6c973b22cbe9a967
      https://github.com/Evolveum/midpoint/commit/de8e21e10a7aa0d7e31c865e6c973b22cbe9a967
  Author: Andrej Zan <andrej.zan at evolveum.com>
  Date:   2025-10-13 (Mon, 13 Oct 2025)

  Changed paths:
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/CategoriesDistribution.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/FocusTypeCounter.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/MagnitudeCounter.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/PropertyStats.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/PropertyStatsCounter.java

  Log Message:
  -----------
  Focus statistics improvements and changes

**What**

- Collect statistics also for properties in custom schema extensions.
- Change all counts to ratios (except objects count).
- Add new stats like ratio of multi valued properties.
- Use order of magnitudes for object counts instead of exact counts.
- Remove "floating" threshold for distribution threshold.
- Do not calculate cardinality if distribution is shown.
- Do not consider null values (basically missing property) as a
  "category" (each set of identical values is one category). That means:
  - Missing properties are not counted in cardinality.
  - Missing properties are not part of the distribution array.

**Why**

Most of the changes were done from compliance reasons. We do not want to
expose potentially sensitive statistics.

(cherry picked from commit 691c9ccf24e4486e3a83efc22bf6eaebf9a65c46)


  Commit: e5be041fc455862381fcc9bcac6b61ea740e6175
      https://github.com/Evolveum/midpoint/commit/e5be041fc455862381fcc9bcac6b61ea740e6175
  Author: Andrej Zan <andrej.zan at evolveum.com>
  Date:   2025-10-13 (Mon, 13 Oct 2025)

  Changed paths:
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/ExportFocusStatisticsAction.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/FocusTypeCounter.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/StatsCounter.java

  Log Message:
  -----------
  Support items exclusion in focus statistics export

**What**

Support the `-ei` option to exclude particular items from the
statistics.

**Why**

To allow user decide, what items are too much sensitive to count any
statistics on them.

(cherry picked from commit 13f773331164c6172947418c71682748a6aa6390)


  Commit: 8865ba7d6ca11899dd8996e2257cfa95ea7a1785
      https://github.com/Evolveum/midpoint/commit/8865ba7d6ca11899dd8996e2257cfa95ea7a1785
  Author: Andrej Zan <andrej.zan at evolveum.com>
  Date:   2025-10-13 (Mon, 13 Oct 2025)

  Changed paths:
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/ExportFocusStatisticsAction.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/FocusTypeCounter.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/StatsCounter.java

  Log Message:
  -----------
  Hard-code items allow list to Ninja's focus stats export

**What**

Add hard-coded list of items, which are by default included in focus
stats export.

**Why**

Not all items are interesting for our needs, and some items may be also
sensitive. Because of that we decided to include just some particular
items. Only these hard-coded items will be included in the export.

Items can still be explicitly excluded by user with the use of `-ei`
option.

Explicitly excluding items that are not by default included has no
effect, since they would not be exported anyway.

(cherry picked from commit eba3e89200b522559b7a88bfa5c7cb1407db267a)


  Commit: a4a0b95dcc1301f843188c3329f158a4afaadbc2
      https://github.com/Evolveum/midpoint/commit/a4a0b95dcc1301f843188c3329f158a4afaadbc2
  Author: Andrej Zan <andrej.zan at evolveum.com>
  Date:   2025-10-13 (Mon, 13 Oct 2025)

  Changed paths:
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/PropertyStatsCounter.java

  Log Message:
  -----------
  Add limit on size of map with properties values

**What**

Add a limit on a size of map which holds "categories" of properties
values across all objects of the same type.

**Why**

To lower potentially high memory usage.

(cherry picked from commit 820e7ed748b6a4c46e92c16c5caf884eb136ac06)


  Commit: 6468a58a6a177f2d258ed7c83502079614f23211
      https://github.com/Evolveum/midpoint/commit/6468a58a6a177f2d258ed7c83502079614f23211
  Author: Andrej Zan <andrej.zan at evolveum.com>
  Date:   2025-10-13 (Mon, 13 Oct 2025)

  Changed paths:
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportShadowStatisticsAction.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/ExportFocusStatisticsAction.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/MagnitudeCounter.java

  Log Message:
  -----------
  Change shadow statistics to approximations

**What**

Change the exact counts in shadow statistics to approximations in the
order of magnitude precision.

**Why**

Because of compliance reasons, we don't want to expose exact counts, as
they could be seen as sensitive data.

(cherry picked from commit 63c84cee815b5d9c35ad283b89bd2a655c7ed1db)


  Commit: f5dd564a1ab6355dca252c97fab26793e0eb02c8
      https://github.com/Evolveum/midpoint/commit/f5dd564a1ab6355dca252c97fab26793e0eb02c8
  Author: Andrej Zan <andrej.zan at evolveum.com>
  Date:   2025-10-13 (Mon, 13 Oct 2025)

  Changed paths:
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportShadowStatisticsAction.java

  Log Message:
  -----------
  Add synchronization situation to the shadow statistics

**What**

Instead of correlation situation use synchronization situation in
aggregate query.

(cherry picked from commit 70bec637bf1161f91cd886510629e13db0eb963b)


  Commit: 046c66e1a833c349d4e9224251a3e7a9eb81a850
      https://github.com/Evolveum/midpoint/commit/046c66e1a833c349d4e9224251a3e7a9eb81a850
  Author: Andrej Zan <andrej.zan at evolveum.com>
  Date:   2025-10-13 (Mon, 13 Oct 2025)

  Changed paths:
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportConfigurationAction.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/worker/ExportConfigurationSeparateWorker.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/worker/ExportConfigurationWorker.java

  Log Message:
  -----------
  Exclude of unnecessary items from configuration export

**What**

Remove items, which could be considered as sensitive or are not so
interesting.

**Why**

Mainly because of compliance.

(cherry picked from commit 9fb3cf974ed6446fba7d15df33f90785d9b337b6)


  Commit: 5974fad7da3d83376dd2999d224028424674b489
      https://github.com/Evolveum/midpoint/commit/5974fad7da3d83376dd2999d224028424674b489
  Author: Andrej Zan <andrej.zan at evolveum.com>
  Date:   2025-10-13 (Mon, 13 Oct 2025)

  Changed paths:
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/ExportFocusStatisticsAction.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/FocusTypeCounter.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/StatsCounter.java

  Log Message:
  -----------
  Fix inclusion of extensions in focus stats export

**What**

Fix the issue with unmatched extensions properties to include in focus
stats export.

**Note**

The main problem was, that the definition does not contain whole path,
but just the last segment (item name). When the include precondition was
applied, it did not matched any extension property, because the
precondition expected that the path will start with `extension/`. The
fix was to use `Item` instead of `ItemDefinition` in the precondition.
Item is aware of it's parents, so it contains whole path, not just item
name.

(cherry picked from commit df10f572fc2e3060b3641d8f8a3956c23887b1d1)


  Commit: 38e7dc8e7cd04fd681f1aa8754c0c1eb5f9231ac
      https://github.com/Evolveum/midpoint/commit/38e7dc8e7cd04fd681f1aa8754c0c1eb5f9231ac
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2025-10-14 (Tue, 14 Oct 2025)

  Changed paths:
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/worker/ExportConfigurationWorker.java

  Log Message:
  -----------
  Update exported configuration items

1. Resource schema is now correctly exported
2. Additional connector config is removed
3. AdminGuiConfiguration is removed
4. GUI-related things from ArchetypeType are removed
5. Irrelevant parts of ConnectorType are removed
6. Removal of protected values is fixed, plus we mask
protected value embedded deep in JAXB structures.

(cherry picked from commit 198418e5f188a4151fae5cb2d21800f2d65e90d3)


  Commit: 9cfbe2cd2872577ff67f6b552bc6a95900fa8910
      https://github.com/Evolveum/midpoint/commit/9cfbe2cd2872577ff67f6b552bc6a95900fa8910
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2025-10-14 (Tue, 14 Oct 2025)

  Changed paths:
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportResourceSchemaAction.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/worker/ExportConfigurationWorker.java

  Log Message:
  -----------
  Update exported configuration items

1. Added exclusion for extension (doesn't work now).
2. Added capabilities to export-schema.

(cherry picked from commit 44d1d6aee15c425f6c86d2c3b91fcaba2eb05f3f)


  Commit: a1a418a75ce7e2155391a8773ed985b32ed6f0f7
      https://github.com/Evolveum/midpoint/commit/a1a418a75ce7e2155391a8773ed985b32ed6f0f7
  Author: Andrej Zan <andrej.zan at evolveum.com>
  Date:   2025-10-14 (Tue, 14 Oct 2025)

  Changed paths:
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportResourceSchemaAction.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/worker/ExportConfigurationWorker.java

  Log Message:
  -----------
  Small changes without functionality impact

(cherry picked from commit e91ace6dd988831414d83abb0d8e7483ca265614)


  Commit: ec35fec177556c07038585155120c0921f60bbcd
      https://github.com/Evolveum/midpoint/commit/ec35fec177556c07038585155120c0921f60bbcd
  Author: Andrej Zan <andrej.zan at evolveum.com>
  Date:   2025-10-14 (Tue, 14 Oct 2025)

  Changed paths:
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportConfigurationAction.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportConfigurationOptions.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportResourceSchemaAction.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportResourceSchemaOptions.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/ExportShadowStatisticsOptions.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/ExportFocusStatisticsAction.java
    A tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/stats/ExportFocusStatisticsOptions.java
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/impl/Command.java
    M tools/ninja/src/main/resources/messages.properties

  Log Message:
  -----------
  Fix description text of few Ninja export commands

**What**

Add description to following export commands:

- export-configuration
- export-focus-statistics
- export-shadow-statistics
- export-resource-schema

(cherry picked from commit 3ee723b827a9acdcd6f20cbc73b9d028d54abd69)


  Commit: 5ca038fb53a31698a73b702379150839160edae7
      https://github.com/Evolveum/midpoint/commit/5ca038fb53a31698a73b702379150839160edae7
  Author: Pavol Mederly <mederly at evolveum.com>
  Date:   2025-10-14 (Tue, 14 Oct 2025)

  Changed paths:
    M tools/ninja/src/main/java/com/evolveum/midpoint/ninja/action/worker/ExportConfigurationWorker.java

  Log Message:
  -----------
  Avoid ObjectCleaner in ninja config export

This commit replaces ObjectCleaner by own logic when
removing unwanted items. The reason is that it does not
support abstract object types, so its configuration
would get too complex. Another reason is that we need
to backport this logic to support-4.8, where this
class does not exist.

Also fixed some minor bugs, like assuming all items
to be excluded are properties + now supporting item
paths with intermediate multivalued containers.

(cherry picked from commit 95f09e24cc6be9e672d284948af6109515f686cf)


Compare: https://github.com/Evolveum/midpoint/compare/8170cb51b135...5ca038fb53a3

To unsubscribe from these emails, change your notification settings at https://github.com/Evolveum/midpoint/settings/notifications

More information about the midPoint-svn mailing list