[midPoint] 401 Unauthorized when clustering Midpoint

[Jacob Burley]

Hi all,

We're running into an issue with our Midpoint installation where our nodes
(both fresh installs of v4.7) don't seem to be able to communicate with one
another.

We provision two nodes using Ansible, and we generate a keystore on both
nodes. Presumably this means that the keystores themselves contain
differing encryption keys, although the keystore passphrases and encryption
key passphrases are the same.
We are using a Postgres server that is separate from the nodes, which both
nodes can communicate with.

These nodes live behind a load balancer with sticky sessions. They can
communicate with each other freely. On the first node (midpoint-0-dev is
serving the session in this scenario), whenever I go to
/midpoint/admin/nodes, I get the following entry in the logs:

2023-04-21 11:36:18,614 [TASK_MANAGER] [http-nio-8080-exec-4] WARN
> (com.evolveum.midpoint.task.quartzimpl.execution.remote.RestConnector):
> Querying remote scheduler information on midpoint-1-dev.cit.internal
> finished with status 401: Unauthorized


At the same time, on the midpoint-1-dev node, I see this in the
midpoint logs:

> 2023-04-21 11:36:18,580 [] [http-nio-8080-exec-5] INFO
> (com.evolveum.midpoint.authentication.impl.provider.ClusterProvider):
> Authentication failed for 127.0.0.1:
> web.security.flexAuth.cluster.auth.null
>

The error on midpoint-1-dev appears to be here
<https://github.com/Evolveum/midpoint/blob/f2892548a3f407f61c3132741f7fe3ae35d31365/model/authentication-impl/src/main/java/com/evolveum/midpoint/authentication/impl/provider/ClusterProvider.java#L58-L59>
in
the midpoint source for v4.7. As far as I can find, this isn't (directly)
related to anything I've configured.

What issues could this be indicative of? Do I need to make sure each node
has a copy of the other nodes key in its keystore? It's unclear to me how
the nodes are supposed to authenticate when communicating with each other.

Thanks,

Jacob Burley (he/him)

Endpoint System Engineer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20230421/dc410152/attachment.htm>