[midPoint] Discussion: Future of ConnId

[Radovan Semancik]

Dear community,

As I have announced earlier 
(https://groups.google.com/d/msg/connid-dev/o3zsI40wFBA/_yQsWMIUBgAJ), I 
plan to initiate a discussion about future of ConnId. This is something 
that should not be taken lightly, therefore I'm starting to put together 
all the data that may be relevant for the discussion. I would like to 
share those data and my preliminary agenda with you before the actual 
discussions, so everybody may be properly prepared.

First of all, we have conducted a survey among midPoint users last year. 
There was a section of this survey that is related to ConnId. I have 
took the time to compile detailed results of this survey:

https://wiki.evolveum.com/display/midPoint/MidPoint+2019+Surver%3A+ConnId-related+results

It looks like most people have mixed feelings about ConnId. There is a 
lot of positive feelings, but also negative feelings. I can understand 
that and this is one of the purposes of those discussions. Also, it 
looks like most people would like to see support for complex attributes. 
This is something that is also a pain point for us. There are also other 
issues in ConnId.

My goal would be to think about ConnId 2.0. There are too many issues 
and good solutions to those issues would require major updates and, 
quite likely, also some incompatibilities.

My personal list of topics to discuss includes:

1. Schema and Identifiers

* Support complex attributes. Keep existing schema "language"? Or use 
something that is more suitable?
* Clean up the schema, e.g. make the old __NAME__ and __UID__ optional.
* Clean up the concept of identifiers. E.g. allow clean use of both 
entryUUID and DN at the same time.

2. Remote Connector Servers

Both Java and .NET connector servers are under-maintained (to be 
politically correct). We have been overlooking them for a long time. We 
have to do something with that. My proposal:
* Maintain Java server and gradually modernize it (e.g. improve logging, 
error handling, etc.)
* Drop .NET server. Ancient AD connectors based on .NET are long gone, 
therefore there seems to be very little incentive in maintaining the server.

3. API/SPI Operations

* Create new GetApiOp and GetOp, to split search and get. This should be 
more natural and it should reduce complexity in some connectors.
* Create new CountApiOp and CountOp for object counting. Very similar 
parameters to existing search methods.
* We have too many "update" operations now. UpdateDelta is clearly 
superior to all others. Do we still keep the old operations?

4. Result handlers

What to do with those pesky things? It looks like they just get into the 
way most of the time. We should at least make them disabled by default 
in ConnId 2.

5. Asynchronous operations

How to support operations that do not return immediately? E.g. 
operations that implement "manual provisioning"?

6. Misc little things

* Clarify definition of runAsUser, maybe rework the parameters to 
properly use identifiers
* Improve the documentation

7. Testing

* Does anyone know how the testing framework really works? Is it 
sufficient for our needs?
* How to include connector server in testing suite?

8. Low priority issues (not in scope)

Those are things that deserve some attention, but I would leave them for 
later:
Capabilities, versioning, error handling, synchronization improvements, 
service accounts, transactions, entitlements

And then the BIG QUESTION:

How do we go about this? Revolution or evolution? Do we re-write the 
code? We can get rid of that CDDL and replace it with a more reasonable 
license. We can also get rid of other legacy and troublesome parts. But 
it will be a huge amount of work. Or do we evolve existing code? That 
would mean that we will need to stick with CDDL pretty much forever. But 
it may be much more feasible approach.

Of course, this is not a plan that can be implemented in a couple of 
months. This will require a lot of resources and funding. It may take 
years to get there and I'm not promising any particular dates or 
deliveries here. I just want to make sure that we all have the same 
goal. That we can agree on the design. And then we can think about 
practical ways how to implement that.

I would like to make this a live discussion (video conference) rather 
than a mailing list thread because there are many options to consider. 
Especially for rewrite/evolve, schema and async questions. We need a bit 
of brainstorming there. I will summarize the results after the 
brainstorming so the community will not be kept in the dark.

I expect that the braninstorming wil take place in late January. I will 
agree on a date/time with Francesco directly as I do not want to bother 
the entire community with this. I will announce the dates in ConnId 
mailing list and everybody is welcomed to join. If someone really wants 
to make sure that the date suits them please let me know directly.

I'm cross-posting this to both ConnId and midPoint mailing lists, as I 
expect that this may be interesting for midPoint community as well. All 
further communication will be kept on ConnId mailing list, therefore 
anyone interested in those discussions should join/watch connid-dev 
mailing list.

-- 
Radovan Semancik
Software Architect
evolveum.com