[midPoint] Use in higher education?

Mon Feb 22 10:14:24 CET 2016

Hello Richard,

our experience at Comenius University is quite similar to yours - even 
if our home-made integration solution is much younger than yours 
(started in 2003).

Systems connected are: O365, AD, LDAP, Kerberos server, LMS, library, 
cards, SAP ERP+HR, ... (about 15-20 in total)
Primary technology is like yours: messaging (done via SonicMQ/SonicESB).
The overall size is comparable: 30K students, 4-5K full time employees, 
a couple of thousands part-timers.

Even the primary problem is similar: there are only two developers (~ 
1,5 FTE) that maintain the core of the solution (120 KSLOC primarily in 
Java and XSLT) + one developer that maintains sophisticated scripts that 
do O365/AD/LDAP/Kerberos provisioning. Obviously, this is not feasible 
in the long run.

So, we are undergoing partial or full migration to midPoint. [The choice 
of midPoint is perhaps also because both of us Java developers are 
working on midPoint as well :) Also, because recently midPoint got 
features that are crucial for our university environment, like delegated 
administration.] We do it gradually. First, we installed and integrated 
midPoint in order to provide some functionality that was needed and was 
not covered by the existing solution. Second, we plan to replace 
provisioning scripts by midPoint. And then we'll probably gradually 
migrate the core functionality to midPoint, adding maybe some advanced 
features not provided by existing solution (approval workflows, 
certification, ...).

As for your questions:

> How painful is it to have a large number of attributes on a user?
How many of them do you expect?

I see three views there:
1. Developer view - how to maintain the attributes. Extension attributes 
can be stored in one or more XSD files, so maintaining a long list of 
them should be no problem.
2. Performance view. Non-searchable extension attributes are stored 
within the XML clob column (as part of the whole object XML 
representation), so they do not impact performance directly, only if 
they would be excessively large. Searchable ones are stored as key-value 
pairs. But how many extension attributes you need to be searchable? So, 
generally, I don't think this should be a problem.
3. User experience view. The default page rendering is such that all 
extension attributes are present as a flat list. However, there is a 
mechanism for creating  custom tabs in GUI e.g. when editing a user. So 
I don't think this would be a problem either.

> As noted in the comparison, academia doesn't have clean organizational 
> units. 
What exactly do you mean? Our universities are divided into faculties 
(13 at Comenius), then to departments (~ 500 at Comenius). Each 
department belongs to one faculty. Each teacher belongs to one or more 
departments (if having part-time allocations). Each department 
participates on a couple study programmes (and most programmes are 
covered by more than one department, and even by more than one faculty), 
and on one or more research or other projects.

Because midPoint has a very flexible organization structure model 
(arbitrary directed acyclic graph), all of this can be easily modeled in it.

> Is there any sensible way to store something like course registrations 
> in the system? I'm guessing that I'll still need to manage that 
> separately for the systems that need that information and/or to make 
> decisions off of that information.
If you'd like to have simple list of courses a student is currently 
enrolled in, it can be done by a single multivalued string extension 
attribute. We at Comenius don't use even this. We are OK with 
maintaining study programme(s) one is currently enrolled in (again, as a 
multivalued string attribute).

If you'd like to provide more information (e.g. from/to dates, or any 
additional attributes), this might or might not be a problem. Maybe you 
could model this via org structure (e.g. along the path of department -> 
study programme -> course). If you need just to pass this information 
from source to destination systems, it could be maybe done by a 
non-searchable extension attribute.

> It's not immediately obvious to me, how hard is it to write a 
> connector that operates asynchronously?
This is really interesting question; how to match asynchronous nature of 
our existing system with basically synchronous operation of midPoint. 
I'm currently not sure how we (at Comenius) will deal with it. But there 
are plans to provide fully asynchronous operation in midPoint; Radovan 
will surely provide more specific information. Also, I thought of 
pushing the responsibility for "store-and-forward" functionality 
(currently provided by MQ) directly to midPoint. Its advanced 
consistency mechanism provides the retry functionality, with a bonus of 
having real-time overview of e.g. incomplete operations. (Although not 
available in GUI yet, it can be easily added there.)

But, overall, we plan to keep MQ for other integration tasks, not 
directly involving IdM - like distribution of card-related events from 
one system to another.

I hope that this provides you at least a partial information. I'm sure 
others would give you more details.

Best regards,
Pavol

On 18.02.2016 23:15, Richard Frovarp wrote:
> I am evaluating open source IAM solutions for use at my institution. 
> We are currently running a custom engineered solution, that while it 
> handles most of what we want, only a couple of us are familiar with 
> it. I'm hoping to get more capability, and better ease of use by 
> switching to an open source solution.
>
> We have around 40,000 active accounts, plus several thousand more for 
> another institution that we provide some IAM services for.
>
> Our current solution has evolved over the 20+ years since we first 
> started doing IDM. We tie pretty much everything we can into IAM. We 
> are doing the standard O365, Google Apps, and AD, plus of course our 
> LMS. We're also pushing data to the library system, the clinic, the 
> gym, and the system that lets students check out bikes for 30 minutes 
> at a time on campus and in town. That last one also has to pull card 
> data from the card system. Going the other way we provision printing 
> allocation to the card system each semester. I have vendors that need 
> flat files via scp or SFTP, or via web services. I have connectors 
> running in Java, Perl, Powershell, and Go. The current solution relies 
> on sending JSON messages via RabbitMQ, which works very well. If 
> RabbitMQ is down, the changes are attempted to be pushed at a later 
> time. If the end points are down, they just check in with RabbitMQ at 
> a later point. I also have some services such as Office 365, that when 
> it is provisioned, it triggers a modify in AD, and in Java which uses 
> Graph after a DirSync is ran. The Java then pushes off new messages 
> that are ran in PowerShell against mailboxes.
>
> My question is how much of my need can be met by midPoint? I 
> understand that it is a provisioning engine, which is what I'm after. 
> Our self service needs are unique enough that this functionality would 
> remain a separate system.
>
> How painful is it to have a large number of attributes on a user? I 
> see in the demo how a couple were added. To start tracking everything, 
> I'm going to need a lot more. As noted in the comparison, academia 
> doesn't have clean organizational units.
>
> Is there any sensible way to store something like course registrations 
> in the system? I'm guessing that I'll still need to manage that 
> separately for the systems that need that information and/or to make 
> decisions off of that information.
>
> It's not immediately obvious to me, how hard is it to write a 
> connector that operates asynchronously? The RabbitMQ model works quite 
> well for us, and it would be quickest and easiest to to move those 
> over as is until time permits new options to be developed. For some of 
> them doing PowerShell, there might not be good other options anyway.
>
> How does it handle username changes? The connectors would need to know 
> what the old username was, and what the new one will be so they can 
> find and update the username in the remote system.
>
> We also need to keep track of training / terms of service usage that 
> needs to be renewed. This then drives access to other systems. I'm 
> guessing that we could implement that via assignments with expiration 
> dates that are then used as dependencies for other services?
>
> Thank you in advance for any information that is provided. It looks 
> like a good product, it's just that higher education presents some 
> very unique challenges.
>
> Thanks,
>
> Richard
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com
> http://lists.evolveum.com/mailman/listinfo/midpoint