[midPoint] XML attributes (was: Re: Inbound mapping to custom extension "attributes")

[Radovan Semancik]

Hi Deepak,

I guess I have a bad news for you. But this is quite a long story, so 
let's start at the beginning.

When we have designed midPoint a couple of years ago we have built it 
quite tightly on top of XML. That was still the obvious choice at that 
time and also some kind of a best practice. And it was not a bad choice. 
Especially considering that alternatives such as JSON were still it 
their infancy ... and actually they haven't evolved a bit in all these 
years - but that's a different story. So, we have built midPoint on top 
of XML.

But the XML stroke back. I was not naive and I was aware that XML is not 
perfect when we started with midPoint. But I somehow expected that we 
can live with it at least for few years. I was wrong. XML and XSD and 
WSDL are terribly bad at handling dynamic schemas. I mean schemas that 
are only available at runtime. MidPoint schema extension is one example 
of such schema. But there are many more examples: resource schema, 
connector schema, reports, ...

Most other IDM system obviously deal with this problem by ignoring it. 
They do not support schema at all. But this was not the path that we 
wanted to take with midPoint. MidPoint is fully schema-based from the 
bottom (connectors) to the top (GUI). And there are huge advantages to 
this architecture. But there are also challenges. And dealing with XSD 
and especially Sun XML libraries was a huge challenge.

And that was the origin of "Prism Objects". We started to slowly replace 
XML libraries with a more generic data representation layer. See:
https://wiki.evolveum.com/display/midPoint/Prism+Objects

That was approximately the time when JSON became more popular than XML. 
And midPoint users started to ask about JSON support. I personally do 
not like JSON way of doing things. But we need to listen to user 
requests. And we have realized that we can easily support JSON and also 
other languages with our Prism layer. And I actually see an advantage in 
using readable languages such as YAML. Therefore we have chosen to go 
for language independence. XML is just one of possible data 
representation languages now. JSON and YAML will be available soon.

Currently midPoint (v3.0) is almost completely based on Prism. It is not 
using the XML layer directly perhaps except for several places that we 
plan to rewrite in next releases. This gives us the ability to represent 
data theoretically in any reasonable format. XML is the primary one. But 
it is not the only one.

And now it gets to the point of XML attributes. XML is a very strange 
format for data representation. It has nice features (such as safe 
extensibility with namespaces) and it has really bad features. 
Attributes are one of the worse features. Non-structured data item can 
be represented both as an attribute and as an sub-element. This creates 
a dichotomy that puzzled software engineers almost since XML was 
created. And it is a deadly trap for language-neutral data abstractions 
such as our Prism. Also languages such as JSON do not have attributes at 
all. And even though we try to have complete schema for everything there 
are some corner cases when we need to work without a schema. And then 
there may be problem whether to represent JSON key as XML attribute or 
sub-element.

Therefore we have decided to slowly phase out the support for XML 
attributes. MidPoint version 3.0 interprets attributes in the same way 
as sub-elements. And in fact these are interchangeable at many places in 
midPoint XML structures. And more places will appear in next versions. 
This is the plan for the entire 3.x generation. The 4.x versions will 
probably not use attributes at all.

So, the use of attributes is still somehow supported. But not 
recommended. And the way how you try to use the attributes as a map will 
not work in midPoint 3.x. It is not compatible with our data representation.

When it comes to your specific case I'm sure that the data can be 
represented in an alternative way without the use of attributes. Of 
course you can use this:
<group>
   <id>1</id>
   <value>a</value>
</group>
<group>
   <id>2</id>
   <value>b</value>
</group>

And there is another catch. As we are moving away from XML we also have 
to move away from XML-dependent mechanisms. Such as XPath. Therefore 
XPath has limited capabilities in midPoint 3.x. In fact if you define a 
schema then midPoint will not think about the data in XML terms. It will 
not see XML elements any more. It does not see the document, it sees the 
data. It will see the data structure as multi-value structured property 
"group". XPath may not be applicable here.

Therefore I guess this is still not the best way to represent your data. 
Maybe I could help you design the data structure if you describe the way 
how you plan to use the group data from the user object.

-- 

                                            Radovan Semancik
                                           Software Architect
                                              evolveum.com



On 05/26/2014 01:32 PM, Deepak Natarajan wrote:
> Hi everyone -
>
> I am trying to achieve the following inbound mapping :
>
> from resource :
> "groups" : "a_1,b_2,c_3,d_4"
>
> to
>
> <user>
>      <group id="1">a</group>
>      <group id="2">b</group>
>      <group id="3">c</group>
>    </user>
>
> My custom schema extension seems to be accepted by Midpoint. I have
> extended the UserType to support a multi-valued child element <group>
> with an "id" attribute (I have a GroupType defined in my custom schema).
>
> I can achieve the simple case, without the "id" attribute. So I can see
> in the Admin Console that the User has several groups, which I do with
> the following mapping :
>
> <attribute>
>          <ref>ri:groups</ref>
>          <inbound>
>              <expression>
>                  <script>
>                      <code>
>                           [parse input and return array of string [a, b,
> c...]
>                      </code>
>                  </script>
>              </expression>
>              <target>
>                  <path>$user/extension/my:group</path>
>              </target>
>          </inbound>
> </attribute>
>
> Now I'm trying to map attributes for the group element
> ($user/extension/my:group/@id)
>
> Has anyone tried this? Thanks for any input!
>
> BR/