[midPoint] [Newsletter] Re: [Newsletter] Re: [Newsletter] Re: [Newsletter] Re: [Newsletter] Re: Migrating from 3.7.x to 3.8 with custom schema
Pavol Mederly
mederly at evolveum.com
Mon Jul 23 19:51:42 CEST 2018
Hello Chris,
the spelling difference is OK.
I have now used your XSD in my case and run the export. It is as it
should be:
<service
xmlns="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
xmlns:c="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
xmlns:icfs="http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/resource-schema-3"
xmlns:org="http://midpoint.evolveum.com/xml/ns/public/common/org-3"
xmlns:q="http://prism.evolveum.com/xml/ns/public/query-3"
xmlns:ri="http://midpoint.evolveum.com/xml/ns/public/resource/instance-3"
xmlns:t="http://prism.evolveum.com/xml/ns/public/types-3"
oid="bfb7ae2b-2473-46df-9c12-eb62103d3eca" version="5">
<name>Extension 123</name>
<extension
xmlns:gen633="http://sample.evolveum.com/xml/ns/sample-idm/extension">
<gen633:rsExtensionFirstName
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:type="xsd:string">Foo</gen633:rsExtensionFirstName>
<gen633:rsExtensionLastName
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:type="xsd:string">Bar</gen633:rsExtensionLastName>
<gen633:rsExtensionNumber
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:type="xsd:string">123</gen633:rsExtensionNumber>
<gen633:rsExtensionDecomissioned
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:type="xsd:boolean">false</gen633:rsExtensionDecomissioned>
</extension>
The file used to export is the one present in
https://evolveum.com/downloads/midpoint/3.7.2/midpoint-3.7.2-dist.zip
i.e.
ninja.jar, 73380648 bytes, SHA1 BA7B17AD64385ED098400A8454890316EDE413E4.
it is a bit different from ninja-3.7.2.jar from 3.8 distribution, which
is 73380649 bytes long with SHA1
27320463CE95EDFB40EDFC009CEBA89FAAD505AD (but the difference seems to be
only because it was built on a different time).
For me, both ninja JARs produce the correct result.
Are you really sure you are using one of these to generate the export?
Best regards,
Pavol Mederly
Software developer
evolveum.com
On 23.07.2018 16:22, Chris Woods wrote:
>
> Hi Pavol,
>
> no Problem. I noticed the spelling mistake in “SericeExtensionType”
> -> instead of ServiceExtensionType however. As far as I know this was
> in the PoC as well, though.
>
> Cheers,
>
> Chris.
>
> *From:*midPoint <midpoint-bounces at lists.evolveum.com> *On Behalf Of
> *Pavol Mederly
> *Sent:* Monday, July 23, 2018 12:06 PM
> *To:* midpoint at lists.evolveum.com
> *Subject:* *EXT* [Newsletter] Re: [midPoint] [Newsletter] Re:
> [Newsletter] Re: [Newsletter] Re: [Newsletter] Re: Migrating from
> 3.7.x to 3.8 with custom schema
>
> Hello Chris,
>
> this is strange indeed. I have tried to replicate your problem using
> freshly built midPoint 3.7.2:
>
> 1. Took a clean repository
> 2. Created an extension XSD file containing the 4 attributes
> (rsExtensionFirstName, rsExtensionLastName, rsExtensionNumber,
> rsExtensionDecommisioned)
> 3. Imported your service object (with xsi:type removed)
> 4. Stopped midPoint and exported the object using ninja from 3.7.2
>
> Got this - exactly as it should look like:
>
> <service
> xmlns="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
> <http://midpoint.evolveum.com/xml/ns/public/common/common-3>
> xmlns:c="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
> <http://midpoint.evolveum.com/xml/ns/public/common/common-3>
> xmlns:icfs="http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/resource-schema-3"
> <http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/resource-schema-3>
> xmlns:org="http://midpoint.evolveum.com/xml/ns/public/common/org-3"
> <http://midpoint.evolveum.com/xml/ns/public/common/org-3>
> xmlns:q="http://prism.evolveum.com/xml/ns/public/query-3"
> <http://prism.evolveum.com/xml/ns/public/query-3>
> xmlns:ri="http://midpoint.evolveum.com/xml/ns/public/resource/instance-3"
> <http://midpoint.evolveum.com/xml/ns/public/resource/instance-3>
> xmlns:t="http://prism.evolveum.com/xml/ns/public/types-3"
> <http://prism.evolveum.com/xml/ns/public/types-3>
> oid="bfb7ae2b-2473-46df-9c12-eb62103d3eca" version="5">
> <name>Extension 123</name>
> <extension
> xmlns:gen118="http://sample.evolveum.com/xml/ns/sample-idm/extension"
> <http://sample.evolveum.com/xml/ns/sample-idm/extension>>
> <gen118:rsExtensionFirstName
> xmlns:xsd="http://www.w3.org/2001/XMLSchema"
> <http://www.w3.org/2001/XMLSchema>
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> <http://www.w3.org/2001/XMLSchema-instance>
> xsi:type="xsd:string">Foo</gen118:rsExtensionFirstName>
> <gen118:rsExtensionLastName
> xmlns:xsd="http://www.w3.org/2001/XMLSchema"
> <http://www.w3.org/2001/XMLSchema>
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> <http://www.w3.org/2001/XMLSchema-instance>
> xsi:type="xsd:string">Bar</gen118:rsExtensionLastName>
> <gen118:rsExtensionNumber
> xmlns:xsd="http://www.w3.org/2001/XMLSchema"
> <http://www.w3.org/2001/XMLSchema>
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> <http://www.w3.org/2001/XMLSchema-instance>
> xsi:type="xsd:string">123</gen118:rsExtensionNumber>
> <gen118:rsExtensionDecomissioned
> xmlns:xsd="http://www.w3.org/2001/XMLSchema"
> <http://www.w3.org/2001/XMLSchema>
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> <http://www.w3.org/2001/XMLSchema-instance>
> xsi:type="xsd:boolean">false</gen118:rsExtensionDecomissioned>
> </extension>
>
> Maybe the difference is in the XSD file. Please, could you send it to
> me (maybe privately, if needed).
>
> Best regards,
>
> Pavol Mederly
> Software developer
> evolveum.com
>
>
> On 09.07.2018 7:35, Chris Woods wrote:
>
> Hi Pavol,
>
> same result, I’m afraid. I even tried with the ninja.jar from the
> 3.7.2 distribution (1 byte size difference).
>
> the resulting output is the same (except, of course for the genXXX)
>
> <service
> xmlns="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
> <http://midpoint.evolveum.com/xml/ns/public/common/common-3>
> xmlns:c="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
> <http://midpoint.evolveum.com/xml/ns/public/common/common-3>
> xmlns:icfs="http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/resource-schema-3"
> <http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/resource-schema-3>
> xmlns:org="http://midpoint.evolveum.com/xml/ns/public/common/org-3"
> <http://midpoint.evolveum.com/xml/ns/public/common/org-3>
> xmlns:q="http://prism.evolveum.com/xml/ns/public/query-3"
> <http://prism.evolveum.com/xml/ns/public/query-3>
> xmlns:ri="http://midpoint.evolveum.com/xml/ns/public/resource/instance-3"
> <http://midpoint.evolveum.com/xml/ns/public/resource/instance-3>
> xmlns:t="http://prism.evolveum.com/xml/ns/public/types-3"
> <http://prism.evolveum.com/xml/ns/public/types-3>
> oid="bfb7ae2b-2473-46df-9c12-eb62103d3eca" version="5">
>
> <name>Extension 123</name>
>
> <extension
> xmlns:gen802="http://sample.evolveum.com/xml/ns/sample-idm/extension"
> <http://sample.evolveum.com/xml/ns/sample-idm/extension>
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> <http://www.w3.org/2001/XMLSchema-instance>
> xsi:type="c:ExtensionType">
>
> <gen802:rsExtensionFirstName>Foo</gen802:rsExtensionFirstName>
>
> <gen802:rsExtensionLastName>Bar</gen802:rsExtensionLastName>
>
> <gen802:rsExtensionNumber>123</gen802:rsExtensionNumber>
>
> <gen802:rsExtensionDecomissioned>false</gen802:rsExtensionDecomissioned>
>
> </extension>
>
> Cheers,
>
> Chris.
>
> *From:*midPoint <midpoint-bounces at lists.evolveum.com>
> <mailto:midpoint-bounces at lists.evolveum.com> *On Behalf Of *Pavol
> Mederly
> *Sent:* Friday, July 6, 2018 2:51 PM
> *To:* midpoint at lists.evolveum.com <mailto:midpoint at lists.evolveum.com>
> *Subject:* *EXT* [Newsletter] Re: [midPoint] [Newsletter] Re:
> [Newsletter] Re: [Newsletter] Re: Migrating from 3.7.x to 3.8 with
> custom schema
>
> Chris,
>
> yes, please try that and let us know.
>
> Pavol Mederly
>
> Software developer
>
> evolveum.com
>
>
>
> On 06.07.2018 14:47, Chris Woods wrote:
>
> Hi Pavol,
>
> my ninja-3.7.2.jar is also 73380649 bytes with sha1sum
> 27320463ce95edfb40edfc009ceba89faad505ad. I still have the old
> database so it’s no problem to re-export and see if the new
> export is different.
>
> Cheers,
>
> Chris.
>
> *From:*midPoint <midpoint-bounces at lists.evolveum.com>
> <mailto:midpoint-bounces at lists.evolveum.com> *On Behalf Of
> *Pavol Mederly
> *Sent:* Friday, July 6, 2018 11:34 AM
> *To:* midpoint at lists.evolveum.com
> <mailto:midpoint at lists.evolveum.com>
> *Subject:* *EXT* [Newsletter] Re: [midPoint] [Newsletter] Re:
> [Newsletter] Re: Migrating from 3.7.x to 3.8 with custom schema
>
> Hello Chris,
>
> yes, this is exactly the same as in that other case.
>
> Which is really strange. Please, could you tell me which
> ninja-3.7.2.jar you used to do the export?
>
> Mine - which produces correct xsi:type information - has
> *73,380,649* bytes, is dated *13.6.2018 10:22*, and its sha1
> is *27320463CE95EDFB40EDFC009CEBA89FAAD505AD*.
>
> Just btw, it is possible to fix the export XML simply by
> deleting *xsi:type="c:ExtensionType"* string. But if you used
> wrong JAR, it is fairly possible that the export of attributes
> in shadows will be wrong (xsi:type is obligatory for their
> values).
>
> Best regards,
>
>
>
> Pavol Mederly
>
> Software developer
>
> evolveum.com
>
> On 06.07.2018 11:27, Chris Woods wrote:
>
> Hi Pavol,
>
>
>
> yes, no problem - there are quite a few objects that can't be imported, but here's a snippet from one of the services (Radovan should recognize it ;-): btw. the attribute is rsExtensionXXX because it is supposed to represent a telephone extension - and nothing to do with schema extension :-)
>
>
>
> <extension xmlns:gen774="http://sample.evolveum.com/xml/ns/sample-idm/extension"
> <http://sample.evolveum.com/xml/ns/sample-idm/extension> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> <http://www.w3.org/2001/XMLSchema-instance> xsi:type="c:ExtensionType">
>
> <gen774:rsExtensionFirstName>Foo</gen774:rsExtensionFirstName>
>
> <gen774:rsExtensionLastName>Bar</gen774:rsExtensionLastName>
>
> <gen774:rsExtensionNumber>123</gen774:rsExtensionNumber>
>
> <gen774:rsExtensionDecomissioned>false</gen774:rsExtensionDecomissioned>
>
> </extension>
>
>
>
> Regards,
>
> Chris.
>
>
>
> -----Original Message-----
>
> From: midPoint<midpoint-bounces at lists.evolveum.com>
> <mailto:midpoint-bounces at lists.evolveum.com> On Behalf Of Pavol Mederly
>
> Sent: Friday, July 6, 2018 8:40 AM
>
> To:midpoint at lists.evolveum.com
> <mailto:midpoint at lists.evolveum.com>
>
> Subject: *EXT* [Newsletter] Re: [midPoint] [Newsletter] Re: Migrating from 3.7.x to 3.8 with custom schema
>
>
>
> Hello Chris,
>
>
>
> I have seen such an error recently. In that particular case it was probably caused by doing the export by a different version of ninja-3.7, resulting in wrong placement of xsi:type declarations in resulting XML.
>
>
>
> Could you paste a snipped of <extension> element of an object that couldn't be imported in your case just to verify this?
>
>
>
> Best regards,
>
>
>
> Pavol Mederly
>
> Software developer
>
> evolveum.com
>
>
>
> On 05.07.2018 14:05, Wojciech Staszewski wrote:
>
> Hmmm... then I have no idea how to import this data. Maybe someone
>
> from Evolveum should reply.
>
> Regards!
>
> WS
>
>
>
> W dniu 05.07.2018 o 13:37, Chris Woods pisze:
>
> Hi Wojciech,
>
>
>
> I kept the installation directory the same (I used the upgrade.sh
>
> scripts from the 3.8 dist beforehand) so the schema extensions are
>
> still there. Originally I didn't start midpoint 3.8 before starting
>
> the import (there is now mention in the upgrade instructions that
>
> this is necessary). Even after starting midpoint and then retrying
>
> the import I still received the same errors.
>
>
>
> Cheers,
>
> Chris.
>
>
>
> -----Original Message-----
>
> From: midPoint<midpoint-bounces at lists.evolveum.com>
> <mailto:midpoint-bounces at lists.evolveum.com> On Behalf Of
>
> Wojciech Staszewski
>
> Sent: Thursday, July 5, 2018 1:23 PM
>
> To:midpoint at lists.evolveum.com
> <mailto:midpoint at lists.evolveum.com>
>
> Subject: *EXT* [Newsletter] Re: [midPoint] Migrating from 3.7.x to
>
> 3.8 with custom schema
>
>
>
> Is your custom schema xml copied to the v3.8 and was MidPoint
>
> restarted after that?
>
>
>
> W dniu 05.07.2018 o 12:22, Chris Woods pisze:
>
> Hi!
>
>
>
> I am currently trying to migrate our 3.7.2 installation (PoC) to
>
> 3.8. The export worked fine (after realizing I also needed to use –U
>
> with the jdbc url,not just the –j option for the jdbc driver). I
>
> also have my backup.zip containing all the objects – these were
>
> backed up from our existing MySQL 5.6 database.
>
>
>
> I am, however, restoring to a fresh MySQL v8.0 database (used the
>
> 3.8 create script to create the tables). The only problem seems to
>
> be with objects that have a value in an attribute from our custom
>
> schema – ninja skips those objects with the following Exception(s):
>
>
>
> com.evolveum.midpoint.util.exception.SchemaException: Item
>
> '{http://sample.evolveum.com/xml/ns/sample-idm/extension}rsCompany'
>
> without definition can't be saved.
>
>
>
> at
>
> com.evolveum.midpoint.repo.sql.helpers.ObjectUpdater.createDataObjec
>
> tF
>
> romJAXB(ObjectUpdater.java:604) ~[repo-sql-impl-3.8.jar!/:na]
>
>
>
> at
>
> com.evolveum.midpoint.repo.sql.helpers.ObjectUpdater.addObjectAttemp
>
> t(
>
> ObjectUpdater.java:131) ~[repo-sql-impl-3.8.jar!/:na]
>
>
>
> at
>
> com.evolveum.midpoint.repo.sql.SqlRepositoryServiceImpl.addObject(Sq
>
> lR
>
> epositoryServiceImpl.java:431) ~[repo-sql-impl-3.8.jar!/:na]
>
>
>
> at
>
> com.evolveum.midpoint.ninja.action.worker.ImportConsumerWorker.run(I
>
> mp
>
> ortConsumerWorker.java:65) ~[classes!/:na]
>
>
>
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor
>
> .j
>
> ava:1142) [na:1.8.0_131]
>
>
>
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
>
> java:617) [na:1.8.0_131]
>
>
>
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
>
>
>
> Caused by: java.lang.reflect.InvocationTargetException: null
>
>
>
> at sun.reflect.GeneratedMethodAccessor309.invoke(Unknown
>
> Source) ~[na:na]
>
>
>
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcce
>
> ss
>
> orImpl.java:43) ~[na:1.8.0_131]
>
>
>
> at java.lang.reflect.Method.invoke(Method.java:498)
>
> ~[na:1.8.0_131]
>
>
>
> at
>
> com.evolveum.midpoint.repo.sql.helpers.ObjectUpdater.createDataObjec
>
> tF
>
> romJAXB(ObjectUpdater.java:594) ~[repo-sql-impl-3.8.jar!/:na]
>
>
>
> ... 6 common frames omitted
>
>
>
> Caused by:
>
> com.evolveum.midpoint.repo.sql.util.DtoTranslationException: Item
>
> '{http://sample.evolveum.com/xml/ns/sample-idm/extension}rsCompany'
>
> without definition can't be saved.
>
>
>
> at
>
> com.evolveum.midpoint.repo.sql.data.common.RObject.copyFromJAXB(RObj
>
> ec
>
> t.java:760) ~[repo-sql-impl-3.8.jar!/:na]
>
>
>
> at
>
> com.evolveum.midpoint.repo.sql.data.common.RObject.copyFromJAXB(RObj
>
> ec
>
> t.java:726) ~[repo-sql-impl-3.8.jar!/:na]
>
>
>
> at
>
> com.evolveum.midpoint.repo.sql.data.common.RFocus.copyFromJAXB(RFocus.
>
> java:312) ~[repo-sql-impl-3.8.jar!/:na]
>
>
>
> at
>
> com.evolveum.midpoint.repo.sql.data.common.RUser.copyFromJAXB(RUser.
>
> ja
>
> va:248) ~[repo-sql-impl-3.8.jar!/:na]
>
>
>
> ... 10 common frames omitted
>
>
>
> Caused by: java.lang.IllegalArgumentException: Item
>
> '{http://sample.evolveum.com/xml/ns/sample-idm/extension}rsCompany'
>
> without definition can't be saved.
>
>
>
> at
>
> org.apache.commons.lang.Validate.notNull(Validate.java:192)
>
> ~[commons-lang-2.6.jar!/:2.6]
>
>
>
> at
>
> com.evolveum.midpoint.repo.sql.data.common.any.RAnyConverter.convert
>
> To
>
> RValue(RAnyConverter.java:168) ~[repo-sql-impl-3.8.jar!/:na]
>
>
>
> at
>
> com.evolveum.midpoint.repo.sql.data.common.RObject.copyFromJAXB(RObj
>
> ec
>
> t.java:752) ~[repo-sql-impl-3.8.jar!/:na]
>
>
>
> ... 13 common frames omitted
>
>
>
> Does anyone know what I’m doing wrong?
>
>
>
> Regards,
>
>
>
> Chris.
>
>
>
>
>
>
>
> _______________________________________________
>
> midPoint mailing list
>
> midPoint at lists.evolveum.com
> <mailto:midPoint at lists.evolveum.com>
>
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
>
>
>
>
> --
>
> Wojciech Staszewski
>
> Administrator Systemów Sieciowych
>
> tel. kom: 663 680 236
>
> www.diagnostyka.pl <http://www.diagnostyka.pl>
>
> Diagnostyka Sp. z o. o.
>
> ul. Prof. M. Życzkowskiego 16, 31-864 Kraków Numer KRS: 0000381559
>
> (Sąd Rejonowy dla Krakowa-Śródmieścia w Krakowie, XI Wydział
>
> Gospodarczy KRS)
>
> NIP: 675-12-65-009; REGON: 356366975
>
> Kapitał zakładowy: 33 756 500 zł.
>
>
>
> Pomyśl o środowisku zanim wydrukujesz ten e-mail.
>
> _______________________________________________
>
> midPoint mailing list
>
> midPoint at lists.evolveum.com
> <mailto:midPoint at lists.evolveum.com>
>
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
> _______________________________________________
>
> midPoint mailing list
>
> midPoint at lists.evolveum.com
> <mailto:midPoint at lists.evolveum.com>
>
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
>
>
> _______________________________________________
>
> midPoint mailing list
>
> midPoint at lists.evolveum.com
> <mailto:midPoint at lists.evolveum.com>
>
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
>
>
> _______________________________________________
>
> midPoint mailing list
>
> midPoint at lists.evolveum.com
> <mailto:midPoint at lists.evolveum.com>
>
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
> _______________________________________________
>
> midPoint mailing list
>
> midPoint at lists.evolveum.com
> <mailto:midPoint at lists.evolveum.com>
>
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
>
>
>
>
> _______________________________________________
>
> midPoint mailing list
>
> midPoint at lists.evolveum.com <mailto:midPoint at lists.evolveum.com>
>
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
>
>
>
> _______________________________________________
>
> midPoint mailing list
>
> midPoint at lists.evolveum.com <mailto:midPoint at lists.evolveum.com>
>
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
>
>
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com
> http://lists.evolveum.com/mailman/listinfo/midpoint
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20180723/a0f13dd1/attachment.htm>
More information about the midPoint
mailing list