[midPoint] [Newsletter] Re: [Newsletter] Re: [Newsletter] Re: [Newsletter] Re: Migrating from 3.7.x to 3.8 with custom schema
Pavol Mederly
mederly at evolveum.com
Mon Jul 23 12:06:17 CEST 2018
Hello Chris,
this is strange indeed. I have tried to replicate your problem using
freshly built midPoint 3.7.2:
1. Took a clean repository
2. Created an extension XSD file containing the 4 attributes
(rsExtensionFirstName, rsExtensionLastName, rsExtensionNumber,
rsExtensionDecommisioned)
3. Imported your service object (with xsi:type removed)
4. Stopped midPoint and exported the object using ninja from 3.7.2
Got this - exactly as it should look like:
<service
xmlns="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
xmlns:c="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
xmlns:icfs="http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/resource-schema-3"
xmlns:org="http://midpoint.evolveum.com/xml/ns/public/common/org-3"
xmlns:q="http://prism.evolveum.com/xml/ns/public/query-3"
xmlns:ri="http://midpoint.evolveum.com/xml/ns/public/resource/instance-3"
xmlns:t="http://prism.evolveum.com/xml/ns/public/types-3"
oid="bfb7ae2b-2473-46df-9c12-eb62103d3eca" version="5">
<name>Extension 123</name>
<extension
xmlns:gen118="http://sample.evolveum.com/xml/ns/sample-idm/extension">
<gen118:rsExtensionFirstName
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:type="xsd:string">Foo</gen118:rsExtensionFirstName>
<gen118:rsExtensionLastName
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:type="xsd:string">Bar</gen118:rsExtensionLastName>
<gen118:rsExtensionNumber
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:type="xsd:string">123</gen118:rsExtensionNumber>
<gen118:rsExtensionDecomissioned
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:type="xsd:boolean">false</gen118:rsExtensionDecomissioned>
</extension>
Maybe the difference is in the XSD file. Please, could you send it to me
(maybe privately, if needed).
Best regards,
Pavol Mederly
Software developer
evolveum.com
On 09.07.2018 7:35, Chris Woods wrote:
>
> Hi Pavol,
>
> same result, I’m afraid. I even tried with the ninja.jar from the
> 3.7.2 distribution (1 byte size difference).
>
> the resulting output is the same (except, of course for the genXXX)
>
> <service
> xmlns="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
> xmlns:c="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
> xmlns:icfs="http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/resource-schema-3"
> xmlns:org="http://midpoint.evolveum.com/xml/ns/public/common/org-3"
> xmlns:q="http://prism.evolveum.com/xml/ns/public/query-3"
> xmlns:ri="http://midpoint.evolveum.com/xml/ns/public/resource/instance-3"
> xmlns:t="http://prism.evolveum.com/xml/ns/public/types-3"
> oid="bfb7ae2b-2473-46df-9c12-eb62103d3eca" version="5">
>
> <name>Extension 123</name>
>
> <extension
> xmlns:gen802="http://sample.evolveum.com/xml/ns/sample-idm/extension"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:type="c:ExtensionType">
>
> <gen802:rsExtensionFirstName>Foo</gen802:rsExtensionFirstName>
>
> <gen802:rsExtensionLastName>Bar</gen802:rsExtensionLastName>
>
> <gen802:rsExtensionNumber>123</gen802:rsExtensionNumber>
>
> <gen802:rsExtensionDecomissioned>false</gen802:rsExtensionDecomissioned>
>
> </extension>
>
> Cheers,
>
> Chris.
>
> *From:*midPoint <midpoint-bounces at lists.evolveum.com> *On Behalf Of
> *Pavol Mederly
> *Sent:* Friday, July 6, 2018 2:51 PM
> *To:* midpoint at lists.evolveum.com
> *Subject:* *EXT* [Newsletter] Re: [midPoint] [Newsletter] Re:
> [Newsletter] Re: [Newsletter] Re: Migrating from 3.7.x to 3.8 with
> custom schema
>
> Chris,
>
> yes, please try that and let us know.
>
> Pavol Mederly
> Software developer
> evolveum.com
>
>
> On 06.07.2018 14:47, Chris Woods wrote:
>
> Hi Pavol,
>
> my ninja-3.7.2.jar is also 73380649 bytes with sha1sum
> 27320463ce95edfb40edfc009ceba89faad505ad. I still have the old
> database so it’s no problem to re-export and see if the new export
> is different.
>
> Cheers,
>
> Chris.
>
> *From:*midPoint <midpoint-bounces at lists.evolveum.com>
> <mailto:midpoint-bounces at lists.evolveum.com> *On Behalf Of *Pavol
> Mederly
> *Sent:* Friday, July 6, 2018 11:34 AM
> *To:* midpoint at lists.evolveum.com <mailto:midpoint at lists.evolveum.com>
> *Subject:* *EXT* [Newsletter] Re: [midPoint] [Newsletter] Re:
> [Newsletter] Re: Migrating from 3.7.x to 3.8 with custom schema
>
> Hello Chris,
>
> yes, this is exactly the same as in that other case.
>
> Which is really strange. Please, could you tell me which
> ninja-3.7.2.jar you used to do the export?
>
> Mine - which produces correct xsi:type information - has
> *73,380,649* bytes, is dated *13.6.2018 10:22*, and its sha1 is
> *27320463CE95EDFB40EDFC009CEBA89FAAD505AD*.
>
> Just btw, it is possible to fix the export XML simply by deleting
> *xsi:type="c:ExtensionType"* string. But if you used wrong JAR, it
> is fairly possible that the export of attributes in shadows will
> be wrong (xsi:type is obligatory for their values).
>
> Best regards,
>
>
> Pavol Mederly
>
> Software developer
>
> evolveum.com
>
> On 06.07.2018 11:27, Chris Woods wrote:
>
> Hi Pavol,
>
>
>
> yes, no problem - there are quite a few objects that can't be imported, but here's a snippet from one of the services (Radovan should recognize it ;-): btw. the attribute is rsExtensionXXX because it is supposed to represent a telephone extension - and nothing to do with schema extension :-)
>
>
>
> <extension xmlns:gen774="http://sample.evolveum.com/xml/ns/sample-idm/extension"
> <http://sample.evolveum.com/xml/ns/sample-idm/extension> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> <http://www.w3.org/2001/XMLSchema-instance> xsi:type="c:ExtensionType">
>
> <gen774:rsExtensionFirstName>Foo</gen774:rsExtensionFirstName>
>
> <gen774:rsExtensionLastName>Bar</gen774:rsExtensionLastName>
>
> <gen774:rsExtensionNumber>123</gen774:rsExtensionNumber>
>
> <gen774:rsExtensionDecomissioned>false</gen774:rsExtensionDecomissioned>
>
> </extension>
>
>
>
> Regards,
>
> Chris.
>
>
>
> -----Original Message-----
>
> From: midPoint<midpoint-bounces at lists.evolveum.com>
> <mailto:midpoint-bounces at lists.evolveum.com> On Behalf Of Pavol Mederly
>
> Sent: Friday, July 6, 2018 8:40 AM
>
> To:midpoint at lists.evolveum.com <mailto:midpoint at lists.evolveum.com>
>
> Subject: *EXT* [Newsletter] Re: [midPoint] [Newsletter] Re: Migrating from 3.7.x to 3.8 with custom schema
>
>
>
> Hello Chris,
>
>
>
> I have seen such an error recently. In that particular case it was probably caused by doing the export by a different version of ninja-3.7, resulting in wrong placement of xsi:type declarations in resulting XML.
>
>
>
> Could you paste a snipped of <extension> element of an object that couldn't be imported in your case just to verify this?
>
>
>
> Best regards,
>
>
>
> Pavol Mederly
>
> Software developer
>
> evolveum.com
>
>
>
> On 05.07.2018 14:05, Wojciech Staszewski wrote:
>
> Hmmm... then I have no idea how to import this data. Maybe someone
>
> from Evolveum should reply.
>
> Regards!
>
> WS
>
>
>
> W dniu 05.07.2018 o 13:37, Chris Woods pisze:
>
> Hi Wojciech,
>
>
>
> I kept the installation directory the same (I used the upgrade.sh
>
> scripts from the 3.8 dist beforehand) so the schema extensions are
>
> still there. Originally I didn't start midpoint 3.8 before starting
>
> the import (there is now mention in the upgrade instructions that
>
> this is necessary). Even after starting midpoint and then retrying
>
> the import I still received the same errors.
>
>
>
> Cheers,
>
> Chris.
>
>
>
> -----Original Message-----
>
> From: midPoint<midpoint-bounces at lists.evolveum.com>
> <mailto:midpoint-bounces at lists.evolveum.com> On Behalf Of
>
> Wojciech Staszewski
>
> Sent: Thursday, July 5, 2018 1:23 PM
>
> To:midpoint at lists.evolveum.com
> <mailto:midpoint at lists.evolveum.com>
>
> Subject: *EXT* [Newsletter] Re: [midPoint] Migrating from 3.7.x to
>
> 3.8 with custom schema
>
>
>
> Is your custom schema xml copied to the v3.8 and was MidPoint
>
> restarted after that?
>
>
>
> W dniu 05.07.2018 o 12:22, Chris Woods pisze:
>
> Hi!
>
>
>
> I am currently trying to migrate our 3.7.2 installation (PoC) to
>
> 3.8. The export worked fine (after realizing I also needed to use –U
>
> with the jdbc url,not just the –j option for the jdbc driver). I
>
> also have my backup.zip containing all the objects – these were
>
> backed up from our existing MySQL 5.6 database.
>
>
>
> I am, however, restoring to a fresh MySQL v8.0 database (used the
>
> 3.8 create script to create the tables). The only problem seems to
>
> be with objects that have a value in an attribute from our custom
>
> schema – ninja skips those objects with the following Exception(s):
>
>
>
> com.evolveum.midpoint.util.exception.SchemaException: Item
>
> '{http://sample.evolveum.com/xml/ns/sample-idm/extension}rsCompany'
>
> without definition can't be saved.
>
>
>
> at
>
> com.evolveum.midpoint.repo.sql.helpers.ObjectUpdater.createDataObjec
>
> tF
>
> romJAXB(ObjectUpdater.java:604) ~[repo-sql-impl-3.8.jar!/:na]
>
>
>
> at
>
> com.evolveum.midpoint.repo.sql.helpers.ObjectUpdater.addObjectAttemp
>
> t(
>
> ObjectUpdater.java:131) ~[repo-sql-impl-3.8.jar!/:na]
>
>
>
> at
>
> com.evolveum.midpoint.repo.sql.SqlRepositoryServiceImpl.addObject(Sq
>
> lR
>
> epositoryServiceImpl.java:431) ~[repo-sql-impl-3.8.jar!/:na]
>
>
>
> at
>
> com.evolveum.midpoint.ninja.action.worker.ImportConsumerWorker.run(I
>
> mp
>
> ortConsumerWorker.java:65) ~[classes!/:na]
>
>
>
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor
>
> .j
>
> ava:1142) [na:1.8.0_131]
>
>
>
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
>
> java:617) [na:1.8.0_131]
>
>
>
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
>
>
>
> Caused by: java.lang.reflect.InvocationTargetException: null
>
>
>
> at sun.reflect.GeneratedMethodAccessor309.invoke(Unknown
>
> Source) ~[na:na]
>
>
>
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcce
>
> ss
>
> orImpl.java:43) ~[na:1.8.0_131]
>
>
>
> at java.lang.reflect.Method.invoke(Method.java:498)
>
> ~[na:1.8.0_131]
>
>
>
> at
>
> com.evolveum.midpoint.repo.sql.helpers.ObjectUpdater.createDataObjec
>
> tF
>
> romJAXB(ObjectUpdater.java:594) ~[repo-sql-impl-3.8.jar!/:na]
>
>
>
> ... 6 common frames omitted
>
>
>
> Caused by:
>
> com.evolveum.midpoint.repo.sql.util.DtoTranslationException: Item
>
> '{http://sample.evolveum.com/xml/ns/sample-idm/extension}rsCompany'
>
> without definition can't be saved.
>
>
>
> at
>
> com.evolveum.midpoint.repo.sql.data.common.RObject.copyFromJAXB(RObj
>
> ec
>
> t.java:760) ~[repo-sql-impl-3.8.jar!/:na]
>
>
>
> at
>
> com.evolveum.midpoint.repo.sql.data.common.RObject.copyFromJAXB(RObj
>
> ec
>
> t.java:726) ~[repo-sql-impl-3.8.jar!/:na]
>
>
>
> at
>
> com.evolveum.midpoint.repo.sql.data.common.RFocus.copyFromJAXB(RFocus.
>
> java:312) ~[repo-sql-impl-3.8.jar!/:na]
>
>
>
> at
>
> com.evolveum.midpoint.repo.sql.data.common.RUser.copyFromJAXB(RUser.
>
> ja
>
> va:248) ~[repo-sql-impl-3.8.jar!/:na]
>
>
>
> ... 10 common frames omitted
>
>
>
> Caused by: java.lang.IllegalArgumentException: Item
>
> '{http://sample.evolveum.com/xml/ns/sample-idm/extension}rsCompany'
>
> without definition can't be saved.
>
>
>
> at
>
> org.apache.commons.lang.Validate.notNull(Validate.java:192)
>
> ~[commons-lang-2.6.jar!/:2.6]
>
>
>
> at
>
> com.evolveum.midpoint.repo.sql.data.common.any.RAnyConverter.convert
>
> To
>
> RValue(RAnyConverter.java:168) ~[repo-sql-impl-3.8.jar!/:na]
>
>
>
> at
>
> com.evolveum.midpoint.repo.sql.data.common.RObject.copyFromJAXB(RObj
>
> ec
>
> t.java:752) ~[repo-sql-impl-3.8.jar!/:na]
>
>
>
> ... 13 common frames omitted
>
>
>
> Does anyone know what I’m doing wrong?
>
>
>
> Regards,
>
>
>
> Chris.
>
>
>
>
>
>
>
> _______________________________________________
>
> midPoint mailing list
>
> midPoint at lists.evolveum.com
> <mailto:midPoint at lists.evolveum.com>
>
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
>
>
>
>
> --
>
> Wojciech Staszewski
>
> Administrator Systemów Sieciowych
>
> tel. kom: 663 680 236
>
> www.diagnostyka.pl <http://www.diagnostyka.pl>
>
> Diagnostyka Sp. z o. o.
>
> ul. Prof. M. Życzkowskiego 16, 31-864 Kraków Numer KRS: 0000381559
>
> (Sąd Rejonowy dla Krakowa-Śródmieścia w Krakowie, XI Wydział
>
> Gospodarczy KRS)
>
> NIP: 675-12-65-009; REGON: 356366975
>
> Kapitał zakładowy: 33 756 500 zł.
>
>
>
> Pomyśl o środowisku zanim wydrukujesz ten e-mail.
>
> _______________________________________________
>
> midPoint mailing list
>
> midPoint at lists.evolveum.com
> <mailto:midPoint at lists.evolveum.com>
>
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
> _______________________________________________
>
> midPoint mailing list
>
> midPoint at lists.evolveum.com
> <mailto:midPoint at lists.evolveum.com>
>
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
>
>
> _______________________________________________
>
> midPoint mailing list
>
> midPoint at lists.evolveum.com
> <mailto:midPoint at lists.evolveum.com>
>
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
>
>
> _______________________________________________
>
> midPoint mailing list
>
> midPoint at lists.evolveum.com <mailto:midPoint at lists.evolveum.com>
>
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
> _______________________________________________
>
> midPoint mailing list
>
> midPoint at lists.evolveum.com <mailto:midPoint at lists.evolveum.com>
>
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
>
>
>
> _______________________________________________
>
> midPoint mailing list
>
> midPoint at lists.evolveum.com <mailto:midPoint at lists.evolveum.com>
>
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
>
>
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com
> http://lists.evolveum.com/mailman/listinfo/midpoint
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20180723/829922f0/attachment.htm>
More information about the midPoint
mailing list