[midPoint] [Newsletter] Re: [Newsletter] Re: [Newsletter] Re: [Newsletter] Re: [Newsletter] Re: [Newsletter] Re: Migrating from 3.7.x to 3.8 with custom schema
Chris Woods
Chris.Woods at rohde-schwarz.com
Tue Jul 31 13:41:44 CEST 2018
Hi Pavol,
pretty sure. I’ll have another attempt when I get back from holiday. Thanks for checking – it’s only our PoC environment anyway, so if the worst comes to the worst I can reimport everything from the resources.
Cheers,
Chris.
From: midPoint <midpoint-bounces at lists.evolveum.com> On Behalf Of Pavol Mederly
Sent: Monday, July 23, 2018 7:52 PM
To: midpoint at lists.evolveum.com
Subject: *EXT* [Newsletter] Re: [midPoint] [Newsletter] Re: [Newsletter] Re: [Newsletter] Re: [Newsletter] Re: [Newsletter] Re: Migrating from 3.7.x to 3.8 with custom schema
Hello Chris,
the spelling difference is OK.
I have now used your XSD in my case and run the export. It is as it should be:
<service xmlns="http://midpoint.evolveum.com/xml/ns/public/common/common-3"<http://midpoint.evolveum.com/xml/ns/public/common/common-3> xmlns:c="http://midpoint.evolveum.com/xml/ns/public/common/common-3"<http://midpoint.evolveum.com/xml/ns/public/common/common-3> xmlns:icfs="http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/resource-schema-3"<http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/resource-schema-3> xmlns:org="http://midpoint.evolveum.com/xml/ns/public/common/org-3"<http://midpoint.evolveum.com/xml/ns/public/common/org-3> xmlns:q="http://prism.evolveum.com/xml/ns/public/query-3"<http://prism.evolveum.com/xml/ns/public/query-3> xmlns:ri="http://midpoint.evolveum.com/xml/ns/public/resource/instance-3"<http://midpoint.evolveum.com/xml/ns/public/resource/instance-3> xmlns:t="http://prism.evolveum.com/xml/ns/public/types-3"<http://prism.evolveum.com/xml/ns/public/types-3> oid="bfb7ae2b-2473-46df-9c12-eb62103d3eca" version="5">
<name>Extension 123</name>
<extension xmlns:gen633="http://sample.evolveum.com/xml/ns/sample-idm/extension"<http://sample.evolveum.com/xml/ns/sample-idm/extension>>
<gen633:rsExtensionFirstName xmlns:xsd="http://www.w3.org/2001/XMLSchema"<http://www.w3.org/2001/XMLSchema> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<http://www.w3.org/2001/XMLSchema-instance> xsi:type="xsd:string">Foo</gen633:rsExtensionFirstName>
<gen633:rsExtensionLastName xmlns:xsd="http://www.w3.org/2001/XMLSchema"<http://www.w3.org/2001/XMLSchema> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<http://www.w3.org/2001/XMLSchema-instance> xsi:type="xsd:string">Bar</gen633:rsExtensionLastName>
<gen633:rsExtensionNumber xmlns:xsd="http://www.w3.org/2001/XMLSchema"<http://www.w3.org/2001/XMLSchema> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<http://www.w3.org/2001/XMLSchema-instance> xsi:type="xsd:string">123</gen633:rsExtensionNumber>
<gen633:rsExtensionDecomissioned xmlns:xsd="http://www.w3.org/2001/XMLSchema"<http://www.w3.org/2001/XMLSchema> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<http://www.w3.org/2001/XMLSchema-instance> xsi:type="xsd:boolean">false</gen633:rsExtensionDecomissioned>
</extension>
The file used to export is the one present in https://evolveum.com/downloads/midpoint/3.7.2/midpoint-3.7.2-dist.zip
i.e.
ninja.jar, 73380648 bytes, SHA1 BA7B17AD64385ED098400A8454890316EDE413E4.
it is a bit different from ninja-3.7.2.jar from 3.8 distribution, which is 73380649 bytes long with SHA1 27320463CE95EDFB40EDFC009CEBA89FAAD505AD (but the difference seems to be only because it was built on a different time).
For me, both ninja JARs produce the correct result.
Are you really sure you are using one of these to generate the export?
Best regards,
Pavol Mederly
Software developer
evolveum.com
On 23.07.2018 16:22, Chris Woods wrote:
Hi Pavol,
no Problem. I noticed the spelling mistake in “SericeExtensionType” -> instead of ServiceExtensionType however. As far as I know this was in the PoC as well, though.
Cheers,
Chris.
From: midPoint <midpoint-bounces at lists.evolveum.com><mailto:midpoint-bounces at lists.evolveum.com> On Behalf Of Pavol Mederly
Sent: Monday, July 23, 2018 12:06 PM
To: midpoint at lists.evolveum.com<mailto:midpoint at lists.evolveum.com>
Subject: *EXT* [Newsletter] Re: [midPoint] [Newsletter] Re: [Newsletter] Re: [Newsletter] Re: [Newsletter] Re: Migrating from 3.7.x to 3.8 with custom schema
Hello Chris,
this is strange indeed. I have tried to replicate your problem using freshly built midPoint 3.7.2:
1. Took a clean repository
2. Created an extension XSD file containing the 4 attributes (rsExtensionFirstName, rsExtensionLastName, rsExtensionNumber, rsExtensionDecommisioned)
3. Imported your service object (with xsi:type removed)
4. Stopped midPoint and exported the object using ninja from 3.7.2
Got this - exactly as it should look like:
<service xmlns="http://midpoint.evolveum.com/xml/ns/public/common/common-3"<http://midpoint.evolveum.com/xml/ns/public/common/common-3> xmlns:c="http://midpoint.evolveum.com/xml/ns/public/common/common-3"<http://midpoint.evolveum.com/xml/ns/public/common/common-3> xmlns:icfs="http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/resource-schema-3"<http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/resource-schema-3> xmlns:org="http://midpoint.evolveum.com/xml/ns/public/common/org-3"<http://midpoint.evolveum.com/xml/ns/public/common/org-3> xmlns:q="http://prism.evolveum.com/xml/ns/public/query-3"<http://prism.evolveum.com/xml/ns/public/query-3> xmlns:ri="http://midpoint.evolveum.com/xml/ns/public/resource/instance-3"<http://midpoint.evolveum.com/xml/ns/public/resource/instance-3> xmlns:t="http://prism.evolveum.com/xml/ns/public/types-3"<http://prism.evolveum.com/xml/ns/public/types-3> oid="bfb7ae2b-2473-46df-9c12-eb62103d3eca" version="5">
<name>Extension 123</name>
<extension xmlns:gen118="http://sample.evolveum.com/xml/ns/sample-idm/extension"<http://sample.evolveum.com/xml/ns/sample-idm/extension>>
<gen118:rsExtensionFirstName xmlns:xsd="http://www.w3.org/2001/XMLSchema"<http://www.w3.org/2001/XMLSchema> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<http://www.w3.org/2001/XMLSchema-instance> xsi:type="xsd:string">Foo</gen118:rsExtensionFirstName>
<gen118:rsExtensionLastName xmlns:xsd="http://www.w3.org/2001/XMLSchema"<http://www.w3.org/2001/XMLSchema> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<http://www.w3.org/2001/XMLSchema-instance> xsi:type="xsd:string">Bar</gen118:rsExtensionLastName>
<gen118:rsExtensionNumber xmlns:xsd="http://www.w3.org/2001/XMLSchema"<http://www.w3.org/2001/XMLSchema> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<http://www.w3.org/2001/XMLSchema-instance> xsi:type="xsd:string">123</gen118:rsExtensionNumber>
<gen118:rsExtensionDecomissioned xmlns:xsd="http://www.w3.org/2001/XMLSchema"<http://www.w3.org/2001/XMLSchema> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<http://www.w3.org/2001/XMLSchema-instance> xsi:type="xsd:boolean">false</gen118:rsExtensionDecomissioned>
</extension>
Maybe the difference is in the XSD file. Please, could you send it to me (maybe privately, if needed).
Best regards,
Pavol Mederly
Software developer
evolveum.com
On 09.07.2018 7:35, Chris Woods wrote:
Hi Pavol,
same result, I’m afraid. I even tried with the ninja.jar from the 3.7.2 distribution (1 byte size difference).
the resulting output is the same (except, of course for the genXXX)
<service xmlns="http://midpoint.evolveum.com/xml/ns/public/common/common-3"<http://midpoint.evolveum.com/xml/ns/public/common/common-3> xmlns:c="http://midpoint.evolveum.com/xml/ns/public/common/common-3"<http://midpoint.evolveum.com/xml/ns/public/common/common-3> xmlns:icfs="http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/resource-schema-3"<http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/resource-schema-3> xmlns:org="http://midpoint.evolveum.com/xml/ns/public/common/org-3"<http://midpoint.evolveum.com/xml/ns/public/common/org-3> xmlns:q="http://prism.evolveum.com/xml/ns/public/query-3"<http://prism.evolveum.com/xml/ns/public/query-3> xmlns:ri="http://midpoint.evolveum.com/xml/ns/public/resource/instance-3"<http://midpoint.evolveum.com/xml/ns/public/resource/instance-3> xmlns:t="http://prism.evolveum.com/xml/ns/public/types-3"<http://prism.evolveum.com/xml/ns/public/types-3> oid="bfb7ae2b-2473-46df-9c12-eb62103d3eca" version="5">
<name>Extension 123</name>
<extension xmlns:gen802="http://sample.evolveum.com/xml/ns/sample-idm/extension"<http://sample.evolveum.com/xml/ns/sample-idm/extension> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<http://www.w3.org/2001/XMLSchema-instance> xsi:type="c:ExtensionType">
<gen802:rsExtensionFirstName>Foo</gen802:rsExtensionFirstName>
<gen802:rsExtensionLastName>Bar</gen802:rsExtensionLastName>
<gen802:rsExtensionNumber>123</gen802:rsExtensionNumber>
<gen802:rsExtensionDecomissioned>false</gen802:rsExtensionDecomissioned>
</extension>
Cheers,
Chris.
From: midPoint <midpoint-bounces at lists.evolveum.com><mailto:midpoint-bounces at lists.evolveum.com> On Behalf Of Pavol Mederly
Sent: Friday, July 6, 2018 2:51 PM
To: midpoint at lists.evolveum.com<mailto:midpoint at lists.evolveum.com>
Subject: *EXT* [Newsletter] Re: [midPoint] [Newsletter] Re: [Newsletter] Re: [Newsletter] Re: Migrating from 3.7.x to 3.8 with custom schema
Chris,
yes, please try that and let us know.
Pavol Mederly
Software developer
evolveum.com
On 06.07.2018 14:47, Chris Woods wrote:
Hi Pavol,
my ninja-3.7.2.jar is also 73380649 bytes with sha1sum 27320463ce95edfb40edfc009ceba89faad505ad. I still have the old database so it’s no problem to re-export and see if the new export is different.
Cheers,
Chris.
From: midPoint <midpoint-bounces at lists.evolveum.com><mailto:midpoint-bounces at lists.evolveum.com> On Behalf Of Pavol Mederly
Sent: Friday, July 6, 2018 11:34 AM
To: midpoint at lists.evolveum.com<mailto:midpoint at lists.evolveum.com>
Subject: *EXT* [Newsletter] Re: [midPoint] [Newsletter] Re: [Newsletter] Re: Migrating from 3.7.x to 3.8 with custom schema
Hello Chris,
yes, this is exactly the same as in that other case.
Which is really strange. Please, could you tell me which ninja-3.7.2.jar you used to do the export?
Mine - which produces correct xsi:type information - has 73,380,649 bytes, is dated 13.6.2018 10:22, and its sha1 is 27320463CE95EDFB40EDFC009CEBA89FAAD505AD.
Just btw, it is possible to fix the export XML simply by deleting xsi:type="c:ExtensionType" string. But if you used wrong JAR, it is fairly possible that the export of attributes in shadows will be wrong (xsi:type is obligatory for their values).
Best regards,
Pavol Mederly
Software developer
evolveum.com
On 06.07.2018 11:27, Chris Woods wrote:
Hi Pavol,
yes, no problem - there are quite a few objects that can't be imported, but here's a snippet from one of the services (Radovan should recognize it ;-): btw. the attribute is rsExtensionXXX because it is supposed to represent a telephone extension - and nothing to do with schema extension :-)
<extension xmlns:gen774="http://sample.evolveum.com/xml/ns/sample-idm/extension"<http://sample.evolveum.com/xml/ns/sample-idm/extension> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<http://www.w3.org/2001/XMLSchema-instance> xsi:type="c:ExtensionType">
<gen774:rsExtensionFirstName>Foo</gen774:rsExtensionFirstName>
<gen774:rsExtensionLastName>Bar</gen774:rsExtensionLastName>
<gen774:rsExtensionNumber>123</gen774:rsExtensionNumber>
<gen774:rsExtensionDecomissioned>false</gen774:rsExtensionDecomissioned>
</extension>
Regards,
Chris.
-----Original Message-----
From: midPoint <midpoint-bounces at lists.evolveum.com><mailto:midpoint-bounces at lists.evolveum.com> On Behalf Of Pavol Mederly
Sent: Friday, July 6, 2018 8:40 AM
To: midpoint at lists.evolveum.com<mailto:midpoint at lists.evolveum.com>
Subject: *EXT* [Newsletter] Re: [midPoint] [Newsletter] Re: Migrating from 3.7.x to 3.8 with custom schema
Hello Chris,
I have seen such an error recently. In that particular case it was probably caused by doing the export by a different version of ninja-3.7, resulting in wrong placement of xsi:type declarations in resulting XML.
Could you paste a snipped of <extension> element of an object that couldn't be imported in your case just to verify this?
Best regards,
Pavol Mederly
Software developer
evolveum.com
On 05.07.2018 14:05, Wojciech Staszewski wrote:
Hmmm... then I have no idea how to import this data. Maybe someone
from Evolveum should reply.
Regards!
WS
W dniu 05.07.2018 o 13:37, Chris Woods pisze:
Hi Wojciech,
I kept the installation directory the same (I used the upgrade.sh
scripts from the 3.8 dist beforehand) so the schema extensions are
still there. Originally I didn't start midpoint 3.8 before starting
the import (there is now mention in the upgrade instructions that
this is necessary). Even after starting midpoint and then retrying
the import I still received the same errors.
Cheers,
Chris.
-----Original Message-----
From: midPoint <midpoint-bounces at lists.evolveum.com><mailto:midpoint-bounces at lists.evolveum.com> On Behalf Of
Wojciech Staszewski
Sent: Thursday, July 5, 2018 1:23 PM
To: midpoint at lists.evolveum.com<mailto:midpoint at lists.evolveum.com>
Subject: *EXT* [Newsletter] Re: [midPoint] Migrating from 3.7.x to
3.8 with custom schema
Is your custom schema xml copied to the v3.8 and was MidPoint
restarted after that?
W dniu 05.07.2018 o 12:22, Chris Woods pisze:
Hi!
I am currently trying to migrate our 3.7.2 installation (PoC) to
3.8. The export worked fine (after realizing I also needed to use –U
with the jdbc url,not just the –j option for the jdbc driver). I
also have my backup.zip containing all the objects – these were
backed up from our existing MySQL 5.6 database.
I am, however, restoring to a fresh MySQL v8.0 database (used the
3.8 create script to create the tables). The only problem seems to
be with objects that have a value in an attribute from our custom
schema – ninja skips those objects with the following Exception(s):
com.evolveum.midpoint.util.exception.SchemaException: Item
'{http://sample.evolveum.com/xml/ns/sample-idm/extension}rsCompany'
without definition can't be saved.
at
com.evolveum.midpoint.repo.sql.helpers.ObjectUpdater.createDataObjec
tF
romJAXB(ObjectUpdater.java:604) ~[repo-sql-impl-3.8.jar!/:na]
at
com.evolveum.midpoint.repo.sql.helpers.ObjectUpdater.addObjectAttemp
t(
ObjectUpdater.java:131) ~[repo-sql-impl-3.8.jar!/:na]
at
com.evolveum.midpoint.repo.sql.SqlRepositoryServiceImpl.addObject(Sq
lR
epositoryServiceImpl.java:431) ~[repo-sql-impl-3.8.jar!/:na]
at
com.evolveum.midpoint.ninja.action.worker.ImportConsumerWorker.run(I
mp
ortConsumerWorker.java:65) ~[classes!/:na]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor
.j
ava:1142) [na:1.8.0_131]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
java:617) [na:1.8.0_131]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
Caused by: java.lang.reflect.InvocationTargetException: null
at sun.reflect.GeneratedMethodAccessor309.invoke(Unknown
Source) ~[na:na]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcce
ss
orImpl.java:43) ~[na:1.8.0_131]
at java.lang.reflect.Method.invoke(Method.java:498)
~[na:1.8.0_131]
at
com.evolveum.midpoint.repo.sql.helpers.ObjectUpdater.createDataObjec
tF
romJAXB(ObjectUpdater.java:594) ~[repo-sql-impl-3.8.jar!/:na]
... 6 common frames omitted
Caused by:
com.evolveum.midpoint.repo.sql.util.DtoTranslationException: Item
'{http://sample.evolveum.com/xml/ns/sample-idm/extension}rsCompany'
without definition can't be saved.
at
com.evolveum.midpoint.repo.sql.data.common.RObject.copyFromJAXB(RObj
ec
t.java:760) ~[repo-sql-impl-3.8.jar!/:na]
at
com.evolveum.midpoint.repo.sql.data.common.RObject.copyFromJAXB(RObj
ec
t.java:726) ~[repo-sql-impl-3.8.jar!/:na]
at
com.evolveum.midpoint.repo.sql.data.common.RFocus.copyFromJAXB(RFocus.
java:312) ~[repo-sql-impl-3.8.jar!/:na]
at
com.evolveum.midpoint.repo.sql.data.common.RUser.copyFromJAXB(RUser.
ja
va:248) ~[repo-sql-impl-3.8.jar!/:na]
... 10 common frames omitted
Caused by: java.lang.IllegalArgumentException: Item
'{http://sample.evolveum.com/xml/ns/sample-idm/extension}rsCompany'
without definition can't be saved.
at
org.apache.commons.lang.Validate.notNull(Validate.java:192)
~[commons-lang-2.6.jar!/:2.6]
at
com.evolveum.midpoint.repo.sql.data.common.any.RAnyConverter.convert
To
RValue(RAnyConverter.java:168) ~[repo-sql-impl-3.8.jar!/:na]
at
com.evolveum.midpoint.repo.sql.data.common.RObject.copyFromJAXB(RObj
ec
t.java:752) ~[repo-sql-impl-3.8.jar!/:na]
... 13 common frames omitted
Does anyone know what I’m doing wrong?
Regards,
Chris.
_______________________________________________
midPoint mailing list
midPoint at lists.evolveum.com<mailto:midPoint at lists.evolveum.com>
http://lists.evolveum.com/mailman/listinfo/midpoint
--
Wojciech Staszewski
Administrator Systemów Sieciowych
tel. kom: 663 680 236
www.diagnostyka.pl<http://www.diagnostyka.pl>
Diagnostyka Sp. z o. o.
ul. Prof. M. Życzkowskiego 16, 31-864 Kraków Numer KRS: 0000381559
(Sąd Rejonowy dla Krakowa-Śródmieścia w Krakowie, XI Wydział
Gospodarczy KRS)
NIP: 675-12-65-009; REGON: 356366975
Kapitał zakładowy: 33 756 500 zł.
Pomyśl o środowisku zanim wydrukujesz ten e-mail.
_______________________________________________
midPoint mailing list
midPoint at lists.evolveum.com<mailto:midPoint at lists.evolveum.com>
http://lists.evolveum.com/mailman/listinfo/midpoint
_______________________________________________
midPoint mailing list
midPoint at lists.evolveum.com<mailto:midPoint at lists.evolveum.com>
http://lists.evolveum.com/mailman/listinfo/midpoint
_______________________________________________
midPoint mailing list
midPoint at lists.evolveum.com<mailto:midPoint at lists.evolveum.com>
http://lists.evolveum.com/mailman/listinfo/midpoint
_______________________________________________
midPoint mailing list
midPoint at lists.evolveum.com<mailto:midPoint at lists.evolveum.com>
http://lists.evolveum.com/mailman/listinfo/midpoint
_______________________________________________
midPoint mailing list
midPoint at lists.evolveum.com<mailto:midPoint at lists.evolveum.com>
http://lists.evolveum.com/mailman/listinfo/midpoint
_______________________________________________
midPoint mailing list
midPoint at lists.evolveum.com<mailto:midPoint at lists.evolveum.com>
http://lists.evolveum.com/mailman/listinfo/midpoint
_______________________________________________
midPoint mailing list
midPoint at lists.evolveum.com<mailto:midPoint at lists.evolveum.com>
http://lists.evolveum.com/mailman/listinfo/midpoint
_______________________________________________
midPoint mailing list
midPoint at lists.evolveum.com<mailto:midPoint at lists.evolveum.com>
http://lists.evolveum.com/mailman/listinfo/midpoint
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20180731/62db0bcb/attachment.htm>
More information about the midPoint
mailing list