[midPoint] [External] RE: Unable to deploy midPoint 4.8 with postgresql using Docker
Drew Roberts
aroberts at apu.edu
Wed Nov 22 07:40:16 CET 2023
Hey Huttm & Deshak,
After reviewing this further I realized I missed a few things that were in
my config and are essential to make this work. I really hope there is a
simpler way to do a native postgresql database because the way I got it
working is ridiculous for the new recommended default option. Anyways,
here's a more complete guide to my maniac method:
1. Add these variables to your midPoint container:
1. MP_SET_midpoint_repository_type=native
2.
MP_SET_midpoint_repository_jdbcUrl=jdbc:postgresql://db.at.your.address:5432/midpoint
3. MP_SET_midpoint_repository_jdbcUsername=midpoint
4.
MP_SET_midpoint_repository_jdbcPassword_FILE=/run/secrets/mp_database_password.txt
5.
MP_SET_midpoint_keystore_keyStorePassword_FILE=/run/secrets/mp_keystore_password.txt
2. Make sure your midPoint container does NOT have these variables:
1. - REPO_DATABASE_TYPE=postgresql
2. - REPO_HOST=midpoint_data
3. - REPO_DATABASE=midpoint_db
4. - REPO_USER=midpoint
5. - REPO_PASSWORD_FILE=/run/secrets/mp_database_password.txt
6. - MP_KEYSTORE_PASSWORD_FILE=/run/secrets/mp_keystore_password.txt
3. Create a config.xml file and put it in /opt/midpoint/var on your
midPoint container (you could probably fill in these variables instead of
using the MP_SET stuff above if you want. I prefer to use environment
variables when I can):
>
> <?xml version="1.0"?>
> <!--
> ~ Copyright (C) 2010-2021 Evolveum and contributors
> ~
> ~ This work is dual-licensed under the Apache License 2.0
> ~ and European Union Public License. See LICENSE file for details.
> -->
>
> <!--
> Example config.xml for Native PG repository.
> To use it:
> 1. Copy it to $MIDPOINT_HOME, e.g. under midpoint.../var directory in
> the extracted archive.
> 2. Rename the copied file in $MIDPOINT_HOME to config.xml.
> 3. Check DB connection parameters and change them as needed.
> 4. Start midPoint (bin/start.sh) and observe the success in
> $MIDPOINT_HOME/log/midpoint.log.
>
> For more info see:
>
> * midPoint home:
>
> https://docs.evolveum.com/midpoint/reference/deployment/midpoint-home-directory/
>
> * Overriding config.xml parameters from command line:
>
> https://docs.evolveum.com/midpoint/reference/deployment/midpoint-home-directory/overriding-config-xml-parameters/
>
> * Repository configuration:
> https://docs.evolveum.com/midpoint/reference/repository/configuration/
>
> * Canonical config.xml auto-created if not present in $MIDPOINT_HOME
> (master branch):
>
> https://github.com/Evolveum/midpoint/blob/master/repo/system-init/src/main/resources/config.xml
> -->
> <configuration>
> <midpoint>
> <webApplication>
> <importFolder>${midpoint.home}/import</importFolder>
> </webApplication>
> <repository>
> <type>native</type>
> <jdbcUrl>jdbc:postgresql://localhost:5432/midpoint</jdbcUrl
> >
> <jdbcUsername>midpoint</jdbcUsername>
> <jdbcPassword>password</jdbcPassword>
> </repository>
> <audit>
> <auditService>
> <auditServiceFactoryClass>
> com.evolveum.midpoint.audit.impl.LoggerAuditServiceFactory</
> auditServiceFactoryClass>
> </auditService>
> <auditService>
> <auditServiceFactoryClass>
> com.evolveum.midpoint.repo.sqale.audit.SqaleAuditServiceFactory</
> auditServiceFactoryClass>
> </auditService>
> </audit>
> <icf>
> <scanClasspath>true</scanClasspath>
> <scanDirectory>${midpoint.home}/icf-connectors</
> scanDirectory>
> </icf>
> <keystore>
> <keyStorePath>${midpoint.home}/keystore.jceks</keyStorePath
> >
> <keyStorePassword>changeit</keyStorePassword>
> <encryptionKeyAlias>default</encryptionKeyAlias>
> </keystore>
> </midpoint>
> </configuration>
>
4. Set things so your midPoint container doesn't boot til after your
database is configured
5. Recreate this logic somehow on the database container for the first
launch (maybe make an init container to run these commands or something?):
> sudo su - postgres
> psql
> CREATE USER midpoint WITH PASSWORD '<PASSWORD>' LOGIN SUPERUSER;
> CREATE DATABASE midpointdev WITH OWNER = midpoint ENCODING = 'UTF8'
> TABLESPACE = pg_default LC_COLLATE = 'en_US.UTF-8' LC_CTYPE = 'en_US.UTF-8'
> CONNECTION LIMIT = -1;
> \q
>
> curl
> https://raw.githubusercontent.com/Evolveum/midpoint/support-4.8/config/sql/native/postgres.sql
> --output postgres.sql
> curl
> https://raw.githubusercontent.com/Evolveum/midpoint/support-4.8/config/sql/native/postgres-audit.sql
> --output postgres-audit.sql
> curl
> https://raw.githubusercontent.com/Evolveum/midpoint/support-4.8/config/sql/native/postgres-quartz.sql
> --output postgres-quartz.sql
>
> psql -h localhost -d midpointdev -U midpoint -f postgres.sql -f
> postgres-audit.sql -f postgres-quartz.sql
I hope this time I've included everything you need to get it going.
Apologies for the incomplete information before.
On Tue, Nov 21, 2023 at 9:37 PM lazy tech <huutm at lazytech.biz.vn> wrote:
> Hello Drew,
> I followed the same steps as outlined in your guide, but I am still
> encountering an error. Could you provide further assistance?
>
> 2023-11-22 12:05:30 ... 94 common frames omitted
> 2023-11-22 12:05:30 Caused by:
> com.evolveum.midpoint.util.exception.SystemException: AuditServiceFactory
> implementation class com.evolveum.midpoint.repo.sql.SqlAuditServiceFactory
> failed to initialize: Couldn't find AuditServiceFactory for class
> com.evolveum.midpoint.repo.sql.SqlAuditServiceFactory
> 2023-11-22 12:05:30 at
> com.evolveum.midpoint.init.AuditFactory.init(AuditFactory.java:84)
> 2023-11-22 12:05:30 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> 2023-11-22 12:05:30 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> 2023-11-22 12:05:30 at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2023-11-22 12:05:30 at
> java.base/java.lang.reflect.Method.invoke(Method.java:568)
> 2023-11-22 12:05:30 at
> org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMethod.invoke(InitDestroyAnnotationBeanPostProcessor.java:457)
> 2023-11-22 12:05:30 at
> org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:401)
> 2023-11-22 12:05:30 at
> org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:219)
> 2023-11-22 12:05:30 ... 115 common frames omitted
> 2023-11-22 12:05:30 Caused by:
> com.evolveum.midpoint.util.exception.SystemException: Couldn't find
> AuditServiceFactory for class
> com.evolveum.midpoint.repo.sql.SqlAuditServiceFactory
> 2023-11-22 12:05:30 at
> com.evolveum.midpoint.init.AuditFactory.getFactory(AuditFactory.java:97)
> 2023-11-22 12:05:30 at
> com.evolveum.midpoint.init.AuditFactory.init(AuditFactory.java:75)
> 2023-11-22 12:05:30 ... 122 common frames omitted
> 2023-11-22 12:05:30 ERROR initializing midPoint:
> org.springframework.context.ApplicationContextException: Unable to start
> web server
> 2023-11-22 12:05:30 ROOT cause:
> com.evolveum.midpoint.util.exception.SystemException: Couldn't find
> AuditServiceFactory for class
> com.evolveum.midpoint.repo.sql.SqlAuditServiceFactory
> 2023-11-22 12:05:30 See midpoint.log for more details.
>
>
> Thanks,
> Huutm
>
>
>
> ---- Trên Tue, 14 Nov 2023 23:42:18 +0700 *Drew Roberts via midPoint
> <midpoint at lists.evolveum.com <midpoint at lists.evolveum.com>>* đã viết ---
>
> Hey Deshak,
>
> That's the error you should get if you only did the first step I gave you.
> Run the commands I gave you in step 3 on the database, then reboot the
> midpoint container and your issue should be resolved.
>
> On Tue, Nov 14, 2023 at 4:45 AM BHATNAGAR Deshak <
> deshak.bhatnagar at thalesgroup.com> wrote:
>
> THALES GROUP LIMITED DISTRIBUTION to email recipients
>
>
>
> Hello Drew,
>
>
>
> Thanks for the swift reply and the possible solution.
>
>
>
> I tried the same as explained below but faced another error:
>
>
>
> ERROR initializing midPoint:
> org.springframework.context.ApplicationContextException: Unable to start
> web server
>
> ROOT cause: org.postgresql.util.PSQLException: ERROR: relation
> "m_global_metadata" does not exist
>
> Position: 31
>
>
>
> I have attached the complete error below in the error 14 Nov 2023 file.
>
>
>
> It would be great if you can share the docker-compose & and any other
> adjustments or changes you have made to your setup, so I can get a better
> understanding of it.
>
>
>
> Kind regards,
>
> Deshak Bhatnagar
>
> *From:* Drew Roberts <aroberts at apu.edu>
> *Sent:* Saturday, November 11, 2023 3:50 AM
> *To:* midPoint General Discussion <midpoint at lists.evolveum.com>
> *Cc:* BHATNAGAR Deshak <deshak.bhatnagar at thalesgroup.com>
> *Subject:* Re: [External] Re: [midPoint] Unable to deploy midPoint 4.8
> with postgresql using Docker
>
>
>
> Hey Deshak,
>
>
>
> If you take a look at the Release Notes for midPoint 4.8
> <https://docs.evolveum.com/midpoint/release/4.8/> and use your browser's
> Find function with the term "native", a couple results in you'll see:
>
>
> - Generic Repository with PostgreSQL is not supported, if you are
> using PostgreSQL with generic repository, please migrate to PostgreSQL
> native repository
> <https://docs.evolveum.com/midpoint/reference/master/repository/native-postgresql/>.
>
>
> I believe the issue you're experiencing is due to this change. My
> environment and setup are slightly different from yours so your mileage may
> vary but here are my two cents:
>
> 1. Add a MP_SET_midpoint_repository_type=native environment variable
> to your midPoint container
> 2. Set things so your midPoint container doesn't boot til after your
> database is configured
> 3. Recreate this logic somehow on the database container for the first
> launch (maybe make an init container to run these commands or something?):
>
> sudo su - postgres
> psql
> CREATE USER midpoint WITH PASSWORD '<PASSWORD>' LOGIN SUPERUSER;
> CREATE DATABASE midpointdev WITH OWNER = midpoint ENCODING = 'UTF8'
> TABLESPACE = pg_default LC_COLLATE = 'en_US.UTF-8' LC_CTYPE = 'en_US.UTF-8'
> CONNECTION LIMIT = -1;
> \q
>
> curl
> https://raw.githubusercontent.com/Evolveum/midpoint/support-4.8/config/sql/native/postgres.sql
> --output postgres.sql
> curl
> https://raw.githubusercontent.com/Evolveum/midpoint/support-4.8/config/sql/native/postgres-audit.sql
> --output postgres-audit.sql
> curl
> https://raw.githubusercontent.com/Evolveum/midpoint/support-4.8/config/sql/native/postgres-quartz.sql
> --output postgres-quartz.sql
>
> psql -h localhost -d midpointdev -U midpoint -f postgres.sql -f
> postgres-audit.sql -f postgres-quartz.sql
>
>
>
> On Fri, Nov 10, 2023 at 8:37 AM BHATNAGAR Deshak via midPoint <
> midpoint at lists.evolveum.com> wrote:
>
> THALES GROUP LIMITED DISTRIBUTION to email recipients
>
>
>
> Added the docker-compose for reference.
>
>
>
> Thanks
>
> *From:* BHATNAGAR Deshak
> *Sent:* Friday, November 10, 2023 9:43 PM
> *To:* midpoint at lists.evolveum.com
> *Subject:* Unable to deploy midPoint 4.8 with postgresql using Docker
>
>
>
> THALES GROUP LIMITED DISTRIBUTION to email recipients
>
>
>
> Hello midpoint community,
>
> I am deploying the Evolveum midpoint 4.8 with postgresql using Docker but
> I am facing the below mentioned error:
>
>
>
> ERROR initializing midPoint:
> org.springframework.context.ApplicationContextException: Unable to start
> web server
>
> ROOT cause: com.evolveum.midpoint.util.exception.SystemException: DB
> script (/sql/postgresql-4.6-all.sql) couldn't be found
>
>
>
> This above error occurs because the system is not able to create required
> database tables, the log file is attached below (log file (error))
>
>
>
> I was previously using the LTS version midpoint 4.4.4 and used the same
> docker-compose file with changes to midpoint 4.8 image and postgresql-16
> image.
>
>
>
> I have tried with all the recommended version of both midpoint 4.8 as well
> as postgresql db.
>
>
>
> But the error remains the same in all cases.
>
>
>
> Also, a similar error was reported on the Jira of Evolveum regarding the
> schema validation in which the database tables were not created:
> https://jira.evolveum.com/projects/MID/issues/MID-9258?filter=doneissues
>
>
>
> As per the conclusion drawn after my troubleshooting.
>
> I found out that the config/sql file in midpoint 4.8 repository on Github
> do not have the above-mentioned file in the error i.e.,
> postgresql-4.6-all.sql (
> https://github.com/Evolveum/midpoint/tree/v4.8/config/sql/native).
>
>
>
> In-comparison, all of the previous versions of midpoint on the Github had
> the file (postgresql-4.6-all.sql) in their respective repositories, for
> example: midpoint 4.7.2 had the file postgresql-4.6-all.sql in it on the
> Github (
> https://github.com/Evolveum/midpoint/tree/v4.7.2/config/sql/native-new).
>
>
>
> So, I suspect that the issue is caused due to the file being missing in
> the repositories.
>
>
>
> Please resolve the above issue or let me know the way that I can resolve
> this.
>
>
>
> docker-compose is also attached for the reference.
>
>
>
> Thanks.
>
>
>
> Kind regards,
>
> Deshak Bhatnagar
>
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com
> https://lists.evolveum.com/mailman/listinfo/midpoint
>
>
>
> --
>
> *Drew Roberts* | *Systems Administrator II*
> IT Platform Engineering, Azusa Pacific University
> apu.edu
>
> [image: APU logo]
>
>
>
> --
> *Drew Roberts* | *Systems Administrator II*
> IT Platform Engineering, Azusa Pacific University
> apu.edu
>
> [image: APU logo]
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com
> https://lists.evolveum.com/mailman/listinfo/midpoint
>
>
>
>
--
Drew Roberts | Systems Administrator II
IT Platform Engineering, Azusa Pacific University
apu.edu
[image: APU logo]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20231121/e68eb217/attachment-0001.htm>
More information about the midPoint
mailing list