[midPoint] [midpoint] Active/Active Configuration
Florin. Stingaciu
fstingaciu at mirantis.com
Tue Aug 16 00:29:19 CEST 2016
Awesome! Thanks for all your help today Pavol. I will try and test in my
environment and let you know how that goes.
Thanks again!
-F
On Mon, Aug 15, 2016 at 3:00 PM, Pavol Mederly <mederly at evolveum.com> wrote:
> Florin,
>
> I think I've fixed it: https://github.com/Evolveum/midpoint/commit/
> c00f2979cea6f6ada0a1194b13bad51c0ecec2e3
>
> Cannot verify as the issue isn't visible on my computer, but I am quite
> convinced it should be fixed now. The suspected cause is that some
> query-only transactions weren't closed properly. C3P0 closes them
> automatically but other connection providers probably not
> <http://stackoverflow.com/questions/19396563/does-hibernates-session-close-automatically-rollback-uncommitted-transactions>
> .
>
> You can safely use the master, or you can apply the above patch to 3.4
> yourself. The master is something like "3.4-support" these days (preparing
> for 3.4.1), so it should be of production-level quality, or at least near
> it.
> (I'm going to run full test suite during the night, but I don't expect any
> failures. Or you can check after approx 3 hours here
> <https://travis-ci.org/Evolveum/midpoint>.)
>
> Best regards,
>
> Pavol Mederly
> Software developerevolveum.com
>
> On 15.08.2016 22:57, Florin. Stingaciu wrote:
>
> Hey Pavol,
>
> I uploaded the log file you requested to my gdrive (
> https://drive.google.com/file/d/0B9f4-vWBsztoZy1zWThGYnpGeGs/view?
> usp=sharing) It's a trace level debug so the file is quite large. Let me
> know if anything jumps out at you.
>
> Thanks,
> -F
>
> On Mon, Aug 15, 2016 at 1:07 PM, Pavol Mederly <mederly at evolveum.com>
> wrote:
>
>> Strange enough. I've replicated your setup on my computer (using
>> PostgreSQL, as I have no MySQL currently installed), and it works: midPoint
>> starts cleanly. I am able to add/delete repository objects.
>>
>> Attached are my config.xml and server.xml; they are pretty much the same
>> as yours.
>>
>> So, I have no idea. You could try adding <database>mysql</database> into
>> <repository> element in your config.xml, but it would most probably make no
>> difference.
>>
>> As last resort, you could turn on the most detailed debugging for the
>> repository on startup by adding a line like this
>>
>> <logger name="com.evolveum.midpoint.repo" level="TRACE" />
>>
>> into webapps\midpoint\WEB-INF\classes\logback.xml file and start
>> midPoint. In idm.log there should be quite a detailed record of what's
>> going on.
>>
>> Pavol Mederly
>> Software developerevolveum.com
>>
>> On 15.08.2016 21:03, Florin. Stingaciu wrote:
>>
>> Here's my datasource config:
>>
>> <Resource name="jdbc/mysql" auth="Container"
>> type="javax.sql.DataSource"
>> username="midpoint" password="pass"
>> url="jdbc:mysql://SERVER:3306/midpoint_dev"
>> driverClassName="com.mysql.jdbc.Driver"
>> accessToUnderlyingConnectionAllowed="true"
>> initialSize="5" maxWait="5000"
>> maxActive="30" maxIdle="5"
>> validationQuery="select 1"
>> poolPreparedStatements="true"/>
>>
>> And here's my config.xml:
>>
>> <repository>
>> <embedded>false</embedded>
>> <repositoryServiceFactoryClass
>> >com.evolveum.midpoint.repo.sql.SqlRepositoryFactory</reposi
>> toryServiceFactoryClass>
>> <hibernateHbm2ddl>validate</hibernateHbm2ddl>
>> <hibernateDialect>com.evolveum.midpoint.repo.sql.util.
>> MidPointMySQLDialect</hibernateDialect>
>> <dataSource>java:comp/env/jdbc/mysql</dataSource>
>> </repository>
>> <taskManager>
>> <clustered>true</clustered>
>> <jmxUsername>midpoint</jmxUsername>
>> <jmxPassword>password</jmxPassword>
>> </taskManager>
>>
>> On Mon, Aug 15, 2016 at 11:44 AM, Pavol Mederly <mederly at evolveum.com>
>> wrote:
>>
>>> Hello Florin,
>>>
>>> this is really interesting. Please, could you also share your midPoint
>>> config.xml, as well as your data source configuration? (except credentials,
>>> of course)
>>>
>>> Best regards,
>>>
>>> Pavol Mederly
>>> Software developerevolveum.com
>>>
>>> On 15.08.2016 19:39, Florin. Stingaciu wrote:
>>>
>>> Hello Pavol,
>>>
>>> Thanks for your detailed response. I tried setting up the datasource
>>> with validationQuery set up properly for the mySQL backed I have. However,
>>> upon service restart I get the following errors: http://pastebin.com/8d
>>> pGN0JC
>>>
>>> To save you a click, it seems as though the connection is set up in
>>> readonly mode or some other strange things happen. I've tried setting
>>> readonly="false" in the resource definition in server.xml but that
>>> didn't help. I will continue researching this, however any guidance would
>>> be quite appreciated.
>>>
>>> Thanks!
>>> -F
>>>
>>> On Mon, Aug 15, 2016 at 2:29 AM, Pavol Mederly <mederly at evolveum.com>
>>> wrote:
>>>
>>>> Hello Florin,
>>>>
>>>> having looked at your logs, it seems that maybe explicit setting of
>>>> validationQuery in Quartz data source setup would help. (See
>>>> http://www.quartz-scheduler.org/documentation/quartz-2.x/con
>>>> figuration/ConfigDataSources.html.)
>>>>
>>>> Unfortunately, current midPoint implementation does not allow to
>>>> configure Quartz data source parameters. So, there are the following three
>>>> possibilities:
>>>>
>>>> 1. Take an alternative route, and use application server-defined
>>>> data source (with validationQuery set up).
>>>> 2. Patch Task Manager implementation by adding appropriate lines to
>>>> Quartz configuration (see LocalNodeManager.java:87-90).
>>>> 3. Wait until we implement it - I've created an issue MID-3347
>>>> <https://jira.evolveum.com/browse/MID-3347> for this.
>>>>
>>>> As for the first one (externally defined data source): Please see
>>>> https://wiki.evolveum.com/display/midPoint/Repository+Config
>>>> uration#RepositoryConfiguration-Datasourceconfiguration on how to
>>>> configure midPoint repository with the data source. This data source will
>>>> be used also by Quartz, if not overriden in <taskManager> section. It
>>>> should work but I don't remember if someone actually tested this.
>>>>
>>>> Concerning autoReconnect: I have no experiences with this setting. I
>>>> agree with you that about the hesitation of using it in production
>>>> environment. If really needed, I'd recommend to separate midPoint
>>>> repository configuration from Quartz configuration by using different JDBC
>>>> URLs for the two: the standard one for the repository and the one with
>>>> "autoReconnect=true" for Quartz. In this way, potential negative effects
>>>> should be restricted to task management functionality only. But, overall,
>>>> I'd suggest trying to eliminate the problem by setting validationQuery
>>>> first.
>>>>
>>>> Best regards,
>>>>
>>>> Pavol Mederly
>>>> Software developerevolveum.com
>>>>
>>>> On 14.08.2016 22:40, Florin. Stingaciu wrote:
>>>>
>>>> Hello,
>>>>
>>>> I'm trying to configure an active/active configuration, and
>>>> experiencing some issues with the Quartz scheduler. The SQL connection
>>>> seems to timeout quite often and result in many warning messages. I'm also
>>>> experiencing some errors -- as the timeout closes the connection, some
>>>> processes are still trying to commit using that stale handler.
>>>>
>>>> Here are some relevant logs from both midPoint instances in this
>>>> cluster: http://pastebin.com/JFjqrLnT http://pastebin.com/1vy4iYPY
>>>>
>>>> Also here's the relevant portion of my config.xml file:
>>>> http://pastebin.com/WbydcCC8
>>>>
>>>> Following the suggestions in the warnings, I've started to look at
>>>> autoReconnect propriety of the JDBC connector and applied it to my configs
>>>> like so:
>>>>
>>>> <jdbcUrl>jdbc:mysql://SERVER:3306/midpoint_dev?autoReconnect
>>>>> =true</jdbcUrl>
>>>>
>>>>
>>>> Since implementing this change, the errors and warnings seem to have
>>>> disappeared. I will continue to monitor the logs and ensure this actually
>>>> the case.
>>>>
>>>> Reading the mysql docs, I found that this is not recommended as this
>>>> may cause data inconsistency issues and that stale connection exceptions
>>>> should be properly caught within the application. Namely:
>>>>
>>>> The use of this feature is not recommended, because it has side effects
>>>>> related to session state and data consistency when applications don't
>>>>> handle SQLExceptions properly, and is only designed to be used when you are
>>>>> unable to configure your application to handle SQLExceptions resulting from
>>>>> dead and stale connections properly.
>>>>
>>>>
>>>> Do you have any recommended configuration for this scenario? I'd like
>>>> to move my current production environment in an active active
>>>> configuration, however as of right now I'm hesitant to do so in order to
>>>> avoid any data corruption. Especially since it's quite difficult to test
>>>> for data consistency issues that may arise from using autoReconnect in my
>>>> dev environment.
>>>>
>>>> Thanks,
>>>> -F
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> midPoint mailing listmidPoint at lists.evolveum.comhttp://lists.evolveum.com/mailman/listinfo/midpoint
>>>>
>>>> _______________________________________________ midPoint mailing list
>>>> midPoint at lists.evolveum.com http://lists.evolveum.com/mail
>>>> man/listinfo/midpoint
>>>
>>> _______________________________________________
>>> midPoint mailing listmidPoint at lists.evolveum.comhttp://lists.evolveum.com/mailman/listinfo/midpoint
>>>
>>> _______________________________________________ midPoint mailing list
>>> midPoint at lists.evolveum.com http://lists.evolveum.com/mail
>>> man/listinfo/midpoint
>>
>> _______________________________________________
>> midPoint mailing listmidPoint at lists.evolveum.comhttp://lists.evolveum.com/mailman/listinfo/midpoint
>>
>> _______________________________________________ midPoint mailing list
>> midPoint at lists.evolveum.com http://lists.evolveum.com/mail
>> man/listinfo/midpoint
>
> _______________________________________________
> midPoint mailing listmidPoint at lists.evolveum.comhttp://lists.evolveum.com/mailman/listinfo/midpoint
>
>
> _______________________________________________
> midPoint mailing list
> midPoint at lists.evolveum.com
> http://lists.evolveum.com/mailman/listinfo/midpoint
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.evolveum.com/pipermail/midpoint/attachments/20160815/11afc9c0/attachment.htm>
More information about the midPoint
mailing list