All java process pertaining to IDM stack were getting killed very periodically. And the servers had to be restarted very often.
Weblogic 11g, IDM stack java 1.70.79
- There were no thread dumps or core dumps.
- System logs were clean.
- Greped the weblogic logs for errors, could not find anything
- After carefully going through each and every line of the weblogic logs, I could find below messages:
####<Jul 26, 2017 6:36:47 PM IST> <Notice> <WebLogicServer> <myhost.me.me.com> <AdminServer> <Thread-1> <<WLS Kernel>> <> <ae8121cbb691c6b3:-fc73acf:15d7e34bd34:-8000-00000000000002ff> <1501074407979> <BEA-000388> <JVM called WLS shutdown hook. The server will force shutdown now>
####<Jul 26, 2017 6:36:47 PM IST> <Alert> <WebLogicServer> <myhost.me.me.com> <AdminServer> <Thread-1> <<WLS Kernel>> <> <ae8121cbb691c6b3:-fc73acf:15d7e34bd34:-8000-00000000000002ff> <1501074407980> <BEA-000396> <Server shutdown has been requested by <WLS Kernel>>
The logs clearly indicates that the JVM itself initiated the shutdown of the servers and OS has nothing to do with it.
The problem might occur due to incompatibility of softwares. In our case there was no customization or any of our own developed softwares deployed. Ours was clean install.
This points me to JDK version. After pointing to the correct java, the issue has been resolved.
Hope it helps.