R12.2 OHS was not coming up after unclean shutdown or due to server crash.
Issue:
R12.2 OHS was not coming up after unclean shutdown or due to server crash.
Processes in Instance: EBS_web_OHS1
———————————+——————–+———+———
ias-component | process-type | pid | status
———————————+——————–+———+———
EBS_web | OHS | 9214 | stop
OHS was in STOP state and we were not able to either bring down or bring up. It shows PID as 6194 which doens’t exist in server.
In logs we were getting below errors.
Log file Location
/appsvis/VIS/fs1/FMW_Home/webtier/instances/EBS_web_OHS1/diagnostics/logs/OHS/EBS_web/
console~OHS~1- log :
22/11/01 00:17:19 Stop process
/appsvis/VIS/fs1/FMW_Home/webtier/ohs/bin/apachectl hardstop: httpd (pid 9214?) not running
/appsvis/VIS/fs1/FMW_Home/webtier/ohs/bin/apachectl hardstop: httpd (pid 9214?) not running
22/11/01 00:18:21 Stop process
/appsvis/VIS/fs1/FMW_Home/webtier/ohs/bin/apachectl hardstop: httpd (pid 9214?) not running
22/11/01 00:19:23 Stop process
/appsvis/VIS/fs1/FMW_Home/webtier/ohs/bin/apachectl hardstop: httpd (pid 9214?) not running
/appsvis/VIS/fs1/FMW_Home/webtier/instances/EBS_web_OHS1/diagnostics/logs/OHS/EBS_web/httpd.pid overwritten — Unclean shutdown of previous Apache run?
Caution:
So it is very clear that OHS was not shutdown properly.Due to this some lock file or httpd.pid files with old pid exists. we have to remove this and bring up.
Solution:
- First shutdown opmn process using adopmnctl.sh stop
[applmgr@appsvis scripts]$ adopmnctl.sh stop
You are running adopmnctl.sh version 120.0.12020000.2
Stopping Oracle Process Manager (OPMN) and the managed processes …
opmnctl stopall: stopping opmn and all managed processes…
opmn id=appsprod:6200
Response: 0 of 1 processes stopped.
ias-instance id=EBS_web_OHS1
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
ias-component/process-type/process-set:
EBS_web/OHS/OHS/
Error
–> Process (index=1,uid=625953226,pid=10873)
time out while waiting for a managed process to stop
second stop attempted
time out while waiting for a managed process to stop
Log:
/appsvis/VIS/fs1/FMW_Home/webtier/instances/EBS_web_OHS1/diagnostics/logs/OHS/EBS_web/console~OHS~1.log
opmnctl stopall: stop of processes failed, using forceful shutdown…
adopmnctl.sh: exiting with status 0
adopmnctl.sh: check the logfile /appsvis/VIS/fs1/inst/apps/PROD_appsprod/logs/appl/admin/log/adopmnctl.txt for more information …
- ps -ef | grep 10873
- kill -9
4. Remove httpd.pid file from
/appsvis/VIS/fs1/FMW_Home/webtier/instances/EBS_web_OHS1/diagnostics/logs/OHS/EBS_web
rm -rf httpd.pid
- cd /appsvis/VIS/fs1/FMW_Home/webtier/instances/EBS_web_OHS1/config/OPMN/opmn/states
grep -l 625953226 *
p625953226
6.check hidden file also
[applmgr@appsvis states]$ ls -la
total 12
drwx—— 2 applmgr dba 55 Nov 4 09:21 .
drwx—— 4 applmgr dba 114 Nov 4 09:20 ..
-rw-r—– 1 applmgr dba 9 Nov 4 09:20 .locale
-rw-r—– 1 applmgr dba 16 Nov 6 19:33 .opmndat
-rw-r—– 1 applmgr dba 517 Nov 4 09:21 p906104719
remove all file
7.try to start the OHS.
adopmnctl.sh start
- [applmgr@appsvis scripts]$ adopmnctl.sh status
You are running adopmnctl.sh version 120.0.12020000.2
Checking status of OPMN managed processes…
Processes in Instance: EBS_web_OHS1
———————————+——————–+———+———
ias-component | process-type | pid | status
———————————+——————–+———+———
EBS_web | OHS | 18291 | Alive
adopmnctl.sh: exiting with status 0
adopmnctl.sh: check the logfile /appsvis/VIS/fs1/inst/apps/PROD_appsprod/logs/appl/admin/log/adopmnctl.txt for more information …
[applmgr@appsvis scripts]$ adapcctl.sh status
You are running adapcctl.sh version 120.0.12020000.6
Checking status of OPMN managed Oracle HTTP Server (OHS) instance …
Processes in Instance: EBS_web_OHS1
———————————+——————–+———+———
ias-component | process-type | pid | status
———————————+——————–+———+———
EBS_web | OHS | 18291 | Alive
adapcctl.sh: exiting with status 0
adapcctl.sh: check the logfile /appsvis/VIS/fs1/inst/apps/PROD_appsprod/logs/appl/admin/log/adapcctl.txt for more information …