ORA-01033: ORACLE initialization or shutdown in progress in log_archive_dest_2 for standby

SQL> @error

-----------------------------------------------------

Host Name          : ora-gold1

Ip Address         : 192.168.0.27

Error date         : 18-10-2014 12:25:49

Dest ID            : 2

Status             : ERROR

DB Name            : vihaan_stdy

DB Mode            : UNKNOWN

Recovery Mode      : IDLE

Protection Mode    : MAXIMUM PERFORMANCE

SRL Count          : 0

SRLActive          : 0

Archived Thread#   : 0

ArchivedSeq#       : 0

Applied Thread#    : 0

Destination        : stdy

Archiver           : LGWR

Transmit Mode      : ASYNCHRONOUS

Affirm             : NO

Asynchronous Blocks: 61440

Net Timeout        : 30

Delay (Mins)       : 0

Reopen (Secs)      : 300

Register           : YES

Binding            : OPTIONAL

Compression        : DISABLE

Error              : ORA-01033: ORACLE initialization or shutdown in progress

----------------------------------------------------

check whether are you able to connect with password primary and standby database with password ?

Check Primary to primary with password 

[oracle@ora-gold1 ~]$ sqlplus system/sys@prim

SQL*Plus: Release 12.1.0.2.0 Production on Sat Oct 18 13:06:12 2014

Copyright (c) 1982, 2014, Oracle.  All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

Check standby from primary 

[oracle@ora-gold1 ~]$ sqlplus system/sys@stdy

SQL*Plus: Release 12.1.0.2.0 Production on Sat Oct 18 13:06:23 2014

Copyright (c) 1982, 2014, Oracle.  All rights reserved.

ERROR:
ORA-01033: ORACLE initialization or shutdown in progress
Process ID: 0
Session ID: 0 Serial number: 0


so problem with password file .

copy password file from primary to standby ...

[oracle@ora-golden2 dbs]$ scp oracle@192.168.0.27:/u01/app/oracle/product/12.1.0/db_1/dbs/ora* .
oracle@192.168.0.27's password:
orapwvihaan                                                                                                                           100% 8704     8.5KB/s   00:00


on primary
set linesize 200 pagesize 200
col destination format a30
col ERROR format a40
select dest_id,destination,status,database_mode,recovery_mode,error from V$ARCHIVE_DEST_STATUS
where status != 'INACTIVE';
   DEST_ID DESTINATION                    STATUS    DATABASE_MODE   RECOVERY_MODE           ERROR
---------- ------------------------------ --------- --------------- ----------------------- ----------------------------------------
         1 /u01/app/oracle/ArchiveLog     VALID     OPEN            IDLE
         2 stdy                           VALID     MOUNTED-STANDBY MANAGED REAL TIME APPLY

Show sql for users in the RAC database ..... and Kill session SQL  

show_sql.sql

set serveroutput on size 1000000
declare
    x number;
begin
    for x in
    ( select username||'('||sid||','||serial#||'@'||INST_ID||
                ') ospid = ' ||  process ||
                ' program = ' || program username,
             '-- alter system kill session '||''''||sid||','||serial#||'@'||INST_ID||''''|| ' immediate ;' kill ,
             to_char(LOGON_TIME,' Day DD YYYY HH24:MI') logon_time,
             to_char(sysdate,' Day DD YYYY HH24:MI') current_time,
             sql_address, LAST_CALL_ET
        from gv$session
       where 1=1
          and  status != 'INACTIVE'
      --  and last_call_ET >1000
      --  and MACHINE='ANUJ'
      --  and username ='USER'
          and rawtohex(sql_address) <> '00'
          and username is not null order by last_call_et )
    loop
        for y in ( select max(decode(piece,0,sql_text,null)) ||
                          max(decode(piece,1,sql_text,null)) ||
                          max(decode(piece,2,sql_text,null)) ||
                          max(decode(piece,3,sql_text,null))
                               sql_text
                     from gv$sqltext_with_newlines
                    where address = x.sql_address
                      and piece < 4)
        loop
            if ( y.sql_text not like '%listener.get_cmd%' )
            --and           y.sql_text not like '%RAWTOHEX(SQL_ADDRESS)%')
            then
                dbms_output.put_line( '--------------------' );
                dbms_output.put_line(x.kill);
                dbms_output.put_line( x.username );
                dbms_output.put_line( x.logon_time || ' ' ||
                                      x.current_time||
                                      ' last et = ' ||
                                      x.LAST_CALL_ET);
                dbms_output.put_line(
                          substr( y.sql_text, 1, 250 ) );
            end if;
       end loop;
    end loop;
end;
 

 ==============================

set serveroutput on size 1000000
declare
    x number;
begin
    for x in
    ( select username||'('||sid||','||serial#||'@'||INST_ID|| ') ospid = ' ||  process ||
                ' program = ' || program username,
             '-- alter system kill session '||''''||sid||','||serial#||'@'||INST_ID||''''|| ' immediate ;' kill ,
             to_char(LOGON_TIME,' Day DD YYYY HH24:MI') logon_time,
             to_char(sysdate,' Day DD YYYY HH24:MI') current_time,
            event,
            sql_id,
             sql_address, LAST_CALL_ET,
             status
        from gv$session
       where 1=1
          and  status != 'INACTIVE'
      --  and last_call_ET >1000
      --  and MACHINE='ANUJ'
      --  and username ='USER'
          and  SCHEMANAME not in ( 'SYS' , 'SYSTEM' , 'DBSNMP' , 'SYSMAN' , 'OUTLN' , 'MDSYS' , 'ORDSYS' , 'EXFSYS' , 'DMSYS' , 'WMSYS' , 'CTXSYS' , 'ANONYMOUS' , 'XDB' , 'ORDPLUGINS' , 'OLAPSYS' , 'PUBLIC','WWV_FLOW_PLATFORM' )
          and rawtohex(sql_address) <> '00'
          and username is not null order by last_call_et )
    loop
        for y in ( select max(decode(piece,0,sql_text,null)) ||
                          max(decode(piece,1,sql_text,null)) ||
                          max(decode(piece,2,sql_text,null)) ||
                          max(decode(piece,3,sql_text,null))    sql_text
                     from gv$sqltext_with_newlines
                    where address = x.sql_address
                      and piece < 4)
        loop
            if ( y.sql_text not like '%listener.get_cmd%' )
            --and           y.sql_text not like '%RAWTOHEX(SQL_ADDRESS)%')
            then
                dbms_output.put_line( '--------------------' );
                dbms_output.put_line(x.kill);
                dbms_output.put_line( x.username );
                dbms_output.put_line( x.logon_time || ' ' ||  x.current_time|| ' last et = ' ||  x.LAST_CALL_ET);
                dbms_output.put_line( x.event );
                dbms_output.put_line( x.status );
                dbms_output.put_line( x.sql_id );
                dbms_output.put_line( substr( y.sql_text, 1, 250 ) );
            end if;
       end loop;
    end loop;
end;

output ....

-- alter system kill session '1474,47941@1' immediate ;
ANUJ(1474,47941@1) ospid = 25799 program = sqlplus@rac01
(TNS V1-V3)
Friday    23 2018 07:02  Wednesday 07 2018 07:41 last et = 1039108
resmgr:cpu quantum
ACTIVE
bp061zzjnstuu
BEGIN
        WHILE  1 = 1
        LOOP
                NULL;
        END
LOOP;
END;

============================================


set linesize 300 pagesize 300 
column username           format a30 word_wrapped
column module             format a30 word_wrapped
column action             format a15 word_wrapped
column client_info        format a30 word_wrapped
col kill 				for a15
select ''''||s.sid ||','|| s.serial#||',@'||s.inst_id ||''''  as kill,
 username,
       module,
       action,
       client_info,
      sql_id,
     prev_sql_id,
     event 
from gv$session s
where 1=1
and module||action||client_info is not null
and  schemaname is not null 
and  schemaname not in ('SYS')
;

Delete archivelogs using RMAN until date .....

RMAN delete archive until time

RMAN> run
{
DELETE archivelog until time "to_date('2010-10-08:05:00:00','YYYY-MM-DD:hh24:mi:ss')";

crosscheck archivelog all;
delete noprompt expired archivelog all;

}

PRVF-0002 Could not retrieve local nodename

Oracle Database: PRVF-0002 Could not retrieve local nodename

Cause: Unable to determine local host name using Java network functions. 

Action: Ensure hostname is defined correctly using the 'hostname' command.

Change your hosts file as following , restart your network services and try installer again 

[root@ora-gold1 ~]# cat /etc/hosts

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4

::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.0.27 ora-gold1 ora-gold1

ORA-21561: OID generation failed

[oracle@ora-golden2 admin]$ sqlplus 'sys/vihaan@stdy as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Sun Oct 5 12:49:22 2014

Copyright (c) 1982, 2014, Oracle.  All rights reserved.

ERROR:

ORA-21561: OID generation failed

I have no idea why I am getting above error ..

on Oracle linux 6.5 , work around is restart network services and try again 

as a root ......

[root@ora-golden2 ~]# service network restart

Shutting down interface eth0:  Device state: 3 (disconnected)

                                                           [  OK  ]

Shutting down loopback interface:                          [  OK  ]

Bringing up loopback interface:                            [  OK  ]

Bringing up interface eth0:  Active connection state: activating

Active connection path: /org/freedesktop/NetworkManager/ActiveConnection/3

state: activated

Connection activated

                                                           [  OK  ]

[root@ora-golden2 ~]# su - oracle

[oracle@ora-golden2 ~]$ sqlplus 'sys/vihaan@stdy as sysdba'

SQL*Plus: Release 12.1.0.2.0 Production on Sun Oct 5 12:49:49 2014

Copyright (c) 1982, 2014, Oracle.  All rights reserved.

Connected to an idle instance.