Database Crash With ORA-27063 and OS Error: 5: I/O Error



Oracle Database – Enterprise Edition – Version 8.1.6.0 to 11.2.0.0 [Release 8.1.6 to 11.2]
Oracle Database – Enterprise Edition – Version 11.2.0.2 to 11.2.0.2 [Release 11.2]
Information in this document applies to any platform.
***Checked for relevance on 07-Oct-2011***

SYMPTOMS

Database may crash due to I/O errors depending on the file that reports the error, such as the controlfile. 

Alert log may report messages like these:

ORA-00206: error in writing (block 3, # blocks 1) of control file 

ORA-00202: control file: ‘/u04/oradata/symprd/control03.ctl’ 

ORA-27063: number of bytes read/written is incorrect 

SVR4 Error: 5: I/O error 

Additional information: -1 

Additional information: 16384

 OR

ORA-00345: redo log write error block 24041 count 2 

ORA-00312: online log 5 thread 1: ‘/oradata/bjfhcat/redo05.log’ 

ORA-27063: number of bytes read/written is incorrect 

IBM AIX RISC System/6000 Error: 5: I/O error

Additional information: -1 

Additional information: 1024

OR

ORA-01114: IO error writing block to file 3021 (block # 72721) 

ORA-27063: skgfospo: number of bytes read/written is incorrect 

SVR4 Error: 5: I/O error 

Additional information: -1 

Additional information: 258048

OR

Errors in file /oracle/product/diag/asm/+asm/+ASM1/trace/+ASM1_ora_21814.trc:

ORA-27063: number of bytes read/written is incorrect

SVR4 Error: 5: I/O error

Additional information: -1

Additional information: 1048576

WARNING: Write Failed. group:2 disk:6 AU:262074 offset:0 size:1048576

Errors in file /oracle/product/diag/asm/+asm/+ASM1/trace/+ASM1_ora_21814.trc:

ORA-27063: number of bytes read/written is incorrect

SVR4 Error: 5: I/O error

Additional information: -1

Additional information: 1048576

OR

ORA-01115: IO error reading block from file 397 (block # 8633) 

ORA-01110: data file 397: ‘/u04/flexpm/traffic_medium26.dbf’ 

ORA-27063: number of bytes read/written is incorrect 

SVR4 Error: 5: I/O error 

Additional information: -1 

Additional information: 16384

OR

ORA-27063: number of bytes read/written is incorrect

SVR4 Error: 5: I/O error

Additional information: -1

Additional information: 1048576

WARNING: Write Failed. group:2 disk:1 AU:646 offset:0 size:1048576

path:/dev/rdsk/c10t60060E8005BCE5000000BCE500000243d0s0

incarnation:0xf0f46246 asynchronous result:’I/O error’

subsys:System iop:0xffffffff78841f40 bufp:0xffffffff774bf000 osderr:0x0 osderr1:0x0

ORA-15080: synchronous I/O operation to a disk failed

WARNING: failed to write mirror side 1 of virtual extent 32 logical extent 0 of file 2390 in group 2 on disk 1 allocation unit 646 

*** yyyy-mm-dd hh24:mi:ss.sss 4320 krsh.c

ARC5: Encountered disk I/O error 19502

 

CAUSE

This is due to a problem with the I/O subsystem.

Issues of this nature are common when there is a problem in the I/O subsystem.
This can include, but is not limited to:

  • A bad sector on disk
  • An I/O card that is starting to fail
  • A bad array cable
  • An interruption in network connectivity, in the case of NFS mounts
  • Could also be caused by a OS level bug.
  • etc.

Review the OS Messages file as this will almost certainly reflect errors (for example   Error for Command: write(10) )

SOLUTION

Open a ticket with the disk hardware vendor to

  • run diagnostics on the disk where the problematic file resides and
  • investigate / resolve the cause of the I/O error

点赞