Crash in NETSVCS SVCHOST worker thread

[molotov]

Hello,

I am attempting to assist someone with a recurring crash of the NETSVCS instance of SVCHOST.EXE in XP SP2. The faulting thread is an OLE / RPC worker thread. An analysis of the dump follows, with a small amount of my interpretation (whatever that's worth). But I am unsure of what to do / suggest next based on the dump, and would appreciate any pointers.

Code: Select all

0:066> r
eax=00000000 ebx=00007530 ecx=7c8025f0 edx=7c90eb94 esi=0013d908 edi=7c802540
eip=00000000 esp=0093ff98 ebp=0093ffb4 iopl=0         nv up ei pl zr na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
00000000 ??              ???
0:066> !analyze -v
FAULTING_IP:
+0
00000000 ??              ???

EXCEPTION_RECORD:  ffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 00000000
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 00000000
   Parameter[1]: 00000000
Attempt to read from address 00000000

DEFAULT_BUCKET_ID:  NULL_INSTRUCTION_PTR

PROCESS_NAME:  svchost.exe

ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.

READ_ADDRESS:  00000000

FAILED_INSTRUCTION_ADDRESS:
+0
00000000 ??              ???

NTGLOBALFLAG:  0

APPLICATION_VERIFIER_FLAGS:  0

LAST_CONTROL_TRANSFER:  from 774fe3dc to 00000000

STACK_TEXT: 
WARNING: Frame IP not in any known module. Following frames may be wrong.
0093ff94 774fe3dc 00000000 00000000 0013d908 0x0
0093ffa8 774fe444 00000000 0093ffec 7c80b683 ole32!CRpcThread::WorkerLoop+0x1e
0093ffb4 7c80b683 0013d908 00000000 00000000 ole32!CRpcThreadCache::RpcWorkerThreadEntry+0x1b
0093ffec 00000000 774fe429 0013d908 00000000 kernel32!BaseThreadStart+0x37


FAULTING_THREAD:  00000adc

PRIMARY_PROBLEM_CLASS:  NULL_INSTRUCTION_PTR

BUGCHECK_STR:  APPLICATION_FAULT_NULL_INSTRUCTION_PTR

FOLLOWUP_IP:
ole32!CRpcThread::WorkerLoop+1e
774fe3dc ff15e4124e77    call    dword ptr [ole32!_imp__GetCurrentThread (774e12e4)]

SYMBOL_STACK_INDEX:  1

SYMBOL_NAME:  ole32!CRpcThread::WorkerLoop+1e

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: ole32

IMAGE_NAME:  ole32.dll

DEBUG_FLR_IMAGE_TIMESTAMP:  42e5be93

STACK_COMMAND:  ~66s; .ecxr ; kb

FAILURE_BUCKET_ID:  ole32.dll!CRpcThread::WorkerLoop_c0000005_NULL_INSTRUCTION_PTR

BUCKET_ID:  APPLICATION_FAULT_NULL_INSTRUCTION_PTR_NULL_IP_ole32!CRpcThread::WorkerLoop+1e

Followup: MachineOwner
---------


The problem appears to be this line, in CRpcThread::WorkerLoop:

Code: Select all

774fe3d9 ff5608          call    dword ptr [esi+8]

ESI is the pvParam passed to BaseThreadStart; ESI+8 contains 0. So, the instruction seems to be "call 0", which seems to get the thread into the situation that causes the crash - The instruction at "0x00000000" referenced memory at "0x00000000". The memory could not be "read" (Attempt to read from address 00000000)

ESI doesn't contain any information that leads me to guess what pvParam may be / contain:

Code: Select all

0:066> dd esi
0013d908  00000d8c 00000000 00000000 00000000
0013d918  00000000 00000000 0004000d 00540055
0013d928  000901e0 02ed82e8 00590054 0053005c
0013d938  00530059 00450054 0000004d 00740073
0013d948  00610072 00690074 006e006f 00000000
0013d958  00070006 000e013a 006f0043 006e006e
0013d968  00630065 00690074 006e006f 00740053
0013d978  00740061 00730075 00000000 00000000

Ole32 seems legit:

Code: Select all

774e0000 7761d000   ole32      (pdb symbols)          D:\Symbols\ole32.pdb\683B65B246F4418796D2EE6D4C55EB112\ole32.pdb
    Loaded symbol image file: ole32.dll
    Image path: C:\WINDOWS\system32\ole32.dll
    Image name: ole32.dll
    Timestamp:        Mon Jul 25 23:39:47 2005 (42E5BE93)
    CheckSum:         0013DC6B
    ImageSize:        0013D000
    File version:     5.1.2600.2726
    Product version:  5.1.2600.2726
    File flags:       0 (Mask 3F)
    File OS:          40004 NT Win32
    File type:        2.0 Dll
    File date:        00000000.00000000
    Translations:     0409.04b0
    CompanyName:      Microsoft Corporation
    ProductName:      Microsoft® Windows® Operating System
    InternalName:     OLE32.DLL
    OriginalFilename: OLE32.DLL
    ProductVersion:   5.1.2600.2726
    FileVersion:      5.1.2600.2726 (xpsp_sp2_gdr.050725-1528)
    FileDescription:  Microsoft OLE for Windows
    LegalCopyright:   © Microsoft Corporation. All rights reserved.

Any suggestions on what to look at next in the dump would be great!

Thanks,

--molotov

[VDO]

In such cases like this where we have a NULL pointer and valid call stack I usually recommend to contact MS. Is this pattern consistent across multiple crash dumps? If we use user dump control panel or debug monitor to save first-chance exception dumps do they point to the same module and instruction? Are there any first chance exceptions What about changing memory layout: rebasing ole32dll? What about changing the process heap layout? Sometimes it can be data coming from corrupt pointers residing in overwritten heap-allocated buffers and enabling full pageheap could potentially reveal another dormant bugs and different components.

[molotov]

Thanks so much for the response, Dmitry.

Is this pattern consistent across multiple crash dumps?

Yes.

If we use user dump control panel or debug monitor to save first-chance exception dumps do they point to the same module and instruction?

The user has attached a debugger to the NETSVCS instance of SVCHOST.EXE; have not heard any info back. Will suggest some kind of exception-monitor to grab a dump on first-chance exceptions.

What about changing memory layout: rebasing ole32dll?

Interesting suggestion. Would have to deal with WFP, and this change would affect all processes that load ole32.dll. Not that they would necessarily have problems, of course. I'll certainly run it by the user.

enabling full pageheap

I'd actually suggested this to the user in the following fashion, so as to not disrupt the other instances of svchost.exe:
For each service in the NETSVCS group at [HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\SvcHost\netsvcs], change the ImagePath to point to a copy of the SVCHOST.EXE file - SVCHOST_NETSVCS.EXE, for example. Then, set up various GFLAGS for SVCHOST_NETSVCS.EXE, and attach a debugger to SVCHOST_NETSVCS.EXE after rebooting. Initial thought for gflags would be turning on full page heap.

The user was at the time more interested in seeing what details may have been obtained by attaching a debugger outright.

I'll prompt the user for an update, get any details, suggest what you've offered, and see what we get.

Thanks again!

Kind regards,

--molotov