I'm running CentOS 8.1 and my machine has kernel panic'd. I installed the kernel-debuginfo package and I am generally following the steps in Section 7.11 : Analyzing a core dump.
Here is an abbreviated version of my debugging session :
# crash /usr/lib/debug/usr/lib/modules/4.18.0-147.el8.x86_64/vmlinux /var/crash/XXX/vmcore
.
.
.
WARNING: kernel relocated [336MB]: patching 93296 gdb minimal_symbol values
KERNEL: /usr/lib/debug/usr/lib/modules/4.18.0-147.el8.x86_64/vmlinux
DUMPFILE: /var/crash/XXX/vmcore [PARTIAL DUMP]
CPUS: 48
DATE: Sun Jan 10 13:36:04 2021
UPTIME: 23 days, 22:18:40
LOAD AVERAGE: 10.00, 10.01, 10.00
TASKS: 1966
NODENAME: YYY
RELEASE: 4.18.0-147.el8.x86_64
VERSION: #1 SMP Wed Dec 4 21:51:45 UTC 2019
MACHINE: x86_64 (2794 Mhz)
MEMORY: 2035.9 GB
PANIC: "Kernel panic - not syncing: Hard LOCKUP"
PID: 27666
COMMAND: "R"
TASK: ffff8ff017978000 [THREAD_INFO: ffff8ff017978000]
CPU: 2
STATE: TASK_RUNNING (PANIC)
crash> bt
PID: 27666 TASK: ffff8ff017978000 CPU: 2 COMMAND: "R"
#0 [ffffb5187c6c7a50] machine_kexec at ffffffff96057c4e
#1 [ffffb5187c6c7aa8] __crash_kexec at ffffffff96155b8d
#2 [ffffb5187c6c7b70] panic at ffffffff960b0578
#3 [ffffb5187c6c7bf8] watchdog_overflow_callback.cold.8 at ffffffff9618bb11
#4 [ffffb5187c6c7c08] __perf_event_overflow at ffffffff961f54f2
#5 [ffffb5187c6c7c38] x86_pmu_handle_irq at ffffffff96007a16
#6 [ffffb5187c6c7e88] amd_pmu_handle_irq at ffffffff96008b14
#7 [ffffb5187c6c7ea0] perf_event_nmi_handler at ffffffff960060cd
#8 [ffffb5187c6c7eb8] nmi_handle at ffffffff96021843
#9 [ffffb5187c6c7f10] default_do_nmi at ffffffff96021cce
#10 [ffffb5187c6c7f30] do_nmi at ffffffff96021ea8
#11 [ffffb5187c6c7f50] nmi at ffffffff96a01537
RIP: 0000146816a69d6e RSP: 00007ffc378f6270 RFLAGS: 00000216
RAX: 000000006655e8df RBX: 0000000000000003 RCX: 000000000003ad90
RDX: 0000000000000003 RSI: 0000000007ccb060 RDI: 000000076fe90140
RBP: 0000000000085fc6 R8: 0000000000b13039 R9: 00000007770c0758
R10: 0000000778ba8a38 R11: 0000000000000c36 R12: 000000000070e070
R13: 0000000000000000 R14: 00007ffc378f6410 R15: 00007ffc378f6430
ORIG_RAX: ffffffffffffffff CS: 0033 SS: 002b
Clearly the culprit is an R process which I was running (see COMMAND: "R"). Looking at the bt above, it seems like it is only returning the kernel level functions. I want to know what line in my R code (or installed R libraries) causing the issue. Trying
crash> gdb bt
No stack.
gdb: gdb request failed: bt
crash>
is not useful. Looking at man crash it isn't directly obvious how to do that. Some of the R libraries possibly involved have C++ code that was compiled with debug flags. I have a vague hope of the cause of the error being in one of these.
QUESTION :
How do I use the Linux crash utility to recover the line of code (either R or C++) that caused the kernel to panic?
What is a "Kernel panic - not syncing: Hard LOCKUP"
I am using node 12.14.1, karma 4.4.1 and sinon 7.5.0
I Want to update sinon to the latest version (8.1.1), but I got this error:
28 01 2020 12:03:52.901:INFO [karma-server]: Karma v4.4.1 server started at http://0.0.0.0:9876/
28 01 2020 12:03:52.904:INFO [launcher]: Launching browsers PhantomJS with concurrency unlimited
28 01 2020 12:03:53.045:INFO [launcher]: Starting browser PhantomJS
28 01 2020 12:03:56.249:INFO [PhantomJS 2.1.1 (Windows 8.0.0)]: Connected on socket nWUU_e4J9lEni4AFAAAA with id 124
2
PhantomJS 2.1.1 (Windows 8.0.0) ERROR
ReferenceError: Can't find variable: Map
at node_modules/sinon/pkg/sinon.js:4113:35
PhantomJS 2.1.1 (Windows 8.0.0) ERROR
ReferenceError: Can't find variable: Map
at node_modules/sinon/pkg/sinon.js:4113:35
What do I need to change or include? I tried to include es6-shim but that does not work.
Steps to reproduce
dotnet build or dotnet run
Expected behavior
Run or Build app
Actual Behavior
Getting ready...
The template "ASP.NET Core with Angular" was created successfully.
Processing post-creation actions...
Running 'dotnet restore' on /home/limup/Documents/Projetos/Limup/salao/salao.csproj...
/usr/share/dotnet/sdk/3.1.101/NuGet.targets(123,5): error : Unable to obtain lock file access on '/tmp/NuGetScratch/lock/b19d3901039706ea82571abad7c98ec690508d4b' for operations on '/home/limup/Documents/Projetos/Limup/salao/obj/salao.csproj.nuget.cache'. This may mean that a different user or administator is holding this lock and that this process does not have permission to access it. If no other process is currently performing an operation on this file it may mean that an earlier NuGet process crashed and left an inaccessible lock file, in this case removing the file '/tmp/NuGetScratch/lock/b19d3901039706ea82571abad7c98ec690508d4b' will allow NuGet to continue. [/home/limup/Documents/Projetos/Limup/salao/salao.csproj]
Restore failed.
Post action failed.
Description: Restore NuGet packages required by this project.
Manual instructions: Run 'dotnet restore'
Environment Data
dotnet --info
.NET Core SDK (reflecting any global.json):
Version: 3.1.101
Commit: b377529961
Runtime Environment:
OS Name: fedora
OS Version: 31
OS Platform: Linux
RID: fedora.31-x64
Base Path: /usr/share/dotnet/sdk/3.1.101/
Host (useful for support):
Version: 3.1.1
Commit: a1388f194c
.NET Core SDKs installed:
3.1.101 [/usr/share/dotnet/sdk]
.NET Core runtimes installed:
Microsoft.AspNetCore.App 3.1.1 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 3.1.1 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
To install additional .NET Core runtimes or SDKs:
https://aka.ms/dotnet-download
Obs
Tried Fixes put in dotnet restore, but I received the same error.
Did not have this problem with dotnet sdk 2.0.
In my case, the problem was caused by ownership of the "lock file" (in Linux).
I was running dotnet build under my user (without sudo) but my project was created using sudo.
Option A)
Use sudo again
sudo dotnet build
Option B)
Change /tmp/NuGetScratch/lock/ ownership:
sudo chown -R <user>:<user> /tmp/NuGetScratch/
Then, the user can run dotnet build without sudo.
I fixed that bug with commands below:
export TMPDIR=/tmp/NuGetScratch/
mkdir -p ${TMPDIR}
but, I've been receive the other error and I opened other question post: Post
I was getting the same error in my Visual Studio 2022 when I was trying to build my new console project.
Severity Code Description Project File Line Suppression State
Error Error occurred while restoring NuGet packages: Unable to obtain
lock file access on for operations on 'NuGet.Config'. This may mean
that a different user or administrator is holding this lock and that
this process does not have permission to access it. If no other
process is currently performing an operation on this file it may mean
that an earlier NuGet process crashed and left an inaccessible lock
file, in this case removing the file will allow NuGet to continue.
The very easy fix you can do is to open your Visual Studio as an administrator and build your project.
#Omair Majid, see:
[limup#localhost tmp]$ ll -a
total 12
drwxrwxrwt. 22 root root 480 Jan 15 16:59 .
dr-xr-xr-x. 18 root root 4096 Jan 10 18:51 ..
drwx------. 2 limup limup 60 Jan 15 16:54 .esd-1000
drwxrwxrwt. 2 root root 40 Jan 15 16:49 .font-unix
drwxrwxrwt. 2 root root 60 Jan 15 16:54 .ICE-unix
drwx------. 3 root root 60 Jan 15 16:49 systemd-private-5f8b1b99edb949b1864fa2e580380675-adb.service-WzMSag
drwx------. 3 root root 60 Jan 15 16:49 systemd-private-5f8b1b99edb949b1864fa2e580380675-chronyd.service-Rw6V8i
drwx------. 3 root root 60 Jan 15 16:50 systemd-private-5f8b1b99edb949b1864fa2e580380675-colord.service-9QFuBh
drwx------. 3 root root 60 Jan 15 16:49 systemd-private-5f8b1b99edb949b1864fa2e580380675-dbus-broker.service-DqpFAf
drwx------. 3 root root 60 Jan 15 16:54 systemd-private-5f8b1b99edb949b1864fa2e580380675-fwupd.service-eBNBLh
drwx------. 3 root root 60 Jan 15 16:54 systemd-private-5f8b1b99edb949b1864fa2e580380675-geoclue.service-O6YLYg
drwx------. 3 root root 60 Jan 15 16:49 systemd-private-5f8b1b99edb949b1864fa2e580380675-ModemManager.service-hHCfHf
drwx------. 3 root root 60 Jan 15 16:49 systemd-private-5f8b1b99edb949b1864fa2e580380675-rtkit-daemon.service-3A8e9f
drwx------. 3 root root 60 Jan 15 16:49 systemd-private-5f8b1b99edb949b1864fa2e580380675-switcheroo-control.service-zSjrXg
drwx------. 3 root root 60 Jan 15 16:49 systemd-private-5f8b1b99edb949b1864fa2e580380675-systemd-logind.service-ohP23f
drwx------. 3 root root 60 Jan 15 16:50 systemd-private-5f8b1b99edb949b1864fa2e580380675-upower.service-kMAiIg
drwx------. 2 limup limup 40 Jan 15 16:58 Temp-31248c7a-04ad-429c-ab49-4c2ed74a1986
drwx------. 2 limup limup 40 Jan 15 16:58 Temp-8fa994cb-0334-4cf9-886d-103347596108
drwxrwxrwt. 2 root root 40 Jan 15 16:49 .Test-unix
drwx------. 2 limup limup 40 Jan 15 17:00 tracker-extract-files.1000
-r--r--r--. 1 limup limup 11 Jan 15 16:54 .X0-lock
-r--------. 1 gdm gdm 11 Jan 15 16:50 .X1024-lock
drwxrwxrwt. 2 root root 80 Jan 15 16:54 .X11-unix
drwxrwxrwt. 2 root root 40 Jan 15 16:49 .XIM-unix
I developed a C++ application that among others use SDL2.
It compiles and run on Ubuntu 18.04 (64 bit machine) and on OSX.
When I try to compile the application on a computer on chip running Ubuntu Mate 18.04 in a 32 bits machine with the libraries compiled by me, it returns an exception.
The exception is happening inside the line SDL_Init( SDL_INIT_VIDEO ) and the message is the following:
dbus[3856]: arguments to dbus_message_new_method_call were incorrect, assertion "path != NULL" failed in file ../../../dbus/dbus-message.c ine 1362.
This is normally a bug in some application using D-Bus library.
D-Bus not built with -rdynamic so unable to print backtrace.
Aborted
It seems a problem happening on Ubuntu running on a 32 bit machine. Have anyone had the same problem?
How can I solve this problem? Or does anyone know what is generating it?
NOTE 1
Running in gdb and using backtrace as suggested:
#0 0xb613d206 in __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:47
#1 0xb614ab32 in __libc_signal_restore_set (set=0xbeffe3a4) at ../sysdeps/unix/sysv/linux/nptl-signals.h:80
#2 0xb614ab32 in __GI_raise (sig=sig#entry=6) at ../sysdeps/unix/sysv/linux/raise.c:48
#3 0xb614b82e in __GI_abort () at abort.c:79
#4 0xb2adddf0 in _dbus_abort () at /lib/arm-linux-gnueabihf/libdbus-1.so.3
#5 0xb2ad7c5a in _dbus_warn_check_failed () at /lib/arm-linux-gnueabihf/libdbus-1.so.3
#6 0xb2ad8104 in _dbus_warn_return_if_fail () at /lib/arm-linux-gnueabihf/libdbus-1.so.3
#7 0xb2acce80 in dbus_message_new_method_call () at /lib/arm-linux-gnueabihf/libdbus-1.so.3
#8 0xb68acdba in () at /usr/lib/arm-linux-gnueabihf/libSDL2-2.0.so.0
NOTE 2
The same happens if I install SDL from the repository instead of building it.
I found a partial solution so anybody with a better one would be welcome.
As #Keltar suggested in the comment I debugged it with gdb and backtrace as already said in the question. The last line of backtrace got me suspicious:
#8 0xb68acdba in () at /usr/lib/arm-linux-gnueabihf/libSDL2-2.0.so.0
which suggested it is not taking the library built by me but the one installed with apt. I checked all the paths in Makefile and everything was pointing to my custom built. So I checked the dynamic library files *.so and I found the situation was the following:
drwxr-xr-x 3 odroid odroid 4096 Oct 15 12:21 cmake
lrwxrwxrwx 1 odroid odroid 17 Oct 15 12:21 libSDL2-2.0d.so -> libSDL2-2.0d.so.0
lrwxrwxrwx 1 odroid odroid 21 Oct 15 12:21 libSDL2-2.0d.so.0 -> libSDL2-2.0d.so.0.8.0
-rw-r--r-- 1 odroid odroid 4317020 Oct 15 12:21 libSDL2-2.0d.so.0.8.0
lrwxrwxrwx 1 odroid odroid 15 Oct 15 12:40 libSDL2-2.0.so -> libSDL2-2.0d.so
-rw-r--r-- 1 odroid odroid 7593354 Oct 15 12:21 libSDL2d.a
-rw-r--r-- 1 odroid odroid 4754 Oct 15 12:19 libSDL2maind.a
lrwxrwxrwx 1 odroid odroid 14 Oct 15 13:38 libSDL2.so -> libSDL2-2.0.so
drwxr-xr-x 2 odroid odroid 4096 Oct 15 12:21 pkgconfig
Note that for the symbolic link file libSDL2.so -> libSDL2-2.0.so, the linked file libSDL2-2.0.so was not created.
My solution was to rebuild the library in Release mode instead of Debug using cmake -DCMAKE_BUILD_TYPE=Release: in that way all the files library are correct. Another solution might be to symlink the file(s) manually but is is easy to mess up something.
So it seems a bug in how SDL cmake write the instructions for the Debug build type because in release mode my program run.
If anyone has a better solution or find any error that I could not find please let me know.
I am analyzing a hanging ASP.NET MVC website problem running under a 64 bit ASP.NET v4.0 AppPool in IIS 7.5, Windows 2008 R2 64. I have taken a dump through taskmgr, and am analysing in WinDBG x64 on Windows 7 64bit.
When running !analyze -v I see the error WRONG_SYMBOLS and BUGCHECK_STR: APPLICATION_FAULT_WRONG_SYMBOLS. This is hindering my debugging as I cannot check some threads that I need to investigate for deadlocking.
Details
My symbol server is SRV*D:\DOWNLOADEDSYMBOLS*http://msdl.microsoft.com/download/symbols
I have deleted this local folder and allowed all symbols to be download from m$.
I have loaded sos with .loadby sos clr
Output from !analyze -v:
*******************************************************************************
* *
* Exception Analysis *
* *
*******************************************************************************
FAULTING_IP:
+0
00000000`00000000 ?? ???
EXCEPTION_RECORD: ffffffffffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 0000000000000000
ExceptionCode: 80000003 (Break instruction exception)
ExceptionFlags: 00000000
NumberParameters: 0
FAULTING_THREAD: 0000000000000544
DEFAULT_BUCKET_ID: WRONG_SYMBOLS
PROCESS_NAME: w3wp.exe
ERROR_CODE: (NTSTATUS) 0x80000003 - {EXCEPTION} Breakpoint A breakpoint has been reached.
EXCEPTION_CODE: (HRESULT) 0x80000003 (2147483651) - One or more arguments are invalid
NTGLOBALFLAG: 0
APPLICATION_VERIFIER_FLAGS: 0
APP: w3wp.exe
MANAGED_STACK: !dumpstack -EE
OS Thread Id: 0x944 (15)
Current frame:
Child-SP RetAddr Caller, Callee
PRIMARY_PROBLEM_CLASS: WRONG_SYMBOLS
BUGCHECK_STR: APPLICATION_FAULT_WRONG_SYMBOLS
LAST_CONTROL_TRANSFER: from 000007fefd8910dc to 000000007712135a
STACK_TEXT:
00000000`000afac8 000007fe`fd8910dc : 00000000`00000000 00000000`00000000 000007fe`f9921630 000007fe`fadb7f66 : ntdll!NtWaitForSingleObject+0xa
00000000`000afad0 000007fe`f99241bc : 00000000`00000000 00000000`ffaf6de0 00000000`00000000 00000000`00000128 : KERNELBASE!WaitForSingleObjectEx+0x79
00000000`000afb70 00000000`ffaf3c60 : 00000000`fffffffe 00000000`00415f90 00000000`ffaf4588 000007fe`f9920000 : w3wphost!AppHostInitialize+0x278
00000000`000afbd0 00000000`ffaf11f1 : 00000000`00000000 00000000`ffaf1351 00000000`00000000 000003fd`deed0e35 : w3wp!wmain+0x470
00000000`000afd60 00000000`76e6652d : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : w3wp!PerfStopProvider+0x19b
00000000`000afda0 00000000`770fc521 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!BaseThreadInitThunk+0xd
00000000`000afdd0 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x1d
STACK_COMMAND: ~0s; .ecxr ; kb
FOLLOWUP_IP:
w3wphost!AppHostInitialize+278
000007fe`f99241bc f605e1b4000003 test byte ptr [w3wphost!g_dwDebugFlags (000007fe`f992f6a4)],3
SYMBOL_STACK_INDEX: 2
SYMBOL_NAME: w3wphost!AppHostInitialize+278
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: w3wphost
IMAGE_NAME: w3wphost.dll
DEBUG_FLR_IMAGE_TIMESTAMP: 4ce7c9ef
FAILURE_BUCKET_ID: WRONG_SYMBOLS_80000003_w3wphost.dll!AppHostInitialize
BUCKET_ID: X64_APPLICATION_FAULT_WRONG_SYMBOLS_w3wphost!AppHostInitialize+278
WATSON_STAGEONE_URL: http://watson.microsoft.com/StageOne/w3wp_exe/7_5_7601_17514/4ce7afa2/unknown/0_0_0_0/bbbbbbb4/80000003/00000000.htm?Retriage=1
Followup: MachineOwner
---------
Output from .chain:
Extension DLL search Path:
C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x64\WINXP;C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x64\winext;C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x64\winext\arcade;C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x64\pri;C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x64;C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x64\winext\arcade;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Microsoft SQL Server\100\Tools\Binn\;C:\Program Files\Microsoft SQL Server\100\Tools\Binn\;C:\Program Files\Microsoft SQL Server\100\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\100\Tools\Binn\VSShell\Common7\IDE\;C:\Program Files (x86)\Microsoft SQL Server\100\DTS\Binn\;C:\Program Files (x86)\Intel\Services\IPT\;C:\Program Files\Intel\WiFi\bin\;C:\Program Files\Common Files\Intel\WirelessCommon\;C:\Program Files (x86)\Microsoft Team Foundation Server 2012 Power Tools\;C:\Program Files (x86)\Microsoft Team Foundation Server 2012 Power Tools\Best Practices Analyzer\;C:\Program Files (x86)\Microsoft SQL Server\110\Tools\Binn\;C:\Program Files\Microsoft SQL Server\110\Tools\Binn\;C:\Program Files\Microsoft SQL Server\110\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\110\Tools\Binn\ManagementStudio\;C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Binn\;C:\Program Files\Microsoft\Web Platform Installer\;C:\Program Files (x86)\Microsoft ASP.NET\ASP.NET Web Pages\v1.0\;C:\Program Files (x86)\Windows Kits\8.0\Windows Performance Toolkit\
Extension DLL chain:
D:\DOWNLOADEDSYMBOLS\sos_AMD64_AMD64_4.0.30319.18034.dll\50B5A78395e000\sos_AMD64_AMD64_4.0.30319.18034.dll: image 4.0.30319.18034, API 1.0.0, built Wed Nov 28 18:45:59 2012
[path: D:\DOWNLOADEDSYMBOLS\sos_AMD64_AMD64_4.0.30319.18034.dll\50B5A78395e000\sos_AMD64_AMD64_4.0.30319.18034.dll]
sosex: image 4.5.0.0, API 1.0.0, built Thu Oct 04 03:57:55 2012
[path: C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x64\sosex.dll]
D:DOWNLOADEDSYMBOLS\sos_AMD64_AMD64_4.0.30319.18034.dll\50B5A78395e000\sos_AMD64_AMD64_4.0.30319.18034.dll: image 4.0.30319.18034, API 1.0.0, built Wed Nov 28 18:45:59 2012
[path: D:\DOWNLOADEDSYMBOLS\sos_AMD64_AMD64_4.0.30319.18034.dll\50B5A78395e000\sos_AMD64_AMD64_4.0.30319.18034.dll]
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\sos: image 4.0.30319.18034, API 1.0.0, built Wed Nov 28 18:45:59 2012
[path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\sos.dll]
dbghelp: image 6.2.9200.20512, API 6.2.6, built Fri Sep 07 17:45:49 2012
[path: C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x64\dbghelp.dll]
ext: image 6.2.9200.20522, API 1.0.0, built Fri Sep 21 20:17:05 2012
[path: C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x64\winext\ext.dll]
exts: image 6.2.9200.16384, API 1.0.0, built Thu Jul 26 14:15:20 2012
[path: C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x64\WINXP\exts.dll]
uext: image 6.2.9200.16384, API 1.0.0, built Thu Jul 26 14:15:09 2012
[path: C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x64\winext\uext.dll]
ntsdexts: image 6.2.9200.16384, API 1.0.0, built Thu Jul 26 14:16:01 2012
[path: C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x64\WINXP\ntsdexts.dll]
Output from .reload with !sym noisy
DBGHELP: ntdll - public symbols
d:\downloadedsymbols\ntdll.pdb\15EB43E23B12409C84E3CC7635BAF5A32\ntdll.pdb
..............................................................
DBGHELP: KERNELBASE - public symbols
d:\downloadedsymbols\kernelbase.pdb\91C72371DD43448192B7B46F5ED10AA02\kernelbase.pdb
Deadlock Analysis
Thread List
Output from !threads:
ID OSID ThreadOBJ State GC Mode GC Alloc Context Domain Lock Count Apt Exception
7 1 7a8 0000000001f92a50 28220 Preemptive 0000000000000000:0000000000000000 000000000119ee70 0 Ukn
13 2 1764 0000000001fa9fb0 2b220 Preemptive 0000000102D5DCC8:0000000102D5F7E8 000000000119ee70 0 MTA (Finalizer)
15 3 944 0000000001ffa010 102a220 Preemptive 0000000000000000:0000000000000000 000000000119ee70 0 MTA (Threadpool Worker)
16 4 bb0 0000000002006ec0 21220 Preemptive 0000000000000000:0000000000000000 000000000119ee70 0 Ukn
6 15 1018 00000000086704d0 20220 Preemptive 0000000000000000:0000000000000000 000000000119ee70 0 Ukn
35 16 12e4 00000000098fc820 202b220 Preemptive 000000010400E5A0:00000001040102B0 0000000002005e90 1 MTA
....etc
Output from !dlk:
*DEADLOCK DETECTED*
CLR thread 0x15 holds the lock on SyncBlock 0000000009913cf8 OBJ:00000000ffe31898
[System.Collections.Generic.Dictionary``2[[System.String, mscorlib],[System.Resources.ResourceLocator, mscorlib]]]
...and is waiting for the lock on SyncBlock 0000000009913ca8 OBJ:00000000ffe318e8[System.Resources.ResourceReader]
CLR thread 0x16 holds the lock on SyncBlock 0000000009913ca8 OBJ:00000000ffe318e8[System.Resources.ResourceReader]
...and is waiting for the lock on SyncBlock 0000000009913cf8 OBJ:00000000ffe31898
[System.Collections.Generic.Dictionary``2[[System.String, mscorlib],[System.Resources.ResourceLocator, mscorlib]]]
CLR Thread 0x15 is waiting at System.Resources.ResourceReader.AllocateStringForNameIndex(Int32, Int32 ByRef)(+0x17 IL,+0xc9 Native)
CLR Thread 0x16 is waiting at System.Resources.RuntimeResourceSet.GetObject(System.String, Boolean, Boolean)(+0xee IL,+0x375 Native)
Analysis of thread 15
Output from ~6s:
ntdll!ZwDelayExecution+0xa:
00000000`771213aa c3 ret
.. and then output from !clrstack:
OS Thread Id: 0x1018 (6)
Child SP IP Call Site
GetFrameContext failed: 1
0000000000000000 0000000000000000 <unknown>
Analysis of thread 16
Output from ~35s:
ntdll!NtWaitForMultipleObjects+0xa:
00000000`771218ca c3 ret
Output from !clrstack:
OS Thread Id: 0x12e4 (35)
Child SP IP Call Site
000000000cc9e188 00000000771218ca [HelperMethodFrame_1OBJ: 000000000cc9e188] System.Threading.WaitHandle.WaitMultiple(System.Threading.WaitHandle[], Int32, Boolean, Boolean)
000000000cc9e2c0 000007fef65e562c System.Threading.WaitHandle.WaitAny(System.Threading.WaitHandle[], Int32, Boolean)
000000000cc9e320 000007fef5670d06 System.Net.TimerThread.ThreadProc()
000000000cc9e3f0 000007fef65a0a05 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
000000000cc9e550 000007fef65a0769 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
000000000cc9e580 000007fef65a0727 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
000000000cc9e5d0 000007fef65b3e81 System.Threading.ThreadHelper.ThreadStart()
000000000cc9e8e8 000007fef9ec07f3 [GCFrame: 000000000cc9e8e8]
000000000cc9ec18 000007fef9ec07f3 [DebuggerU2MCatchHandlerFrame: 000000000cc9ec18]
000000000cc9edf8 000007fef9ec07f3 [ContextTransitionFrame: 000000000cc9edf8]
000000000cc9efe8 000007fef9ec07f3 [DebuggerU2MCatchHandlerFrame: 000000000cc9efe8]
Ignore that error. !analyze -v is only useful for crash dumps (well there are some parameters I believe you can pass to it for things other than crash dumps). What it is reporting is that the reason for the "crash" is that a debug point was hit. Which is true, because when you dump a process, it injects a thread in to the process and causes a break point to be hit, and then dumps the memory for the process. So what the analyze is telling you is that the reason for the "crash" is that a break point was hit. It can be safely ignored. You should be able to see all your threads when you go to look at them. Everything else should work normally.
I think you just misinterpreted the output of the !blk command which is giving us the CLR Thread ID of offending threads and not their ordinal (which is expected by the ~s command).
To get the ordinal of these two threads you can use the output of the !threads command.
From your example :
ID OSID ThreadOBJ State GC Mode GC Alloc Context Domain Lock Count Apt Exception
7 1 7a8 0000000001f92a50 28220 Preemptive 0000000000000000:0000000000000000 000000000119ee70 0 Ukn
7 is the ordinal, 0x1 the CLR Thread ID, 0x7a8 the OS Thread ID