Diagnostics Science

Diagnostics is the mother of problem solving.

All areas of human activity involve the use of diagnostics. Proper diagnostics identifies the right problems to solve. We are now a part of a non-profit organization dedicated to the developing and promoting the application of such diagnostics: systemic and pattern-oriented (pattern-driven and pattern-based).

Software Codiagnostics

Software diagnostics is rarely a straightforward process of extracting the list of diagnostic indicators from software execution artifacts. Usually, it involves artifact transformation through trace and log analysis patterns.

Consider a very large software log. Simple inspection if its trace messages may point to some problem patterns:

However, different log transformations via trace analysis patterns may reveal additional problem patterns:

We call such transformations Software Codiagnostics or Data Codiagnostics in general for pattern-oriented data analysis. The prefix co- signifies cooperative processes and also the fact that such transformations are dual (by analogy with dual categories in mathematics) to diagnostic processes especially when such transformations are reversible (or partially reversible):

Existential Prognostics: Periodic Table of Diagnostic Patterns

One of the features of the Periodic Table of Elements was the prediction of missing elements. In November 2010 we announced the discovery of the Periodic Table of Software Defects as “rules that make it possible to devise a memory dump and software trace analysis equivalent of the Periodic Table of Elements in Chemistry. It allows prediction of abnormal software behavior and structural defects and what patterns to look for after deploying software and collecting its artifacts”.

The publication of the second edition of Encyclopedia of Crash Dump Analysis Patterns makes it possible to see what patterns are expected in your favorite operating system and software product even if they have not been observed or cataloged yet (see its Table of Contents). This is why we call this type of prognostics existential as affirming or implying the existence of a diagnostic pattern, whether it is a problem pattern or problem analysis pattern.

As an example, we can tell the story of pattern prediction and discovery. An engineer expressed the doubt about the existence of Lateral Damage crash dump analysis pattern for Linux systems since he had never observed it during his diagnostic practice. Years passed, and it was recently observed and cataloged when analyzing Linux process core dumps.

Encyclopedia of Crash Dump Analysis Patterns

The following direct links can be used to order the book now:

New! The second edition is now available in PDF format from Software Diagnostics Services

The first edition is available for SkillSoft Books24x7 subscribers

This reference reprints with corrections, additional comments, and classification 373 alphabetically arranged and cross-referenced memory analysis patterns originally published in Memory Dump Analysis Anthology volumes 1 – 9 including 5 analysis patterns from volume 10a. This pattern catalog is a part of pattern-oriented software diagnostics, forensics, prognostics, root cause analysis, and debugging developed by Software Diagnostics Institute (DumpAnalysis.org + TraceAnalysis.org). Most of the analysis patterns are illustrated with examples for WinDbg from Debugging Tools for Windows with a few examples from Mac OS X and Linux for GDB. The second edition includes more than 50 new analysis patterns and more than 70 new examples and comments for analysis patterns published in the first edition.

Product information:

  • Title: Encyclopedia of Crash Dump Analysis Patterns: Detecting Abnormal Software Structure and Behavior in Computer Memory, Second Edition
  • Authors: Dmitry Vostokov, Software Diagnostics Institute
  • Language: English
  • Product Dimensions: 24.6 x 18.9
  • Paperback: 1,200 pages
  • Publisher: OpenTask (March 2017)
  • ISBN-13: 978-1-908043-83-2

Table of Contents

Categorical Foundations of Software Diagnostics

Since extracting information about behaviour from states is a coalgebra* (in our case, we have a behaviour functor from software execution artefacts such as memory snapshots and logs to diagnostic indicators that form concrete problem patterns) we decided to recast pattern-oriented software diagnostics in category theory language terms.

We introduce the following categories:

  • Concrete Execution Artefacts: Category CArtefacts

Example: 3 memory dumps of Windows process with monotonically increasing size. 3 objects from CArtefacts category.

  • Concrete Problem Patterns: Category CProblemPatterns

Example: 3 instances of monotonically increased Windows process heap allocations from specific modules.

  • Concrete Analysis Pattern: Functor FAnalysisPattern

Example: Memory Leak (Process Heap, Windows) specifies the analysis process.

  • Concrete Analysis Patterns: Category CAnalysisPatterns with natural transformations between functors.

Some functors may be similar, for example, Memory Leaks from different platforms. There exists a natural transformation between them. Such natural transformations are called General Analysis Patterns. They form a 2-category.

Some objects from CProblemPatterns may be similar. There exist “generalising” arrows between. The collection of such arrows forms a 2-category of General Problem Patterns.

This is a bottom-up approach. A top-down approach is possible when we start with general categories and select concrete subcategories inside. However, we think in the bottom-up approach general categories arise naturally and correspond to principles of pattern-based part of pattern-oriented diagnostics.

The following diagram illustrated concrete software diagnostics categories:

* Bart Jacobs, Introduction to Coalgebra: Towards Mathematics of States and Observation (ISBN: 978-1107177895)

Reducing Analysis Pattern Complexity via Elementary Analysis Patterns

There are hundreds of debugger commands, such as commands from WinDbg, GDB, LLDB, and other debuggers. A typical diagnostic analysis pattern, for example, a memory dump analysis pattern, may involve many commands (Debugged! Magazine, March 2009). In November 2008 we proposed abstract debugging commands for common diagnostic, forensic, and debugging tasks. After the introduction of pattern-oriented diagnostic thinking we propose another analysis pattern abstraction level of Elementary Analysis Patterns that groups either real or abstract debugging commands and allows chaining analysis activities to uniformly describe diagnostic analysis patterns:

    Typical candidates include the following draft elementary analysis patterns for the new analysis pattern catalog:

  • Setting Symbolic Information
  • Listing Processes
  • Setting Processes
  • Listing Threads
  • Setting Threads
  • Listing Modules
  • Listing Memory Regions
  • Dumping Memory
  • Listing Object Names
  • Counting Objects
  • Dumping Object State
  • Checking Instrumentation
  • Listing Heaps
  • Listing Heap Entries
  • Listing Stack Traces
  • Listing CPUs
  • Dumping Thread State
  • Searching Memory
  • ...

Some Elementary Analysis Patterns may correspond to a single WinDbg command, and some may group several debugger commands. The exact names will be incrementally added and incrementally refined over the course of catalog building process.

Some Elementary Analysis Patterns may be reused across different analysis pattern catalogs, for example, Setting Symbolic Information is also applicable to trace and log analysis, for example, Windows ETW traces (see No Trace Metafile analysis pattern) and Fiber Bundle analysis pattern where symbols are needed for associated stack traces or Adjoint Space where symbols are needed for associated memory snapshots.

Such analysis patterns are different from Elementary Software Diagnostics Patterns since the latter are about observed behavioral signs, but the former are about the analysis process.

Let’s look at one example. We observe increasing memory Counter Values for one Windows server process and look at its memory snapshots. The process doesn’t use .NET so we use process heap Memory Leak analysis pattern which can be split into the following sequentially applied Elementary Analysis Patterns that group appropriate WinDbg commands:

  1. Setting Symbols (.symfix, .sympath+)
  2. Checking Instrumentation (!gflag)
  3. Listing Heaps (!heap -s)
  4. Listing Heap Entries (!heap -k -h)

In the case of missing user mode stack trace database or before asking for it, we can also use the following Elementary Analysis Patterns for symbolic hints such as Module Hint analysis pattern, ASCII, and UNICODE data:

  1. Dumping Memory (dc, dps, dpS, dpa, dpu) for heap entries
  2. Searching Memory (s-sa, s-su) for heap entries

All these provide a better description of analysis patterns. The same approach can be applied to trace and log analysis including network trace analysis, memory forensics, reversing and malware analysis.

2017 – The Year of Theoretical Software Diagnostics

2016 was a pivotal year for pattern-oriented software diagnostics with its mathematical turn and the birth of theoretical software diagnostics discipline. We look ahead to 2017 with more theoretical articles, descriptions of diagnostic analysis patterns, and books already in the pipeline.

Happy New Year!
Software Diagnostics Institute

Principles of Pattern-Oriented Software Data Analysis

2016 is closing, and for 2017 we look forward to applying software diagnostics and software post-construction problem-solving insights gained over the 10 years of Software Diagnostics Institute research to software data analysis in general. In addition to memory snapshots (Dump Artefacts) and logs (Trace Artefacts) and their analysis that we abbreviated as DA+TA, we extend our pattern-oriented approach to additional artifacts as security data, source code, configuration data, telemetry, revision repositories, and stores. We consider all additional software data types as examples of generalized software narratives and traces and abbreviate as simply DATA.

It is time to bring together the principles of pattern-oriented data analysis:

1. Patterns-based

Data patterns and data analysis patterns are classified into catalogs and named to form pattern languages. Every major kind of software data and activity associated with its data analysis can have its own pattern language and catalog. Pattern catalogs are dynamic structures. New patterns are added, old are revised. New catalogs are refined, added, or combined. Pattern names change if necessary to accommodate new data meta-analysis insights (Pattern-Based Software Diagnostics). Patterns can be reused across different data domains.

2. Patterns-driven

Data patterns that are diagnosed using data analysis patterns that are guided by meta-patterns trigger appropriate actionable decisions (Pattern-Driven Software Diagnostics).

3. Systemic-based

Data analysis is a multidisciplinary activity incorporating insights from natural and medical sciences, humanities and social sciences (Systemic Software Diagnostics).

4. Mechanisms-based

The pattern-oriented data analysis may lead to data root cause analysis when coupled with mechanisms.

5. Narrative-based

Software data is a form of a software narrative including data analysis itself (the higher-order narrative analysis).

6. Pattern square-based

There are special and general data patterns and special and general data analysis patterns (Pattern Square).

7. Patterns-assisted

Since software usage is a human activity, software data analysis should be human-assisted. Data analysis patterns facilitate data analysis verifiability, elimination of data analysis errors, and provide independence of data analysis reporting from idiosyncratic data analysis habits*. Software data and data analysis patterns and their languages assist humans in achieving and maintaining software data analysis quality.

* B. Russo, The need for data analysis patterns (in software engineering), Perspectives on Data Science for Software Engineering (ISBN: 978-0128042069)

Pattern-Oriented Diagnostic Analysis Process

Previously we introduced Pattern-Oriented Debugging Process where software diagnostics played the major role as a part of debugging. However, in the case of a separate software diagnostic process, we introduce Pattern-Oriented Diagnostic Analysis Process, that incorporates diagnostic analysis requirements elicitation from problem description analysis and diagnostic report construction. Both new additions require separate pattern catalogs. Problem description analysis pattern catalog is already being extended, and the new catalog for diagnostic report construction is under development and will be published soon. The central process part, diagnostic artifact analysis, already has two extensive analysis pattern catalogs for software log analysis and memory analysis. The process is illustrated in the following diagram:

Software Trace and Log Analysis: A Pattern Reference

New! The Second Edition is now available.

Available for Kindle

Also available in PDF format from Software Diagnostics Services

The first edition is available for SkillSoft Books24x7 subscribers

General trace and log analysis patterns allow application of uniform problem detection and solving approach across diverse software environments. This pattern language covers any execution artifact from a small debugging trace to a distributed log with billions of messages from hundreds of computers, thousands of software components, threads, and processes. Pattern-oriented trace and log analysis is applicable to troubleshooting and debugging Windows, Mac OS X, Linux, FreeBSD, Android, iOS, z/OS, and any other possible computer platform including networking and IoT. Its pattern catalog is a part of pattern-oriented software diagnostics, forensics, prognostics, root cause analysis, and debugging developed by Software Diagnostics Institute (DumpAnalysis.org + TraceAnalysis.org). This reference reprints with corrections 133 patterns originally published in Memory Dump Analysis Anthology volumes 3 - 9 and Software Diagnostics Library (former Crash Dump Analysis blog, DumpAnalysis.org/blog). Full-color diagrams accompany most pattern descriptions. The second edition includes 33 more patterns and improved pattern index.

Product information:

  • Title: Software Trace and Log Analysis: A Pattern Reference
  • Authors: Dmitry Vostokov, Software Diagnostics Institute
  • Language: English
  • Product Dimensions: 21.6 x 14.0
  • Paperback: 224 pages
  • Publisher: OpenTask; 2 edition (November 2016)
  • ISBN-13: 978-1-908043-82-5

Table of Contents
Bird's-eye View of Pages

Software Diagnostic Space as a General Graph of Software Narratives

By connecting various memory spaces (user, kernel, physical, virtual, orbifold, manifold, fiber bundle, adjoint), trace and log spaces, and problem description narratives we introduce Software Diagnostics Space as a search space for finding problem patterns using general and concrete analysis patterns. Using mathematical metaphors we view it as a general graph of statements from Software Problem Narrative (graph vertices) and various software narratives such as logs, traces and memory spaces (edges). Software problem narratives may be different from software problem descriptions (which we get from software users and which have their own analysis patterns) because they are controlled narratives of actor interactions while working with software (top right corner of software narratology square). For completeness, every software narrative edge has vertices by default as start and stop vertices.

We consider Software Diagnostic Space as Trace Mask of Software Problem Narrative with Special and General Traces and Logs.

Let’s look at one example depicted in the following diagram:

We have the problem description from a user who couldn’t exercise some software functionality unless some service was restarted. This is a problem description narrative (no. 1). A software support person constructed the problem reproduction setup narrative (no. 2) and recorded problem narrative no. 3 – 5 with tracing the client and server software and taking memory snapshots (Adjoint Space trace and log analysis pattern) of the corresponding service and another Coupled Process (memory analysis pattern).

This can all be depicted in the following general graph (multigraph) diagram where loops show adjoint spaces (“instantaneous” artifact snapshots like memory, data):

Such graphs may not be connected, and loops may be replaced by additional multiple edges with extra vertices.

The practical usage of such graphs can be demonstrated by their construction during problem analysis. Suppose that we have a problem description:

After its analysis we construct a problem narrative:

During its trace analysis we identify needed software trace edges:

We add multiple edges if tracing involves several software systems or different trace varieties:

Theoretical Software Diagnostics

Available in PDF format from Software Diagnostics Services

Contains reprinted articles in full color from Memory Dump Analysis Anthology related to pattern-oriented software diagnostics with additional comments showing the historical development of this autonomous and distinctive discipline over the last 10 years.

Product information:

  • Title: Theoretical Software Diagnostics: Collected Articles
  • Authors: Dmitry Vostokov, Software Diagnostics Institute
  • Language: English
  • Product Dimensions: 21.6 x 14.0
  • Paperback: 286 pages
  • Publisher: OpenTask (October 2016)
  • ISBN-13: 978-1-908043-98-6

Table of Contents

Topological Software Trace and Log Analysis


Previously we based software trace and log analysis on software narratology. While continuing further research and development in that direction we are now constructing a new software trace and log analysis system called TopoLog based on ideas and techniques from topology (originally called analysis situs in Latin: analysis of the situation) as a part of theoretical software diagnostics. Over the years we described a few trace and log analysis patterns based on topological metaphors: Quotient Trace, Message Cover, Fiber Bundle, Sheaf of Activities, and Adjoint Space. Before starting our pattern work on software trace analysis we considered threads as braids in abstract space, and, after the first analysis patterns, we considered multithreading as multibraiding. For general software traces and logs including memory snapshots we propose topological state analysis, for example, analysis of the covering space via open and closed Memory Regions and Region Strata.

Memory Dump Analysis Anthology, Volume 9b

The following direct links can be used to order the book now:

Buy Kindle version

Also available in PDF format from Software Diagnostics Services

Contains revised, edited, cross-referenced, and thematically organized selected articles from Software Diagnostics Institute (DumpAnalysis.org + TraceAnalysis.org) and Software Diagnostics Library (former Crash Dump Analysis blog, DumpAnalysis.org/blog) about software diagnostics, root cause analysis, debugging, crash and hang dump analysis, software trace and log analysis written in March - September 2016 for software engineers developing and maintaining products on Windows platforms, quality assurance engineers testing software, technical support and escalation engineers dealing with complex software issues, security researchers, reverse engineers, malware and memory forensics analysts. This volume is fully cross-referenced with volumes 1 – 9a and features:

- 11 new crash dump analysis patterns
- 11 new software log and trace analysis patterns
- New structural memory pattern
- Introduction to Riemann root cause analysis language
- Introduction to problem solving as code
- Introduction to Dia|gram graphical diagnostic analysis language
- Introduction to iterative pattern-oriented root cause analysis
- Definition of theoretical software diagnostics

Product information:

  • Title: Memory Dump Analysis Anthology, Volume 9b
  • Authors: Dmitry Vostokov, Software Diagnostics Institute
  • Language: English
  • Product Dimensions: 22.86 x 15.24
  • Paperback: 149 pages
  • Publisher: OpenTask (October 2016)
  • ISBN-13: 978-1-908043-36-8

Table of Contents

Theoretical Software Diagnostics and Education

After writing so much about software diagnostics, we introduce its abstract generalising principles of pattern orientation and systems thinking as Theory of Software Diagnostics. We were thinking about the importance of theory for quite some time until we got acquainted with the work of Leo Klejn who coined a term “theoretical archaeology.” Then we also decided to coin the similar term for software meta-diagnostics since we compiled two books as guides to software diagnostics principles irrespective of software platforms, vendors, and their software products: Software Diagnostics and Principles of Memory Dump Analysis and plan to publish a compilation of related theoretical articles (Theoretical Software Diagnostics, ISBN-13: 978-1-908043-98-6, forthcoming September 2016). Looking for the development of theoretical archaeology as guidance makes sense because it emerged recently in contemporary times and also deals with artefacts, historical reconstruction, and time- and memory-related issues, albeit of a different nature. While working on theoretical foundations and principles for many years, we had to learn theories, ideas, and metaphors of other disciplines used in software diagnostics that we call software para-diagnostic theories by analogy with para-archaeological (coined by Klejn) theories such as history, sociology, linguistics. In his book Introduction to Theoretical Archaeology: Meta-archaeology, Klejn made a few remarks on the required theoretical education. We would like to reformulate them in relation to theoretical software diagnostics:

  • Very few people do theory because theoretical thinking requires broad education and polymath knowledge across many disciplines. We found that:

    • Computer science and software engineering education helps in the practical side of software diagnostics but is not enough;
    • Knowledge of university-level mathematics and natural science education help in understanding of technical diagnostics but is not enough;
    • Knowledge of the principles of medical diagnostics helps because pattern-oriented facet of theoretical software diagnostics is partially based on medical metaphors;
    • Knowledge of semiotics helps in understanding of the role of signs in theoretical software diagnostics;
    • Knowledge of philosophy helps in deeper understanding of foundational aspects of theoretical software diagnostics such as the nature of problems, their phenomenology, meaning, and understanding;
    • Humanities education (analysis of human-made artefacts) is very important since software diagnostics is also based on artefact analysis.
  • Such education is needed from earlier up and in addition to computers and coding should also include history, philology, narratology, and literary theory.
  • In summary, broad reading is required to get acquainted with diagnostics expertise in various domains of human activity.

Iterative Pattern-Oriented Root Cause Analysis

When we introduced A.P.M. patterns-based root cause analysis methodology (Artefacts. Patterns. Mechanisms.), it may have made an impression of a waterfall-type process with some iterations between artefact collection and diagnostic analysis when collected artefacts are not good. However, software post-construction problem solving is usually iterative, with memory dumps and software logs collected again and again after the preliminary root cause analysis.

To illustrate the iterative nature of the process we first name its stages as Artefact Acquisition for Artefacts, Artefact Analysis for Patterns (diagnostics), and Analysis of Analysis for Mechanisms (root cause analysis):

Now we rearrange these AA stages:

After the preliminary root cause analysis (Analysis of Analysis) we may need to gather more artefacts for further diagnostics and more precise RCA, and this is reflected in more focused stages:

Problem Solving as Code

We introduce Problem Solving as Code as a process of developing, managing, and provisioning problem-solving methods and tools. Some problem-solving methodologies such pattern-oriented problem solving developed by Software Diagnostics Institute as a part of Diagnostics Science require constantly evolving pattern catalogues which can be stored in version control systems. For example, pattern-oriented software problem solving involves pattern-oriented problem description analysis and software execution memory and trace artefact acquisition, pattern-driven and pattern-based software diagnostics (including forensics), the patterns-based root cause analysis, and pattern-oriented debugging process which introduced design methodology to debugging. In addition to general problem patterns and problem analysis patterns, there are concrete problem and problem analysis patterns where concrete problems are constantly changing (traditional problem repositories). PSaC (“Problem Sack”) allows using declarative and imperative problem-solving configurations tailored for specific problem domains or specific systems and products by customising pattern catalogues. Specific problem artefact types may require specialized tools and configuration so they can also be designed, developed, managed and provisioned. For example, pattern-oriented problem solving includes DebugWare and DiagWare design patterns.

Dia|gram Graphical Diagnostic Analysis Language

One of the current Software Diagnostics Institute projects is the development of Dia|gram graphical language for pattern-oriented software diagnostics, forensics, prognostics, root cause analysis and debugging. It combines the best features from:

  1. Visual Dump Objects: Graphical Notation for Memory Dumps;
  2. STDiagrams: Software Trace Diagrams;
  3. Visual compression of software traces and logs (including “bird’s eye view” of software traces), first introduced in Characteristic Message Block trace and log analysis pattern;
  4. Minimal Trace Graphs, first introduced in Activity Region trace and log analysis pattern. Numerous examples can be found in Accelerated Windows Software Trace Analysis training course reference and Software Trace and Log Analysis: A Pattern Reference book;
  5. Minimal Stack Trace Diagrams, first introduced in Constant Subtrace memory analysis pattern.

The purpose of Dia|gram language is twofold:

  • To provide a succinct presentation and visualization of software execution state, artefacts, distribution of problem patterns, problem analysis patterns and their relationship;
  • To communicate pattern-oriented software diagnostic analysis results.

Additionally, Dia|gram may be used for presentation and analysis of higher-order pattern narratives.

Software Diagnostics Institute also proposes the UML profile for Software Diagnostics with additional diagram types: artefact acquisition map, activity backtrace, and implementation internals. This work is only started, and more will be presented in subsequent articles.

Software Diagnostics Services plans to include Dia|gram in its forthcoming Advanced Software Trace and Log Analysis training course.

Riemann Root Cause Analysis Pattern Language


Image generated by 3D-XplorMath

Incepted and named in February 2009 shortly before the first software trace and log analysis pattern was published in April the same year, Riemann Programming Language was thought of as a software problem description language capable of generating software problem-solving tools (including TaaS version). A book was planned for publication in 2010: The Riemann Programming Language (ISBN: 978-1906717605). The main motivation at that time for the name was the metaphorical correspondence between multi-valued functions represented by Riemann surfaces and software defects as alternative branches of computation. Since the significant development of pattern-oriented software diagnostics, introduction of network and performance analysis pattern languages and patterns-based root cause analysis methodology we now make Riemann Programming Language an optional coding complement to Riemann Root Cause Analysis Pattern Language. The latter includes diagnostic analysis pattern languages for trace analysis and memory analysis developed by Software Diagnostics Institute including structural memory patterns in the context of general log analysis. We can now consider another analogy with multi-valued functions where the same general diagnostic patterns in a memory dump or log can be generated by different source code. Riemann RCA Pattern Language facilitates the transformation of software narrative artefacts into much shorter analysis narratives through the process of articoding. The resulting analysis artefacts can be programmatically processed to generate diagnostic, troubleshooting and debugging configurations, classes and functions, frameworks and plugins, components and nodes. The following diagram describes this process:

The Riemann programming language should not be confused with Riemann monitoring system which was named and developed later elsewhere by a different group of people and which is about collecting events and not about their collective analysis using pattern-oriented analysis methodology developed by Software Diagnostics Institute. Regarding event monitoring, Software Diagnostics Institute also develops platform-independent software trace and log acquisition patterns for better use of various monitoring systems.

Book: Accelerated Windows Memory Dump Analysis, Fourth Edition

The following direct links can be used to order the book now:

Buy Kindle version

Buy Paperback from Amazon

Buy Paperback from Barnes & Noble

Buy Paperback from Book Depository

Also available for sale in PDF format from Software Diagnostics Services.

The third edition is available for Safari Books Online subscribers

The second edition is available for SkillSoft Books24x7 subscribers

The full transcript of Software Diagnostics Services Training with 28 step-by-step exercises, notes, source code of specially created modeling applications and more than 100 questions and answers. Covers more than 60 crash dump analysis patterns from x86 and x64 process, kernel, complete (physical), and active memory dumps. The 4th edition was fully reworked to use WinDbg 10 and now covers memory dumps from Windows 10 x64. It also includes optional legacy exercises from the previous editions covering Windows Vista and Windows 7.

  • Title: Accelerated Windows Memory Dump Analysis: Training Course Transcript and WinDbg Practice Exercises with Notes, Fourth Edition
  • Authors: Dmitry Vostokov, Software Diagnostics Services
  • Publisher: OpenTask (May 2016)
  • Language: English
  • Product Dimensions: 28.0 x 21.6
  • Paperback: 872 pages
  • ISBN-13: 978-1908043467

Table of Contents
Additional materials referenced in the course

Diagnostics of Things (DoT)

We introduced Narratology of Things as a combination of Software Narratology of Things and Hardware Narratology. Since memory dump analysis may be considered as a part of general trace and log analysis we open a new research direction for Diagnostics of Things (DoT) based on Narratology of Things and Pattern-Oriented Trace and Log Analysis, which also includes software execution artifacts of things and pattern-oriented network trace analysis from IoT.

10 years!

On the 26th of March 2006, 10 years ago, dumpanalysis.org was registered! It was still a long way towards pattern-oriented software diagnostics. The main product of our activity, Memory Dump Analysis Anthology, is now in 10 books.

Software Diagnostics Institute Timeline

Memory Dump Analysis Anthology, Volume 9a

The following direct links can be used to order the book now:

Buy Kindle version

Buy Paperback from Amazon

Buy Paperback from Barnes & Noble

Buy Paperback from Book Depository

Also available in PDF format from Software Diagnostics Services

Contains revised, edited, cross-referenced, and thematically organized selected articles from Software Diagnostics Institute (DumpAnalysis.org + TraceAnalysis.org) and Software Diagnostics Library (former Crash Dump Analysis blog, DumpAnalysis.org/blog) about software diagnostics, root cause analysis, debugging, crash and hang dump analysis, software trace and log analysis written in August 2015 - February 2016 for software engineers developing and maintaining products on Windows platforms, quality assurance engineers testing software, technical support and escalation engineers dealing with complex software issues, security researchers, reverse engineers, malware and memory forensics analysts. This volume is fully cross-referenced with volumes 1 – 8 and features:

- 9 new crash dump analysis patterns
- 9 new software log and trace analysis patterns
- 15 Linux core dump analysis pattern variants
- New workaround pattern
- New memory dump analysis case study
- Introduction to pattern-oriented software internals, pattern paradigms, pattern stacks, pattern repertoire
- Introduction to software diagnostics canvas
- Introduction to patterns-based root cause analysis methodology
- Introduction to a protein metaphor for software traces and logs
- Definition of software diagnostics scope
- Introduction to artificial debugger and pseudo-memory dumps
- Definition of tool-centric and pattern-centric software diagnostics, forensics, prognostics

Product information:

  • Title: Memory Dump Analysis Anthology, Volume 9a
  • Authors: Dmitry Vostokov, Software Diagnostics Institute
  • Language: English
  • Product Dimensions: 22.86 x 15.24
  • Paperback: 179 pages
  • Publisher: OpenTask (February 2016)
  • ISBN-13: 978-1-908043-35-1

Table of Contents

The Scope of Software Diagnostics

Should software diagnostics stand alone as a separate, distinct diagnostics or be a part of some other diagnostics? We considered initially 3 types of diagnostics: medical, technical, and software:

The objects of medical diagnostics are obviously humans and mostly their biological artefacts (we say mostly because linguistic and textual artifacts can also be used for diagnosis). The objects of technical diagnostics are structures and systems made from natural and artificial organic and inorganic engineering materials (Handbook of Technical Diagnostics: Fundamentals and Application to Structures and Systems, ISBN: 978-3642258497, pp. 11 – 16). The objects of software diagnostics are obviously software systems.

We define software diagnostics as “a discipline studying abnormal software structure and behavior in software execution artefacts (such as memory dumps, software and network traces and logs)” (Introduction to Philosophy of Software Diagnostics, Part 1, page 7) or, more generally, to include the context of forensics as “a discipline studying signs of software structure and behavior in software execution artefacts (such as memory dumps, software and network traces and logs)” (Pattern-Oriented Software Forensics: A Foundation of Memory Forensics and Forensics of Things, page 18).

Although, there are many conceptual similarities in general between these diagnostics, there are three features of software that set software diagnostics apart:

  • A software system or its artefact can be copied.
  • The software can be its own model. This follows from the previous feature since we can copy the software execution state and then study the effects of its execution independently. However, it is possible to have different software models for diagnostics as, for example, in Projective Debugging.
  • Software execution artefacts are both symbolic and digital.

There are also humanistic artefacts (A New History of the Humanities: The Search for Principles and Patterns from Antiquity to the Present, ISBN: 978-0199665211) such text (including historical documents), language, literature, and music which are symbolic and digital or can be digitised. Diagnostics takes the form of literary criticism and text reconstruction (philology). Many similarities there gave rise to Software Narratology and Systemic Software Diagnostics:

Teaching Complex Diagnostic Scenarios with Artificial Debugger (ArtDbg) and Pseudo-Memory Dumps

One of the problems in teaching software diagnostics and postmortem debugging is a simulation of complex software problem scenarios. There are plenty of real life memory dumps available but due to security considerations they cannot be shared outside of the organisation. Here we introduce Artificial Debugger project (ArtDbg) that simulates the I/O of the real debugger, for example, WinDbg, GDB, or LLDB.

The real memory dump is analyzed in the real debugger, and all debugger input and output is saved in a log file. This log file is then scanned for any potentially sensitive information, and all such information is eliminated (Security Problem, Memory Dump Analysis Anthology, Volume 1, page 224). It is then converted into a binary pseudo-memory dump format.

ArtDbg is used to open and analyze such pseudo-memory dumps. It allows using the same real debugger commands that were used to generate the log file. Such commands will output the same information that was available from the real debugger. Some real debugger commands that were not used to generate the log file may also be used if possible.

Software Diagnostics Services plans to use the first version of ArtDbg in their new version of Accelerated Windows Memory Dump Analysis training scheduled in March 2016 to teach some very complex enterprise diagnostic scenarios in addition to WinDbg exercises with real memory dumps.

Syndicate content