Special and General Trace and Log Analysis

Most software traces include message timestamps or have an implicit time arrow via sequential ordering. We call such traces Special. The analysis is special too because causality is easily seen. Typical examples of analysis patterns here are Discontinuity, Time Delta, Event Sequence Order, Data Flow (see time dependency markers ↓ in the training course reference), and more recently added patterns such as Back Trace, Timeout, Milestones, and Event Sequence Phase. Inter- and Intra-Correlation analysis is also easy.

On the other side, there are plenty of software logs or digital media artefacts with “chaotic” records where time arrow is missing or only partial. Typical examples are debugger logs from WinDbg debugger from Microsoft Debugging Tools for Windows or logs from debugging sessions on other platforms. Such logs may contain global ordering such as the list of processes and threads (Last Object memory analysis pattern) interspaced with local pockets of stack traces that have reversed ordering. Some logging output may not have any ordering or timing information whatever.

In a more general case, logging may be completely arbitrarily. Typical example is raw stack analysis and its Rough Stack Trace and Past Stack Trace patterns.

We call such traces General. The main task of general trace analysis is to recover causality. It may be possible if another analysis pattern is introduced called Causality Markers. The prototypes of such a pattern are various wait chains, Waiting Thread Time memory analysis pattern and its process memory dump equivalent.