Skip to main content

Tools for analyzing log files

Bravura Security Fabric log files can grow very large. Choosing the right tool for viewing, searching, and filtering logs has a significant impact on the speed and effectiveness of your investigation.

Note

A text editor such as Notepad or Notepad++ uses significantly more system resources than a text viewer or filter, especially with large files. Using an editor on the server itself also takes resources away from the application or database. Where possible, copy log files to a workstation before analysis, and use a viewer or command-line filter rather than an editor.

Linux command-line tools

GNU command-line tools are mature and efficient. System administrators have used them for a long time to parse, filter, and search through large text files such as system and application logs. The following tools are recommended:

less

View large log files without loading the entire file into memory.

grep

Search for strings or regular expression patterns. Supports counting results with -c and displaying context lines with -A, -B, and -C.

awk

Parse and extract specific fields from log entries. Useful for isolating process IDs, timestamps, or message text.

sort and uniq

Order and count collated search results. Use sort | uniq -c to count occurrences of each unique line.

wc

Count returned words and lines. Use wc -l to count matching entries from a grep result.

Windows tools

You can use Linux GNU tools on Windows through packages such as Git Bash, which provides an "Open Git Bash here" option in Windows Explorer for quick access to the tools listed above.

The following tools ship with Windows but have limitations for large log file analysis:

Notepad and Notepad++

Text editors that use more resources (including time and RAM) than a viewer or filter. They can handle small log files under 1GB but do not support searching backward from the end of the file with regular expressions. Backward searching is often needed in log analysis to find what a specific binary logged last.

findstr

Finds relatively simple strings and expressions. It can fail with large files and complex patterns.

PowerShell Select-String

Can search for patterns, but its memory usage is inefficient for large files. It has a low limit for input file length, returns filterable objects rather than strings, and has reduced functionality in older versions of PowerShell.

Regular expressions for log analysis

When you search through logs or filter them for issue monitoring, simple string searches are often not enough. You need to compare substrings in several places within a log entry to identify the relevant entries. Regular expressions allow you to define complex search patterns that match process IDs, timestamps, error codes, and message text simultaneously.

Regular expressions come in many dialects. The dialect you use depends on the tool:

  • grep uses POSIX basic regular expressions by default, or extended expressions with -E.

  • PowerShell Select-String uses .NET regular expressions.

  • The Bravura Security Fabric web UI System logs page uses its own regular expression syntax. Select the Regular expression checkbox to enable it.

The following resources can help you learn and practice regular expressions:

  • RegexOne provides interactive lessons for learning regular expression fundamentals.

  • RegExr provides a sandbox where you can paste log fragments and test patterns against them.

Note

When using a regex sandbox, paste a small fragment of a log that does not contain server names or other confidential data.

In this section: