Filtering Reports with Severity

* Note: the implementation of the severity mechanism is not functional as of this writing and is experimental in the C++ code *

This is inspired by syslog and Log4J priorities and levels. It is common in logging systems to define different levels of importance or detail for logging messages, and to have some way of specifying, either at compile time or run time, which levels should be logged. Too much logging imposes too much overhead on the system in question, on the logging infrastructure, and on the person examining the logs that sees too much information.

The way these systems work is that each debug statement in the code is given a level, such as DEBUG, NOTICE, or WARN. There are extensions as well, these instrumentation points may be given groups, so that you can have a level per group, for example. The key here is that each debug statement has a 'severity' associated with it that does not change.

What changes is a global threshold for what is logged, such that a given statement is only logged if its severity is at least as high as the global threshold. For example, if the threshold is set to WARN, then only those statements that are at least of level WARN will be logged.

This system is simple but works for a range of situations. It assumes that the levels can be totally ordered, which may not fit some situation, but this makes encoding the levels and deciding whether to log a fast operation. For efficiency reasons, you want to decide as soon as possible whether to log a given log statement, and not even process the arguments of the call if it is the case that you will not log. String concatenation can be awfully expensive and becomes wasted effort if you have to concatenate the strings before checking.

X-Trace uses a similar notion of severity, with an important extension enabled by the task-centric, causal tracing: the X-Trace metadata itself can override the global severity threshold. This enables detailed logging only for selected tasks, while the global threshold may be set to a low-verbosity level.

Thus, in X-Trace:

  1. Similar to other systems, a given instrumentation point (a logging statement) can define its own severity
  2. There may be a global X-Trace severity threshold, which can be set statically or dynamically
  3. A task may be assigned a severity threshold, which is carried as the task moves along the system and overrides the global severity threshold.

In both cases, an event is logged if its severity level is greater than or equal to the effective severity threshold.

Severity Levels

The following Severity Levels are defined, from the least severe to the most severe. Severity levels are represented in the X-Trace metadata as an option, and their numeric equivalents are also given below:

  • DEBUG = 8 (least severe, most verbose)
  • INFO = 7
  • NOTICE = 6
  • WARNING = 5
  • ERROR = 4
  • CRITICAL = 3
  • ALERT = 2
  • EMERG = 1 (most severe, least verbose)

The default severity level for an X-Trace event if not specified is NOTICE. The default global severity threshold for X-Trace, if not specified, is NOTICE. The severity level of an X-Trace metadata applies to the task, and overrides the global threshold if present.

In addition to these levels, the severity threshold (both global and the one embedded in the X-Trace metadata), may be set to the two special values:

  • _ALL = 255 (all events logged)
  • _NONE = 0 (no events logged)

Filtering by severity

An X-Trace event with severity level L, in a context with severity threshold T, will be logged if L ⇐ T. L can range from 1 to 8, and T from 0 to 8 and 255.

C++ API

FIXME .This is outdated.

As an example of how to use the severity specification above, we describe the relevant aspects of the C++ Xtr library and API.

Setting the Severity of an XtrMetadata:

   XtrMetadata x;
   x.setSeverity(XtrOptionSeverity::DEBUG);

This sets the (first) severity option in the metadata to be DEBUG. The default value for severity is given by XtrOptionSeverity::DEFAULT, which is equal to XtrOptionSeverity::NOTICE. This is the same default as syslog.

When an XtrEventCtx is created either from the metatada directly, or using the metadata as an incoming edge, the severity will be preserved in the event's own metadata.

  XtrEventCtx e(x);
  assert(e.getMetadata().getSeverity() == XtrOptionSeverity::DEBUG) 

Note that XtrEventCtx.getMetadata() returns a const&, so one cannot set the severity of the context. This has to be done before adding the first edge, if desired.

Finally, the XtrReportCtx set of static functions have the provisions to deal with severity.

  void XtrReportCtx::setSeverityThreshold(u_int8_t severity)
  xtr_result_t XtrReportCtx::getSeverityThreshold()
  bool XtrReportCtx::willReport(u_int8_t severity)

The first two set and get the severity threshold of the reporting context. This is the minimum severity that will be reported. There are two special values, _ALL and _NONE, that respectively mean all reports wil be allowed and none will. willReport checks if, at the current threshold, the given severity would be reported or dropped. This can be used to decrease disabled costs, if the preparation of the message will be costly.

The default threshold is _ALL, if none are specified. The function for reporting has two parameters, the message to be reported and the priority of the message. This SHOULD be the same priority as the included in the metadata in the report, but there is no enforcement by the library. The goal was to make XtrReportCtx not have to parse the message. There is a default priority if it is not specified, and it is XtrOptionSeverity::NOTICE.

  XtrReportCtx::sendReport(const char* msg,
                           u_int8_t severity = XtrOptionSeverity::DEFAULT);

The XtrEventCtx class has a sendReport function that internally calls XtrReportCtx::sendReport(), setting the right severity based on the event's own severity. This function will not prepare the reporting string if the priority threshold won't allow it, saving considerable cost, and is the preferred method of sending reports.

  XtrEventCtx::sendReport()

Usage Cases

Severity to different tasks

Shutting off all reports at the source

Different levels of detail for different code points

Discussion

This is most likely not the best solution, but it is a first step. The aspect that needs more thought is how to combine the severity level that an even inherits from the current metadata with the severity level in the logging statement. Suppose the incoming metadata to an event has severity I, and the logging statement has severity L. The two questions are: (i) what is the severity R of the resulting report message (to be compared to the reporting severity threshold for reporting), and (ii) what is the resulting severity level of the outgoing metadata N.

The current behavior is that is L is set, it overrides I, and both R and N are set to L.

Other possibilities is for R ← L and N ← I, or for R to be some function of L and I.

 
docs/severity.txt · Last modified: 2010/02/24 01:27 by rfonseca
 
Except where otherwise noted, content on this wiki is licensed under the following license:CC Attribution-Noncommercial-Share Alike 3.0 Unported
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki