Package org.logevents.observers

Class AbstractBatchingLogEventObserver

  • All Implemented Interfaces:
    LogEventObserver
    Direct Known Subclasses:
    AbstractHttpPostJsonLogEventObserver, DatabaseLogEventObserver, ElasticsearchLogEventObserver, SmtpLogEventObserver, SystemTrayLogEventObserver, TcpLogEventObserver
    public abstract class AbstractBatchingLogEventObserver
extends AbstractFilteredLogEventObserver
    Used to gather up a number of log event to process as a batch. This is useful when using logging destinations where high frequency of messages would be inefficient or noisy, such as email or Slack and for logging asynchronously to external systems over network such as databases or Elasticsearch.

    AbstractBatchingLogEventObserver can decide how to batch events with batchers, for example ThrottlingBatcher and CooldownBatcher. When processing, the AbstractBatchingLogEventObserver calls processBatch(List) with the whole batch to process the log events, which should be overridden by subclasses. Consecutive Log events with the same message pattern and log level are grouped together so the processor can easily ignore duplicate messages.

    Configuration example with cooldown (don't send messages too frequently)

    The following will send the send messages at level WARN that don't have the Marker "PERSONAL". After each batch, wait at least 10 seconds before sending the next batch. But after each message is received, wait 2 seconds to see if more messages are coming. In any case, never wait more than 30 seconds before sending a batch. 
     observer.sample.threshold=WARN
 observer.sample.suppressMarkers=PERSONAL_DATA
 observer.sample.idleThreshold=PT2S
 observer.sample.cooldownTime=PT10S
 observer.sample.maximumWaitTime=PT30S

    Configuration example with throttle (increasingly larger offsets)

    The following will send the first message at level WARN with the Marker "DAEMON" immediately, then wait at least 30 seconds before sending the next, then 5 minutes, then 15 minutes between each batch. 
     observer.sample.threshold=WARN
 observer.sample.requireMarker=DAEMON
 observer.sample.throttle=PT30S PT5M PT15M

    Marker-specific configuration

    The following will throttle messages with MY_MARKER, grouped by MDC value userId. If there is a log event with an unused userId, it will be sent immediately. Then all events for the next minute for the same userId will be collected, then the next hour and then the next day. If there are log events for other userIds, they will be batched separately. This is useful for situations like API batch users where the credentials have expired, for example. 
     observer.sample.markers.MY_MARKER.throttle=PT1M PT1H PT24H
 observer.sample.markers.MY_MARKER.mdc=userId
    Author:
    Johannes Brodwall
    See Also:
    SlackLogEventObserver, MicrosoftTeamsLogEventObserver, SmtpLogEventObserver, ElasticsearchLogEventObserver, DatabaseLogEventObserver