Announcing RateLimitedLogger

Logging is vital for production-ready, operable code; however, in certain situations, it can be dangerous. It is easy to wipe out throughput of a performance-critical backend component by several orders of magnitude with disk I/O in a performance hotspot, caused by a misplaced log call, or input data changing to something slightly unexpected.

RateLimitedLogger is an Apache-licensed, SLF4J-compatible, simple, fluent API for rate-limited logging in Java. A RateLimitedLog object tracks the rate of log message emission, imposes an internal rate limit, and will efficiently suppress logging if this is exceeded. When a log is suppressed, at the end of the limit period, another log message is output indicating how many log lines were suppressed.

This style of rate limiting is the same as the one used by UNIX syslog; this means it should be comprehensible, easy to predict, and familiar to many users, unlike more complex adaptive rate limits.

Once the rate limit is hit, logging becomes just a couple of java.util.concurrent.AtomicLong operations, so it’s very inexpensive, performance-wise.

More details are in the README.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s