We are creating a logging module that will be added to several console apps and web apps. Throughout the code in the apps, we want to use the logging module to create log messages that end up on a Slack channel. However, we don’t want the logging module to introduce any issues or pauses while posting to the Slack API and we don’t want a scenario where we lose messages. We also don’t want to add complexity. The apps typically generate very few messages, but there have been situations where a thousand or more were generated on a single day.
All of the apps use the same local database and maintain a connection, so we can easily log messages to a database table.
Have an external helper app monitor the table and post messages to Slack API
As a fail-safe:
1, If we cant write to the database, then post a Slack message directly from the app within the logging module.
2. If we can’t post a Slack message, then write to the system log.
Have the logging module post log messages directly to the Slack API. Forget about the complexity of a separate helper app. Write to the system log if there is an error posting to the Slack API.
I’d say that this depends on the number of log messages that you plan to send. How about the following: you try plan b and see if this works if you hit Slack with lots of messages. And when you get problems you can still do something more complicated.
What kind of format are the log messages? Just single lines of text, or potentially chunky blocks of debug info etc?
If simple then I’d do a variation on Plan A and write to rotating text file log, and then have a daemon to poll it and send to Slack. If the messages can be chunky I’d stick with your first thought of a database as the primary log method.
The main point is that you don’t have any control of the availability of Slack’s API, don’t rely on it for logging. Use it as a notification method, not a primary logging method, and it definitely should be async.