what exactly to log ? #systemdesign #softwaredevelopment #software #logging #backenddevelopment #log

Creating a new service/API, but not sure what exactly to log ?
3 key types of logging to cover .. — Logging Series Primer — 1
Logging means recording information about execution of a program.
Types of Logs
We can have generally 3 different types of logging : Each log type has different purpose

- Application logging : To trace the execution of a program
- Behaviour logging: To track things over long period (mostly for data analytics purpose)
- Metric logging : To monitor key business metrics in real time
---------------------------- ----------------------------

Application logging (Use log4j, slf4j)

- Application logs — Human readable logs , allowing developers to trace application behavior to diagnose and fix errors.
- We use Log4j (warnings, info, errors)
- We generally use Kibana for visualization
- Retention period generally 1 — 2 week
---------------------------- ----------------------------

Behaviour and User Interaction Logging

- These logs are about behavior and user interaction
- Are stored in a central log repository and analyzed for insights.
- Data in here gets turned into Indexes, which allow to create nifty charts and graphs for analysis
- Retention period is forever
---------------------------- ----------------------------

Metrics Logging (Use Datadog)

- To track key business metrics in real time..
- Intended for immediate consumption
- Some dropped data is acceptable.
- Generally used to create and display metrics dashboards
- Use tools like StatsEmitter, StatsD or Micrometer
- We generally use Datadog Dashboards for visualization
- Retention period generally 1 year types
- ex: success/failures of requests, execution time of a specific function
---------------------------- ----------------------------
Use cases for logs ?
1. Debugging
2. Monitoring
3. Performance Analysis
4. Alerting: Automated monitoring systems scan the logs for error patterns and trigger notifications
5.Historical analysis : Logs serve as a historical record of the application’s behavior over time.

---------------------------- ----------------------------

Questions for thought:

- Any other major use case of logging. ?
- Common logging mistakes ?
---------------------------- ----------------------------

Detailed Article in comments

---------------------------- ----------------------------

tldr

Always log for
- Exceptions : Logging exception help us debug the API call failures — Devs need this
- Performance, SLO : Logging “Event” creation time so that we can use it to track SLO failures in real time — DevOps need this
- Metrics : Emit log whenever an important “Event” is updated to track key metrics. — PM needs this

---------------------------- ----------------------------

#softwareengineering #kibana #dataanalysis #logging #bestpractices #softwaredevelopment #systemdesign #log4j

Видео what exactly to log ? #systemdesign #softwaredevelopment #software #logging #backenddevelopment #log канала 1minuteconcepts