Welcome to Introduction to Logging! After watching this video, you will be able to: Define application logging, Explain how to determine what information and data to log, And describe why application logging is important. Logging is a series of messages from an application that provide a recorded log of the application's activities. Log messages provide important information for developers when debugging an application and for system administrators maintaining applications in production. An application log contains information about events that have occurred within a software application. These events are logged by the application and saved. They can include errors, informational events and warnings. In the past, application logs were sent to files, but cloud-native applications treat logs as event streams and log to stdout so that they can easily be processed by data collectors. An application normally contains code for recording various types of events to an application log. The log can reveal message flow issues and application problems. It can also contain information about user and system actions that have occurred. Logged events can typically include messages and warnings about the following: Application messages, Transaction flow events, Low disk space, Completed operations, Error events that prevent the application from starting, Success audits to indicate security events. such as successful logons. And failure audits to indicate failed events, such as logon failures. Identifying what information should be collected and how it will be used is an important part of logging. Like applications, logs must be designed, implemented, and tested because application developers must define what, how, and when to log application information and data. And then developers must determine how to extract meaningful data from the logs. You should consider the following when determining what to log: Both incoming and outgoing messages must be recorded with application performance indicator (or API) endpoint URLs, request parameters, request origin and intermediary IPs, request headers, authentication information, request and response bodies, business context, timestamps, and internal processing steps. When a service or function is invoked, it is a good practice to log its context in a lower log level, mostly for debugging purposes (use TRACE or DEBUG). Having these logs helps investigate issues related to business logic. Every application has its unique business cases and user journeys, and they reveal many insights for the domain experts in the system. For instance, whether a certain transaction takes too long or the end users get stuck at some functionality all the time are very critical pieces of information in terms of user experience. Other business-related information, like transaction volumes and active users and their stages, is important for deriving business insights and even can be used for business intelligence purposes too. For security and compliance, it is mandatory for many enterprise applications to keep a separate log for data-related operations with all important information like access IDs, exact service instances and role privileges used, including timestamps, data layer queries, and snapshots of both previous and new states of the changed dataset. System events must capture information on behavior events (startups, stops, restarts, security events), changeover modes (cold, warm, hot), inter-service communication (handshakes, connection building statuses —connect, disconnect, reconnect, retry events), service instance IDs, actively serving APIs, actively listening IP and port ranges, configurations loaded (initial load up and dynamic updates), overall service health, and anything that helps to understand the behavior of the system. Performance statistics, like anomalies or sudden unexpected degradations in services (mostly due to unhandled errors and corrupted data), can happen at any given time. To identify them, it is always recommended to publish stats on overall system health and performance. There are two main reasons why application logs play an important role in your overall strategy: diagnostics and auditing. You can use application logs for diagnostics by tracking and correlating certain information and using it to perform analytics on your software. Some examples of information that might be useful to track and analyze are customer transactions, security threats, timeouts, and consumer behavior. Log management tools can give you valuable insights from the information you’re already collecting. Application logs can also be used for diagnosing issues. You can use application logs to identify and resolve bugs in a production environment. This process takes place regularly during software development. And it’s critical for ensuring long-term optimal performance of an application. Another important reason for application logs is that they can also be used for auditing purposes. These logged messages involve significant events in the application as well as information relating to management and finance. This information may not provide as much benefit on a daily basis, but it’s important for fulfilling business requirements. You know, when one of my students, or a member of my team deploys a new application to the cloud and things aren't going well so they ask for my help to debug it, the first questions I ask is, "what do the logs say?" When they invariably come back and tell me that there's nothing in the logs, I tell them... "Well, it's time to go back to your application and start logging your activities!" Your logs are your first level of understanding of what your application is doing in production. Don't skimp on logging. In this video, you learned: Application logging is a series of messages from an application to provide a logged record of the application's activities; An application log contains information about events that have occurred within a software application; Determine what to log by considering an application's incoming and outgoing messages, services and functions, business cases and user journeys, data operations, system events, performance statistics, and threats and vulnerabilities; Application logging is important because it provides you with information about diagnostics, like correlating tracked information and identifying, diagnosing, and resolving issues, and it is required for auditing purposes because it provides data on management and finances.
