WEBVTT 1 00:00:00.000 --> 00:00:01.320 In this lesson, 2 00:00:01.320 --> 00:00:04.050 we will learn about logging and monitoring. 3 00:00:04.050 --> 00:00:06.570 Logging and monitoring is the recording 4 00:00:06.570 --> 00:00:08.940 and analyzing of access events 5 00:00:08.940 --> 00:00:12.360 and user activities to detect, investigate, 6 00:00:12.360 --> 00:00:15.990 and respond to potential security incidents. 7 00:00:15.990 --> 00:00:20.430 Logging is the collection of data related to user actions, 8 00:00:20.430 --> 00:00:23.340 system changes, and access attempts. 9 00:00:23.340 --> 00:00:25.080 Monitoring on the other hand, 10 00:00:25.080 --> 00:00:29.250 involves the real time or periodic review of logs 11 00:00:29.250 --> 00:00:33.270 to identify unusual or unauthorized activities. 12 00:00:33.270 --> 00:00:36.330 Let's learn more about logging and monitoring. 13 00:00:36.330 --> 00:00:38.280 First, we have logging. 14 00:00:38.280 --> 00:00:41.040 Logging is the process of capturing 15 00:00:41.040 --> 00:00:45.360 and recording detailed information about actions, changes, 16 00:00:45.360 --> 00:00:49.200 and access attempts within a system or network. 17 00:00:49.200 --> 00:00:51.570 In large enterprise environments, 18 00:00:51.570 --> 00:00:55.320 logs are constantly generated by many sources, 19 00:00:55.320 --> 00:00:59.670 such as user activities, system processes, applications, 20 00:00:59.670 --> 00:01:01.500 and network devices. 21 00:01:01.500 --> 00:01:04.890 This volume of logs can become overwhelming 22 00:01:04.890 --> 00:01:08.550 with millions of events being logged each day. 23 00:01:08.550 --> 00:01:12.870 However, these logs are critical for tracking who accessed 24 00:01:12.870 --> 00:01:16.470 what resources, when, and under what conditions, 25 00:01:16.470 --> 00:01:19.440 providing an essential trail of evidence 26 00:01:19.440 --> 00:01:23.010 for security investigations and audits. 27 00:01:23.010 --> 00:01:26.370 To manage this vast amount of logs effectively, 28 00:01:26.370 --> 00:01:29.190 organizations use log collectors. 29 00:01:29.190 --> 00:01:32.070 A log collector is a system that generates 30 00:01:32.070 --> 00:01:35.280 and centralizes logs from multiple sources, 31 00:01:35.280 --> 00:01:37.500 simplifying log management. 32 00:01:37.500 --> 00:01:41.490 Fluentd, for example, is an open-source log collector 33 00:01:41.490 --> 00:01:45.060 that can aggregate logs from servers, applications, 34 00:01:45.060 --> 00:01:46.830 and cloud services. 35 00:01:46.830 --> 00:01:49.320 By collecting logs in one place, 36 00:01:49.320 --> 00:01:53.730 it becomes easier to store, search, and analyze them. 37 00:01:53.730 --> 00:01:57.210 This centralized approach enhances visibility 38 00:01:57.210 --> 00:02:00.240 across the network and simplifies compliance 39 00:02:00.240 --> 00:02:04.350 with regulatory requirements such as maintaining log records 40 00:02:04.350 --> 00:02:06.330 for a specified duration. 41 00:02:06.330 --> 00:02:07.950 Once logs are collected, 42 00:02:07.950 --> 00:02:10.380 they need to be processed and analyzed, 43 00:02:10.380 --> 00:02:12.720 which is where security information 44 00:02:12.720 --> 00:02:15.840 and event management, or SIEM systems, come in. 45 00:02:15.840 --> 00:02:17.700 A key feature of SIEMs, 46 00:02:17.700 --> 00:02:21.810 is their ability to aggregate logs from multiple sources, 47 00:02:21.810 --> 00:02:25.500 like from servers and applications into one place. 48 00:02:25.500 --> 00:02:28.980 After aggregation, the SIEM normalizes the logs, 49 00:02:28.980 --> 00:02:31.800 meaning it converts the logs from different formats 50 00:02:31.800 --> 00:02:33.840 into a consistent structure. 51 00:02:33.840 --> 00:02:36.780 This is needed because logs from different systems 52 00:02:36.780 --> 00:02:40.200 often have varied formats using different fields, 53 00:02:40.200 --> 00:02:42.660 timestamps, or data structures. 54 00:02:42.660 --> 00:02:47.070 Normalization ensures that all logs follow the same format, 55 00:02:47.070 --> 00:02:49.260 making it possible to analyze 56 00:02:49.260 --> 00:02:52.440 and correlate events across systems. 57 00:02:52.440 --> 00:02:55.500 While managing and collecting logs is crucial, 58 00:02:55.500 --> 00:02:56.970 it is not enough. 59 00:02:56.970 --> 00:03:00.210 Logs must be securely stored and retained 60 00:03:00.210 --> 00:03:04.440 to meet regulatory requirements and internal policies. 61 00:03:04.440 --> 00:03:08.670 Additionally, logs need to be easily accessible for audits 62 00:03:08.670 --> 00:03:11.010 or investigations when necessary, 63 00:03:11.010 --> 00:03:14.850 ensuring that organizations can respond effectively 64 00:03:14.850 --> 00:03:18.600 to security incidents and maintain compliance. 65 00:03:18.600 --> 00:03:20.790 Second, we have monitoring. 66 00:03:20.790 --> 00:03:24.360 Monitoring is the process of continuously reviewing 67 00:03:24.360 --> 00:03:27.900 and analyzing logs and events in real time 68 00:03:27.900 --> 00:03:32.070 or at scheduled intervals to detect unusual activities 69 00:03:32.070 --> 00:03:34.590 that may signal security threats. 70 00:03:34.590 --> 00:03:37.470 While logging provides the raw data, 71 00:03:37.470 --> 00:03:41.580 monitoring actively focuses on identifying indicators 72 00:03:41.580 --> 00:03:43.680 of attack or compromise. 73 00:03:43.680 --> 00:03:45.810 In a large enterprise network, 74 00:03:45.810 --> 00:03:48.780 monitoring is crucial to quickly detect 75 00:03:48.780 --> 00:03:50.880 unauthorized access attempts, 76 00:03:50.880 --> 00:03:55.110 policy violations, or other suspicious behaviors. 77 00:03:55.110 --> 00:03:59.280 For example, monitoring tools can be used to scan logs 78 00:03:59.280 --> 00:04:04.050 to identify anomalies such as repeated failed login attempts 79 00:04:04.050 --> 00:04:06.930 or access from unusual locations, 80 00:04:06.930 --> 00:04:09.990 which could indicate a security incident. 81 00:04:09.990 --> 00:04:13.590 Monitoring large scale environments is challenging 82 00:04:13.590 --> 00:04:17.910 due to the volume of log data generated every day. 83 00:04:17.910 --> 00:04:21.930 Security teams cannot manually review all logs, 84 00:04:21.930 --> 00:04:26.790 so automated tools are used for flagging critical events. 85 00:04:26.790 --> 00:04:29.460 These tools use predefined rules 86 00:04:29.460 --> 00:04:33.810 and algorithms to analyze logs for abnormal patterns, 87 00:04:33.810 --> 00:04:38.730 such as unauthorized system changes or potential breaches. 88 00:04:38.730 --> 00:04:40.590 When an anomaly is detected, 89 00:04:40.590 --> 00:04:44.310 the system generates an alert allowing security teams 90 00:04:44.310 --> 00:04:48.900 to investigate and respond before the situation escalates. 91 00:04:48.900 --> 00:04:51.690 Security information and event management, 92 00:04:51.690 --> 00:04:55.530 or SIEM systems, play a key role in monitoring 93 00:04:55.530 --> 00:04:58.680 by providing real time alerts, dashboards, 94 00:04:58.680 --> 00:05:02.220 and data correlation across multiple sources. 95 00:05:02.220 --> 00:05:06.690 SIEMs help security teams detect sophisticated attacks 96 00:05:06.690 --> 00:05:09.840 that span different systems or regions. 97 00:05:09.840 --> 00:05:14.460 For example, a SIEM can connect multiple unauthorized access 98 00:05:14.460 --> 00:05:17.610 attempts across different geographical locations 99 00:05:17.610 --> 00:05:19.140 and trigger an alert. 100 00:05:19.140 --> 00:05:22.200 SIEMs not only detect potential threats, 101 00:05:22.200 --> 00:05:24.690 but also offer tools to support 102 00:05:24.690 --> 00:05:27.810 rapid investigation and response. 103 00:05:27.810 --> 00:05:31.290 Of course, there are other essential tools for monitoring, 104 00:05:31.290 --> 00:05:33.810 including intrusion detection systems 105 00:05:33.810 --> 00:05:36.960 and dedicated log monitoring solutions. 106 00:05:36.960 --> 00:05:40.770 For example, tools like Graylog and SolarWinds 107 00:05:40.770 --> 00:05:45.210 provide advanced log monitoring and analysis capabilities, 108 00:05:45.210 --> 00:05:48.810 allowing organizations to track network activity 109 00:05:48.810 --> 00:05:50.250 in real time. 110 00:05:50.250 --> 00:05:52.890 Also, intrusion detection systems help 111 00:05:52.890 --> 00:05:56.370 by monitoring network traffic for suspicious behavior 112 00:05:56.370 --> 00:06:00.510 and generating when unusual patterns are detected. 113 00:06:00.510 --> 00:06:04.290 Overall, log monitoring helps organizations maintain 114 00:06:04.290 --> 00:06:06.990 continuous oversight of their networks 115 00:06:06.990 --> 00:06:10.320 and respond quickly to security concerns. 116 00:06:10.320 --> 00:06:14.610 So remember, logging and monitoring enable detecting, 117 00:06:14.610 --> 00:06:16.980 investigating, and responding, 118 00:06:16.980 --> 00:06:19.470 to potential security incidents. 119 00:06:19.470 --> 00:06:24.030 Logging captures detailed information about user actions, 120 00:06:24.030 --> 00:06:26.850 system changes, and access attempts, 121 00:06:26.850 --> 00:06:30.810 providing a record of activity within a network. 122 00:06:30.810 --> 00:06:32.490 Monitoring on the other hand, 123 00:06:32.490 --> 00:06:36.150 focuses on reviewing these logs in real time 124 00:06:36.150 --> 00:06:38.550 or at intervals to identify 125 00:06:38.550 --> 00:06:41.490 unusual or unauthorized activity. 126 00:06:41.490 --> 00:06:44.730 Security information and event management systems 127 00:06:44.730 --> 00:06:48.990 and other tools help automate the collection, aggregation, 128 00:06:48.990 --> 00:06:51.150 and analysis of logs. 129 00:06:51.150 --> 00:06:54.720 So together, logging and monitoring ensure 130 00:06:54.720 --> 00:06:58.770 that organizations can quickly detect and respond to 131 00:06:58.770 --> 00:07:01.830 security threats while maintaining compliance 132 00:07:01.830 --> 00:07:04.773 with policies and regulations.