Network monitoring is ideal for getting a real-time view of your connected environment, and with reports, you can look back in time too. Logs are key to this rear-view mirror look, as they contain all the data for all the elements you are monitoring.
But without network log archiving, you can only look back so far. Did you know that according to an IBM/Ponemon study, it takes an average of 287 days to discover and contain a data breach? How are you going to know what happened back then, and how do you make sure it doesn’t happen again if you can’t analyze those nine-month-old logs? You can’t.
To support security forensics and truly understand the network, Smart IT pros implement network log management best practices (which start with logging and analyzing in the first place) and perfect those practices with network log archiving.
Think of a log as a "journal-of-record" for every event or transaction that takes place on a server, computer, or piece of hardware on your network. Just about everything in your IT environment has some kind of log. Microsoft systems generate Windows Event Log files. UNIX-based servers and devices use the System Log (or Syslog) standard. Apache and IIS generate W3C/IIS log files. Log files contain a wealth of information to reduce an organization’s exposure to intruders, malware, damage, loss and legal liabilities.
Log data needs to be collected, stored, analyzed and monitored to meet and report on regulatory compliance standards like Sarbanes Oxley, Basel II, HIPAA, GLB, FISMA, PCI DSS, NISPOM. This is a daunting task since log files come from many different sources, in different formats, and in massive volumes, and many organizations don’t have a proper log management strategy in place to monitor and secure their network. For more details about log types, check out the WhatsUp Gold Best Practices for Event Log Management page.
Logs are detailed records of events, like a pilot’s flight log or a trucker’s mileage log. In each of these cases, the log tells you where you’ve been. In the case of your network, the log contains a detailed record of everything that happened—including user actions. And by network, that includes any network resource you choose to monitor, including servers, devices, applications such as databases and websites, etc. The record includes events and performance data, which is great for applications but also critical for knowing how network pieces have performed over time.
Wow. That’s a lot of stuff. It is, and it can be expensive and cumbersome storing all this data. That’s why so many keep short-term logs and jettison the older data. But it is this older data that lets you perform forensics on that breach you just discovered, but which actually happened 280 days ago.
Log archiving offers tremendous IT value but raises a host of issues. “When IT managers consider logging and archiving, they are faced with a dilemma: Keep enough data, and audit and regulatory needs are met, business continuity is maintained, and recovery after disaster goes off smoothly. Keep too much data, though, and the cost associated with storing that data and the resources needed to maintain the archives could skyrocket, outweighing many of the benefits,” Computerworld argued.
“What's needed is a careful, business-based balance between security and storage. Simple storage of the data isn't enough. How data is stored and how the associated threats to it are mitigated are critical parts of the puzzle. Even the most sophisticated storage-area network (SAN) isn't much use if an attacker can access the logs and delete or otherwise tamper with them.”
Fortunately for WhatsUp Gold users, simple and intuitive log archiving is included with Log Management. IT can now easily comply with regulatory requirements and preserve historical data without impacting performance. You can customize retention periods, choose storage locations and filter which logs get archived. IT can also search, purge or restore from archived logs all within the familiar WhatsUp Gold interface. For more details visit our Log Management feature page.
The economics of archiving are vastly different from that of nearer-line storage, especially if you use compression techniques to squeeze down all this data. This compressed and economically stored data can always be decompressed and loaded into a more active system for analysis.
Other reasons to archive network logs include being able to:
Network Log Archiving is not just a best practice, it is essential for some compliance regulations that require logs to be kept for a year, or the far stricter SOX which compels financial institutions to retain logs for a full seven years. Those of you in banking know exactly what I am talking about.
“Enterprises have regulations to deal with. Whether they’re governmental or industrial, almost all of which require compliance via activity and event data logging. Having a centralized logging system via log management improves the efficiency of these compliance efforts. However, what exactly your enterprise will need to log and for how long will be different for every enterprise. Datalog solutions should be capable of flexibility and adapting to enterprise-specific audit controls,” Solutions Review argued.
Your network logs are crucial for security forensics and having a history of your network for planning upgrades. With archiving, this crucial data is held in a separate location, and is thus completely safe from data loss due to accidental deletion or a cyber-attack.
At the same time, these event logs can be critical if your network itself is compromised and can be used to restore function.
If you collected and kept every bit of log data possible, you would be overwhelmed with data, and footing the bill for massive and ever-growing storage capacity. You should instead collect what you need and archive what you REALLY need.
That begins with collecting data on essential devices—not every little IoT doohickey you have laying around—and then filtering the data collected.
Network Monitoring can collect massive amounts of data and fill your storage database to the point of bursting—which can dramatically slow down your system. Advanced filters, along with the conservative use of applicable settings and/or configuration, can keep storage needs in check.
Logs are critical for identifying trends and detecting patterns. Archiving allows a deeper view as you can go back and load older historical data into whatever analytics engine you choose. This way you can go deep into identifying and understanding network activity. This information can drive future network upgrades and architectures as well as help in the design and implementation of new security measures.
Get our latest blog posts delivered in a weekly email.