Centralized logging should be common practice today in any environment, however it’s still surprisingly common to hear arguments against the practice, citing cost, complexity and usefulness as key reasons to not pursue a centralized logging strategy.
Why You Need Centralized Logging
Though not intended to be a post on the broader topic of centralized logging, I can’t resist laying the groundwork for the rest of this post with some key drivers for why centralized logging is right for you. Also, if you’re not doing any centralized logging, you’re certainly not going to care about making Windows devices first-class citizens in that system, are you? So, what are some reasons you should be embracing centralized logging?
The best decisions are supported by data. Centralized logging provides a vast source of data to mine for information ranging from problems, to performance issues. Performing this from what in essence becomes one data source is much more efficient than crawling and aggregating data in realtime from distinct, distributed data sources.
Having a centralized source allows you to quickly correlate and understand the effect of actions or issues between systems. Good centralized logging systems offer features such as dashboards and complex filtering – this isn’t just syslog anymore! There’s ample opportunity to integrate log data into your monitoring and alerting infrastructure when it’s centralized and easily accessible.
Many businesses are part of industries that are subject to *requirements* related to logging. Every business has the potential to suffer a security incident. A centralized logging system provides a source of data that can provide integrity assurance and retention capabilities not often viable in a distributed per-system log storage model. Would you rely on the logs on a server if you suspected that server had been compromised?
There are some amazing centralized logging solutions available, in particular open source solutions (my focus for this post) such as Graylog2, Kibana, Logstash and others. Many of these you’ll hear used in stacks, e.g Logstash and Kibana are coupled with ElasticSearch to form the ELK stack, Graylog2 uses ElasticSearch, and so on. What results is a powerful combination of search, processing/filtering and visualization. We use Graylog2 heavily, finding it to better meet the needs of many security conscious environments and customers. There are many commercial options as well, with varying cost-benefit tradeoffs to consider.
Ok, now about those Windows Systems
So now you’re sold on centralized logging, why the emphasis on Windows? Windows systems (Server in particular) are a part of the modern enterprise and offer capabilities that many organizations embrace, Active Directory Domain Services being a great example. When the core of your secure authentication directory system is Windows Server, collecting those logs is essential.
Since the 1980s, syslog has been a foundation of Unix and Linux logging, supporting redirection capabilities critical to a centralized logging strategy. Windows systems have historically relied on agents to gather Windows Event Log entries and forward to a centralized logging system of choice. Microsoft introduced event (i.e. log) forwarding and collection capabilities in 2008 (via Server 2008). Based on the Distributed Management Task Force WS-Eventing standard, the collection and forwarding capabilities are robust, but tend to still be better suited to either homogeneous Microsoft environments, or third-party solutions capable of supporting the WS-Eventing standard.
For the many organizations that had centralized logging capabilities prior to 2008 by way of syslog, and for organizations based on heterogeneous environments with a healthy mix of Linux and other open source solutions in addition to Windows, there’s still a considerable appetite to integrate Windows event logs (and other non-event logs) into a scalable, open source, platform agnostic solution such as Graylog2. Fortunately, similar to centralized logging solutions there are a variety of log forwarders to help address this client-side need. Below I describe two agents (there are many others that you could also consider) to help get you up & running.
- Snare: Perhaps the most straightforward agent, and provided by Intersect Alliance. The open source version of the Windows agent provides basic centralized logging capabilities, but for more advanced features such as transport encryption, the enterprise agent would be required. This is your “let’s get logging” agent; established, stable and relatively low effort to configure and maintain. Think of it like the butter knife of Windows forwarders.
- nxlog: If Snare’s your butter knife, nxlog is your Swiss Army Knife. With multi-platform support and a gigantor feature list, nxlog is likely to be your next step when you run into some of the limitations of Snare and other simple forwarders. Capturing data from more locations than event logs? Check! Secure transport? Of course! Without question there’s more of a learning curve, but for most organizations serious about log collection on windows (and other platforms!), it’s well worth the time investment. Oh, and GELF support; if you don’t know what GELF is or why you want it, read on.
Modern centralized logging tools also overcome many pain points from traditional syslog collection systems – message structure retention and length limitations are two great examples. The Graylog Extended Log Format (GELF) addresses these concerns, supporting structured data, compression, and overcoming the 1024 byte syslog message size limit (chunking messages if necessary). Retaining structure enables improved review and decision making, including the potential for event-driven automation based on log data. If you’ve ever seen the “straight to syslog” dump of a Windows event log entry, you know it isn’t pretty…
2014-07-16 11:07:24 host.testdomain.local user notice 0d MSWinEventLog 1 Security 764112764 Wed Jul 16 11:07:24 2014 4634 Microsoft-Windows-Security-Auditing TESTNET\Paul N/A Success Audit dc1.testdomain.local Logoff An account was logged off. Subject: Security ID: S-1-5-21-11111111-1111111111-111111111-11111 Account Name: Paul Account Domain: TESTNET Logon ID: 0x85a06c8f8 Logon Type: 3 This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer. 764073953
Platform Agnostic Recommendations
With all the potential that modern centralized logging supports, there’s the potential to misuse it (related to Windows logs or otherwise), so I end with some words of wisdom (and caution)!
Log what matters – You shouldn’t log everything just because you can. Centralized logging requires large amounts of data storage, and though highly commoditized, log growth can quickly get out of hand. Be selective about what you’re logging, and have a documented plan outlining audit objectives (a requirement of standards such as NIST 800-53!)
Understand existing requirements or work with appropriate stakeholders to define retention requirements for log data.
Think critically about what you’re logging. When defining audit objectives, consider privacy, organizational policy, standards and regulatory requirements. You don’t want to suddenly realize you’ve been storing credit card numbers for all your POS systems in your logging system during a PCI audit. Further, some systems are notoriously noisy when it comes to log generation. You can and should filter events that don’t provide any value to you, either client-side or server-side.
Carefully plan who can access what log data, and why. In some organizations, technical teams will want and need to see all log data. In others, access to specific types of data or data from specific systems will need to be limited. It’s important to plan your logging strategy before deploying a system as these access and privacy decisions can have an impact on your solution.
Centralized logging is no longer a nice to have feature (including for organizations with a heavy reliance on Microsoft systems), rather it’s a critical part of any organization that wants to make informed, data-driven decisions. Best of all, you can assemble your own solution using open source tools. There’s no excuse now!
Are you sold on logging now? Need help getting that up and running? Reach out to see how OpsBot can help!