Because logs are primarily unstructured data, they are well suited to batch data analysis of a discrete event. However, a big data approach to logs makes them poorly suited to the real-time search and stream processing required for timely alerts. The high volumes of disk I/O and network load needed for log exploration are much better aligned to post-hoc analysis, as opposed to the high metric throughput typical of a time series database used for infrastructure monitoring.
For cloud environments, whose goal is to scale infrastructure elastically, you need a purpose-built system focused on metrics and analytics. Real-time aggregation is a job not fit for batch analytics because alerting requires much faster, more flexible insights. Log analysis for deeper exploration and investigation is ultimately a great complement to an infrastructure monitoring solution that handles real-time analytics and alerting on time series data.
APM + Infrastructure Monitoring + Log Management
Today, a more modern approach to infrastructure monitoring can help rationalize the role of the APM and log management tools that development and operations teams already use. The data and insights at each stage of the journey shouldn’t be viewed in three separate silos. Today’s smartest product organizations are managing both effectiveness and cost by flowing insights across all stages of the application lifecycle. SignalFx is the most advanced way to aggregate and alert on streaming metrics, helping today’s dev and ops teams fill the gap between APM’s pre-flight performance engineering and log management’s post-mortem event analysis. SignalFx’s real-time visibility into and analytics on the live production environment also help rationalize your existing investments with better overall results.
Download our new ebook: APM + Logs Need Infrastructure Monitoring »