Breakthrough Science for Anomaly Detection
- For IT Analytics on Amazon Web Services (AWS)
- Complex pattern detection, automated modeling, adaptive learning, reimagined user interface
- Read The Science of Anomaly Detection white paper and the Grok Blog
- Read Grok Use Cases for examples of how IT operators use Grok
Complex Pattern Detection
Catch unusual patterns that others miss.
Grok leverages sophisticated algorithms to analyze connected datastreams, such as those from AWS CloudWatch. Through complex pattern analysis, Grok identifies abnormal conditions or gradual trends – situations that tools based on thresholds or simple statistics can easily miss.
IT professionals can use these early insights to prevent potential problems while avoiding false positives or annoying alerts.
Better support for your systems. Less work for you.
Grok automates the time-consuming process of data analysis and predictive modeling. Once installed, Grok analyzes your data and automatically builds models for each of your systems.
This automation empowers you to deploy Grok without being constrained by the availability of data scientists. And, without the burden of baselining threshold-based tools, you can scale the power of Grok across your IT landscape.
Adjust to changes automatically.
Your business demands that your systems change quickly and frequently, often making months of analytics and modeling obsolete in a short amount of time.
Grok is different. Grok learns continuously in real time - with every data point. Grok doesn't require a continuous investment in re-modeling as the statistics of your high-velocity data stream change. This difference empowers you to be dynamic and nimble - and able to move with the pace of the business.
Mobile User Interface
Monitoring reimagined. Whenever, wherever.
Our smartphone-centric design puts the power of our deep science into your hands, enabling you to see real-time updates on your systems. The most anomalous systems are ranked highest, so you can assess them easily and drill down quickly for more information.
Whether your systems are experiencing problems or humming along without issues, you can be assured that you have visibility into the health of the systems you care most about.
Grok is currently only available for Android.
Grok Feature Summary
Monitor AWS environments via CloudWatch or other streaming data
- Elastic Block Store (EBS)
- Elastic Compute Cloud (EC2)
- Relational DB Service (RDS)
- Elastic Load Balancing (ELB)
- Simple Queue Service (SQS)
- AWS AutoScaling Groups
- Other system data via custom metrics
- Monitor custom Grok Auto Stacks
- View list of monitored instances in hour/day/week
- Rank-ordered to see most anomalous instances
- Charts probability of an unusual state or anomaly
- Click to see details on instance metrics
Get superior system insights from Grok
- Learns complex patterns and then identifies likelihood of unusual behavior
- Detects patterns that thresholds miss
- Continuously learns
- Automatically adapts to changes
- Click on a metric to see metric data detail
- Scroll back in time to see past data
- Updated in near real-time
- User-controlled notifications to provide flexibility and minimize false positivies
Simple and secure setup
- Simple and secure setup via Amazon AMI
- Configuration via Web UI or CLI
- Extensible architecture (i.e. platform)
- Android-only Mobile client provides data visualization of anomaly likelihood for monitored instances and metrics by three time-view options: hour, day, or week
How Does Grok Work?
Tell Grok What to Monitor
You don't need to be a data scientist to use Grok. Setup is simple! Just tell Grok which AWS instances or services to monitor, and Grok will do all of the work for you. Grok is preconfigured to support most AWS instance types, service types, and standard metrics. Grok also supports custom metrics, so you can monitor the things that matter most to your business.
Stream Data from CloudWatch
After setup, Grok will connect to Amazon CloudWatch via the Grok API. Grok collects metric data from CloudWatch continuously to ensure the most up-to-date data is analyzed and ready for you to review. Grok analyzes each stream individually to give you insightful information about the state of your systems.
The Grok engine and API are also scalable so you can monitor the most critical systems on which your business depends.
Grok Builds and Maintains Models Automatically
When Grok receives your data, it kicks off an automated model building process. If you're modeling thousands of data streams, Grok does this for each one, resulting in thousands of individually tuned models. What might take a data scientist days or weeks to accomplish for just one data stream, Grok can do for you in minutes across thousands.
Through this process Grok discovers many unique things about each data stream. Which data is most helpful? Are there recurring patterns in the value you want to predict, such as by time of day? How random is your data? Which metric values are common and which are unseen?
As soon as Grok builds the initial model for each stream, it can detect anomalies immediately. But Grok doesn't stop there, Grok continues to learn automatically with every new data point. If the patterns in your data change, which will inevitably happen, Grok will learn these new patterns and adapt without any interruption or intervention.
And the best part is that you don’t need to set a single threshold! Woohoo!
Get Insights. Take Action.
Grok leverages a reimagined, smartphone-centric way to monitor your IT systems. Grok includes a native mobile client that receives and visualizes your anomaly data in near real-time. For each metric and instance, Grok generates predictions based on past patterns; Grok then compares these predictions against actual data to determine the likelihood of an anomalous condition.
Visualization of anomaly data in the mobile app makes it easy to understand if a high or low probability exists that a server or application is in an unusual state. The Grok mobile user interface also shows you the most anomalous AWS instances first, so you can see what’s most interesting - and where you may need to take action. Because Grok leverages state-of-the-art predictive algorithms, Grok can help you detect unusual conditions before they turn into real problems or business downtime.
What You Get
- Advanced anomaly detection
- Continuous learning of your systems
- Simple setup and automatic maintenance
- Android user interface
- Custom metrics and auto-scaling support
- Monitor a maximum of 200 instances or custom metrics
- Requires a dedicated EC2 AWS Instance
- Grok 1.5 Master Guide Master Guide
- Grok 1.5 Datasheet Datasheet
- Quickstart Guide – Getting Insights from Grok Quickstart Guide
- Use Case – Grok automatically adapts to a system change Case Study
- Use Case – Grok catches abnormal increase in latency Case Study
- Use Case – Grok detects a bad code push Case Study
- Use Case – Grok identifies an unusual server pattern Case Study
- Grok Custom Metrics Grok Support Guide
- Grok AWS ASG and AutoStacks Grok Support Guide
- Generate Restrictive Credentials Grok Support Guide
- Upgrade AWS EC2 Instance Size Grok Support Guide
- Grok Command Line Interface to interact with a Grok Server Grok CLI Guide
- Integration to Datadog Grok Support Guide
Numenta has apps that mimic the way the brain works
Article via VentureBeat - Jul 9, 2014
The Big Data Dump: How Info-Hoarding Can Overwhelm Startups, Spy Agencies
Article via Bloomberg- Jun 3, 2014
Grok for IT Analytics at AWS re:Invent
November 11 - 14, 2014.
The Venetian – Las Vegas, NV.
Cleanup on ASG 3 Jul 22 2014 | Joe Block
I was at the AWS summit in NYC, and when I got back to California, I saw a notification from Grok that our web server auto scaling group (ASG) was acting anomalously. At first I thought it was only because we were seeing heavier than usual traffic since we were mentioned in Information Week and VentureBeat, but when I checked it out in Grok’s mobile app, I noticed a couple of odd and interesting things.Read More
Detecting Anomalies in Stock Volumes Jul 15 2014 | Viraj Sinha
I decided to apply Grok and the Cortical Learning Algorithm to stock data: have the CLA learn patterns and alert me to anomalies. By using Grok to monitor volume anomalies, I can get a notification on my phone when a given share is being traded in an unusual way.Read More
Caught Red Handed Apr 22 2014 | Jared Casner
Like most engineering managers, I like to know when someone is manually touching one of our servers. That’s especially true for production systems, but also applies to QA servers. So, imagine my chagrin when Grok caught me red handed, not just once, but twice this week...Read More
Anomaly of the Week Apr 04 2014 | Subutai Ahmad
It was Thursday night, March 20th, at 8pm. I was cleaning up the kitchen and getting ready to wind down when my smartphone buzzed: Grok had sent an alert. I know that with Grok it just takes a few seconds to check things out, kind of like text messages. So I clicked on the notification and looked at the graph. What followed was interesting and eye opening!Read More