DiscoverCribl: The Stream Life
Cribl: The Stream Life
Claim Ownership

Cribl: The Stream Life

Author: Cribl

Subscribed: 5Played: 253
Share

Description

Welcome to Cribl: The Stream Life, a podcast for IT pros trying to take control of their observability data with a no-compromise approach. With each episode, our hosts will cover the latest insights, trends, and emerging technologies to help IT organizations achieve observability in their operations. We’ll also address specific challenges we’ve seen with hundreds of enterprises over the last several years and sketch out the fundamental capabilities required to overcome them.
108 Episodes
Reverse
In this episode of The Stream Life Podcast, Zac Kilpatrick and Bradley Chambers chat about Cribl's Partner Awards! During our annual company kick off, we were thrilled to announce the Cribl Partner of the Year Award Winners, who are recognized for contributions, loyalty, and mutual commitment to delivering high value to customers within our partner ecosystem. Resources Read the blog to hear all the winners If you want to automatically get every episode of the Stream Life podcast, you can subscribe on your favorite podcast app.   Cribl, the Data Engine for IT and Security, empowers organizations to transform their data strategy. Customers use Cribl’s suite of products to collect, process, route, and analyze all IT and security data, delivering the flexibility, choice, and control required to adapt to their ever-changing needs. We offer free training, certifications, and a free tier across our products. Our community Slack features Cribl engineers, partners, and customers who can answer your questions as you get started and continue to build and evolve. We also offer a variety of hands-on Sandboxes for those interested in how companies globally leverage our products for their data challenges.
CriblCon 2024

CriblCon 2024

2024-03-0614:52

In this episode of The Stream Life Podcast, Mike Dupuis and I chat about CriblCon 2024, what's on the agenda, and why all IT and security engineers should attend. Resources Register for CriblCon 2024! If you want to automatically get every episode of the Stream Life podcast, you can subscribe on your favorite podcast app. Cribl, the Data Engine for IT and Security, empowers organizations to transform their data strategy. Customers use Cribl’s suite of products to collect, process, route, and analyze all IT and security data, delivering the flexibility, choice, and control required to adapt to their ever-changing needs. We offer free training, certifications, and a free tier across our products. Our community Slack features Cribl engineers, partners, and customers who can answer your questions as you get started and continue to build and evolve. We also offer a variety of hands-on Sandboxes for those interested in how companies globally leverage our products for their data challenges.
Cribl for Startups

Cribl for Startups

2024-01-2415:08

In this episode of The Stream Life Podcast, Nick Heudecker and I chat about Cribl for Startups. Cribl for Startups is a new program to support early-stage startups that are building the next generation of data solutions for IT and Security. Resources Nick's Blog Post Press Release If you want to automatically get every episode of the Stream Life podcast, you can subscribe on your favorite podcast app.
In this livestream, Ahmed Kira and I provided more details about the Cribl Stream Reference Architecture, which is designed to help observability admins achieve faster and more valuable stream deployment. We explained the guidelines for deploying the comprehensive reference architecture to meet the needs of large customers with diverse, high-volume data flows. Then, we shared different use cases and discussed their pros and cons.  Cribl’s Reference Architectures provide a way for admins to get 70% of the way towards deploying Cribl Stream. The sample environment below is a template for sending data to many destinations while minimizing data egress costs. It incorporates solutions to some of the challenges typical larger organizations might face.  MS Azure Worker Group In this sample environment, the leader is up in Cribl Cloud and managed by Cribl. On the right-hand side, you’ll see an Azure worker group. There are two reasons to consider putting a worker group in a different cloud provider. The first is to be as close to the data you're collecting as possible. By keeping the data close, you can minimize the amount of processing necessary and cut egress costs. With this setup, you’re also reducing the risks of having competing workloads. Failing small is much better than failing big. Additionally, when establishing a security or observability data lake, you don't need to put all that data in the same data lake, S3 bucket, or blob storage. With Cribl, you can have them in different places and still be able to replay against all of that data. We often see customers with Azure and AWS workers using Cribl-to-Cribl connectivity between the two clouds to exchange data. This way, they can avoid building custom code or dealing with the vagaries of exchanging data between clouds. On-Prem General-Purpose Worker Group The next worker group in our sample architecture above is an on-prem, general-purpose worker group. With this worker group, you can combine most of your data sources and have them go to one worker group in your data center. This is especially useful if you have a lot of Splunk universal forwarders, Cribl Edge agents, and Filebeat agents — you'll want to send those to a dedicated worker group so you're not competing for different workloads. Another big reason for this approach is segmentation. For example, if you need to separate your PCI or PHI workflow, you can use this setup to break up your data or meet compliance requirements. If you need to upload that data to an Elastic or Splunk cloud, having the Cribl Stream worker group allows you to stage your data, manage it, and get it to those destinations. Syslog Worker Group Another architectural consideration worth looking into is having one Syslog worker group. This allows you to do your commit-and-deploys once instead of one region at a time. A lot of organizations struggle with the contention that high-volume Syslog causes. Adding an agent workload can make the situation worse, so having separate worker groups allows you to scale. The difference between this worker group and others is Syslog groups have load balancers that will send data to the local workers in that data center. In Cribl Stream, there will still be one logical Syslog worker group to manage, reducing administrative burden and the maintenance required. If you take one thing away from reading this post or watching the live stream, please DO NOT send your data to a single Syslog destination port! You'll get the best results by getting as many workers involved as possible — do everything you can to avoid being pinned to a single core. Cribl Cloud Worker Group With Cribl Cloud, you will also get at least one worker group by default that you can allocate to all your AWS data sources — like in the sample architecture. But you can also send all of your cloud, on-prem, and other non-AWS data sources there. Either way, you won't have to manage as much infrastructure. Instead, you can leverage the Cribl Cloud worker group and the Cribl Cloud leader if your use case allows for it. This is especially important for threat surface reduction. Taking data in from multiple SaaS platforms means opening up your perimeter to everything that Cloudflare could produce, which is probably half the entire internet. Cribl Cloud can handle all of those threats and keep you secure. Replay Worker Group The last worker group in this reference architecture that people don't typically consider is the Replay worker group. It’s a great practice to allocate your replays to a separate worker group, where the workload can be spun up and spun down — instead of on your production worker groups where you're processing real-time streaming data. Using your production worker group for replay can suddenly add terabytes of data to your existing live data flows and slow everything down. A minimal-cost, ephemeral replay worker group lets you scale up to meet your needs without interrupting your production workloads. A recent customer took advantage of this by deploying their replay worker group in AWS ECS. As more data gets requested and downloaded, ECS spins up additional instances. The worker group scales larger as more data is retrieved and then scales down if there’s nothing to do. Choice and Control Over All of Your Data When you have multiple worker groups, you don’t have to worry about going to different places to manage them — it can all still be done by one Cribl leader. You can also have multiple data lakes and replay from all of them via one central location within Cribl. This flexibility gives you complete control to make the best choices for you. So, if your security team wants to use Azure for its data lake and your operations team wants to use AWS, it’s no problem. Or, if you want to use one S3 bucket for forensics and another for yearly retention, you have that option available. The best part is that all the data in your data lake is vendor-neutral. You can return that data to Cribl Stream using replay and send it to any tool you want. Check out the full live stream for insights on integrating Cribl Stream into any environment, enabling faster value realization with minimal effort. Our goal is to assist SecOps and Observability data admins in spending less time figuring out how to use Cribl Stream and more time getting value. Don't miss out on this opportunity to enhance your observability administration skills. More Videos in our Cribl Reference Architecture Series Introduction to the Cribl Stream Reference Architecture How the All in One Worker Group Fits Into the Cribl Stream Reference Architecture Scaling Syslog Scaling Effectively for a High Volume of Agents How SpyCloud Architected its Cribl Stream Deployment  
In this livestream, I talked to Ryan Saunders - Manager of Security Operations at SpyCloud, about how he used the Cribl Reference Architecture to build a scalable deployment. He explained how this approach enabled SpyCloud to grow alongside its evolving needs without requiring significant rework. The reference architecture also facilitated a repeatable data-onboarding process, reducing administrative time and allowing the team to focus on critical security and data analysis tasks. SpyCloud is a cloud-native organization that generates enormous amounts of data — from hosted email and EDR, sales solutions, and the rest of their sprawling cloud architecture. Before implementing Cribl Stream, they had too many sources and too little time to figure out how to integrate all of them. Saving Valuable Engineering Time Traditional on-prem environments can have many sources, but they generally come from a single area that makes it possible to capture them with a single set of agents. Because of their sprawling cloud architecture, Ryan and his team didn’t have that luxury. During our conversation, Ryan pointed out that engineers come to work at SpyCloud to work in security, not to become a data butler. They don't necessarily know how to architect large data pipelines — they just pull the data in and go to work on it. To that end, the first problem they solved with Cribl Stream was streamlining the process of bringing sources into their detection analytics platform. Data now flows in natively from a source like AWS instead of via a TA or other inefficient, incomplete method. Flexibility in Scaling Security Architecture SpyCloud can’t afford to have data held up in processing — once all their data comes in, it needs to be processed immediately so their security detections fire in real-time. Cribl’s Reference Architecture played a very important role in onboarding their sources and getting things to operate seamlessly. There are times when Ryan and his team get little to no advance notice of a new product or customer, so there may not be much time to add to their logging pipeline. Without Cribl Stream, planning and execution may take weeks or months. But the right tools and a properly designed architecture allow them to scale up in minutes, if not automatically. Splitting Up Worker Groups Spycloud separates worker groups based on data volume workflow and as a way to mitigate risk. Instead of having one large worker group, they have a separate one on the internet with open ports, so they’re able to fail small and manage their blast radius. It’s good practice to split up your worker groups not only by load, but also by connection type and according to your security needs. When I asked Ryan if he was concerned about the management overhead of having a bunch of worker groups, he compared the experience to his days as a Splunk admin. Setting up different indexer clusters was a nightmare because maintenance efforts only scaled linearly. With worker groups, there’s one interface to manage everything. Ryan can copy settings by cloning a worker group, or add and remove pipelines from different worker groups — all from one interface. He sums it up quite nicely: “The biggest win for us with Cribl Stream is that we can upgrade everything from one single pane of glass. I don't have to go out and plan a 12-hour overnight weekend upgrade of my indexers. I just click upgrade in that worker group, and it happens.” - Ryan Saunders, Manager of Security Operations at SpyCloud Taking Advantage of Cribl Edge Ryan and the team at SpyCloud also have Cribl Edge deployed as a log collection agent on all their servers. They have a dozen Edge fleets collecting data that’s sent back to Cribl Stream for processing. Managing fleets in Cribl Edge is just as easy as managing worker groups in Cribl Stream. They have the flexibility to control separate configurations for Windows, Linux, production tests, and other products within the same interface. SpyCloud also uses Cribl Edge to consolidate logging agents within the organization because it’s easier for them to have one agent that multiple teams can control. His team sends the data they need for security to their own tools, and their DevOps teams can extract the operations data they need as well. Everyone can control and manage their data however they see fit, so it's a win for everybody. Best Practices for a Scalable Cribl Stream Deployment Ryan has many years of experience using Cribl’s tools within different organizations and environments, so he has learned some very valuable lessons along the way. His first deployment involved trying to run Kubernetes in a large environment with one giant worker group — so he quickly learned about the importance of splitting them up. You want to be able to do this easily, especially in highly regulated environments. Multinational organizations may not be able to commingle data or send it across national borders. Companies processing healthcare data have strict requirements for handling PII. Even if you don’t fall into either of these categories today, business growth or regulatory requirements might change that, so you’ll need to be able to adjust quickly to split certain data out. Taking advantage of auto-scaling has also proven beneficial for Ryan, and everyone can take advantage of it — just don’t forget to create limits. You want to avoid scaling up until an AWS region explodes, so you don’t wake up one night and find 1000 Kubernetes nodes running because something went sideways. Explaining that bill won’t be much fun the next day. Watch the full livestream to see more on how SpyCloud uses Cribl Stream and Cribl Edge to streamline the onboarding process and get more visibility and insights from their business data. You’ll also learn how to use the Cribl Reference Architectures as a starting point for a scalable deployment so you can reduce administrative time and free up your team to focus on critical security and data analysis tasks. More Videos in our Cribl Reference Architecture Series Introduction to the Cribl Stream Reference Architecture How the All in One Worker Group Fits Into the Cribl Stream Reference Architecture Scaling Syslog Scaling Effectively for a High Volume of Agents  
In this Livestream conversation, I spoke with John Alves from CyberOne Security about the struggles teams face in modernizing a SIEM, controlling costs, and extracting optimal value from their systems. We delve into the issues around single system-of-analysis solutions that attempt to solve detection and analytics use cases within the same tool. We explored the strategic limitations of this type of security architecture, presenting alternative options for effectively mixing and matching data platforms. Be sure to watch the full conversation to get on the path toward achieving the optimal combination of data management and cost control capabilities. If your security architecture is centered around a SIEM that houses all your security and operational data, it’s time for an upgrade. Data quantities, cyber attacks, and regulatory requirements are all on the rise, so having a single destination for your data leaves too much room for vulnerabilities. Until recently, buying a SIEM meant deploying its agents, putting all your data into it, and going on your merry way. You were almost 100% confined to that one framework — if you wanted to use UEBA, your vendor or one of their partners provided it. Operating outside your SIEM or bringing in third-party vendors was very limited. Observability Pipelines to the Rescue About five years ago, the concept of an observability pipeline emerged, allowing organizations to funnel their observability and security data through a consistent data plane. The idea of controlling where your data gets stored was born, and vendor-neutral considerations began gaining popularity. Admins can now make copies of events for their SIEM, data lake, UEBA solution, or someone else's data lake — easily turning one event into four events that power different parts of their security stack. By moving data into a data lake instead, admins can analyze data and build dashboards for operations teams without bloating their ingest. Teams have more choice and control over their data than ever before, so they can consider their specific needs when building out their infrastructure. The Benefits of a Data Security Lake During our discussion, John mentioned how this flexibility is no longer a wish-list item for his clients, but a necessity. As the industry transitions to cloud infrastructure and cloud-based computing, organizations require vendor-neutral data that supports their scalability efforts. There are a host of benefits you get from modernizing your security architecture. Reduced License Costs Routing data that isn’t needed for security to object storage is one of the best ways to reduce SIEM license costs. Ingest costs go down, and you avoid the upsell for archive data — around a 4- 8x markup — as opposed to using your own object storage or your SIEM cloud platforms archive. You can also store it in a vendor-neutral format, giving you enormous flexibility that you wouldn’t get otherwise. We recently worked with a developer team and their debug logs, routing them to a lower-cost S3 bucket instead of their SIEM. All we had to do was create a rule in Cribl Stream to route them to the data lake, and now they’re available to be restored whenever necessary. This is just one example of many where we can set customers up to meet their simultaneous need for availability but lower cost and overhead. Increasing Security While Decreasing Engineering Time When you can reduce your SIEM license costs, you no longer have to choose which data sources you can afford to collect. By removing the constraints for engineers that come from not having the raw data when needed, security teams can focus on security and not just moving data around. No more time spent on tasks like going out to a server to manually zip up and pull in logs. The result? Better detections, analytics, and security. Shared Data Within the Organization Each team has a different use case for the data the organization collects — having different pipelines to transform and send data to different sources is invaluable. Putting firewall, threat, traffic, and systems logs into a single destination is a great way to bloat your ingest. And not all logs from a single data source are security relevant. Routing some of them into a storage account or data lake will not only save on ingestion costs and create less noise for security teams, but you can also give access to relevant logs to your infrastructure, firewall, and other teams. Route your threat logs straight into the SIM, but send traffic and other logs straight into the data lake for your infrastructure network team. Compliance With Retention Requirements Another benefit of keeping raw copies of data is complying with retention requirements. If you're manipulating data before it goes into your SIEM, then you’re not adhering to some necessary standards. Transform events to get what you need for your SIEM, but keep unmanipulated, raw copies in your data lake. Your IR or legal counsel can control forensic copies. Meet Cyber Insurance Requirements As insurance companies get more sophisticated and start hiring engineers as auditors, they’ll dive deeper into your architecture than before. They’ll ensure you have a SIEM in place but also check to see if you’re putting the right data in and using it appropriately. Government auditors will want to see all your data sources and detections. They’ll be ready to write findings if you’re not following best practices. The prevalence of bad data or an overwhelming amount of data leads to various issues with detection, and drives costs higher and higher. It is extremely common to witness a year-over-year cost increase of up to 35%, which is clearly unsustainable. Watch the full livestream to hear John and I talk about alternative options for your SIEM platform, so you can be empowered to re-architect your data strategy. With the right strategies, SIEM platform challenges can be overcome, and we’re here to help as you embark on this transformative journey.  
In this episode of The Stream Life Podcast which was recorded after our announcement earlier this year, Adam Hogan from CrowdStrike joins the show to talk about the current challenges customers have with their data and the potential solutions. Resources Future-Proof Your Observability Strategy With CrowdStrike and Cribl Cribl Wins 2023 CrowdStrike Ecosystem Innovator of the Year Award If you want to automatically get every episode of the Stream Life podcast, you can subscribe on your favorite podcast app.
In this episode of Cybersecurity Awareness Month-themed episode of The Stream Life Podcast, Nick Heudecker and Jackie McGuire talk about the state of cybersecurity, "the people problem, and why hackers aren’t hacking into your network -- they’re just logging in. Resources Security Teams Are Struggling, and Cribl Is Here to Help If you want to automatically get every episode of the Stream Life podcast, you can subscribe on your favorite podcast app.
The cybersecurity industry is experiencing an explosion of innovative tools designed to tackle complex security challenges. However, the hype surrounding these tools has outpaced their actual capabilities, leading many teams to struggle with complexity and extracting value from their investment. In this conversation with Optiv's Randy Lariar, we explore the potential and dangers of bringing advanced data analytics and artificial intelligence tools to the cybersecurity space.
In this episode of The Stream Life Podcast, Nick Heudecker comes back on the show to talk about the recently released Gartner Hype Cycle for Observability and Monitoring.   Resources What is observability? Hype Cycle for Monitoring and Observability, 2023 If you want to automatically get every episode of the Stream Life podcast, you can subscribe on your favorite podcast app.
In this live stream, Cribl’s Ed Bailey and Ahmed Kira go into more detail about the Cribl Stream Reference Architecture, with a focus on scaling syslog. They share a few use cases, some guidelines for handling high-volume UDP and TCP syslog traffic, and talk about the pros and cons of some of the different approaches to tackling this challenge.  
In this episode of The Stream Life Podcast, Nick Heudecker joins the show to dive into an emerging buzzword in the IT and security industries: Telemetry pipelines. Nick explains what it is, why it's important, and why it's becoming popular in 2023. Resources Telemetry 101 If you want to automatically get every episode of the Stream Life podcast, you can subscribe on your favorite podcast app.
In this episode of The Stream Life Podcast, Cribl's Desi Gavis-Hughson and Exabeam's Chris Stewart join the show to talk about the big news out of Black Hat 2023: Cribl and Exabeam's strategic partnership! Resources Press Release Blog Cribl's solutions with Exabeam If you want to automatically get every episode of the Stream Life podcast, you can subscribe on your favorite podcast app.
In this episode of The Stream Life Podcast, Nick Tankersley joins the show to talk in-depth about the upgraded authorization support released in Cribl's 4.2 release. Cribl’s new authorization support enhances security by giving you control over who has permissions and privileges to access Cribl products, capabilities, and resources. This ensures users only see and access what they’re permitted to based on their assigned role. This level of authorization helps safeguard organizations against potential security threats. Resources Turning Up the Heat: Cribl’s Summer Product Launch Different Access for Different Roles: Cribl’s New Authorization Support for Enhanced Security Members and Permissions - Cribl Docs If you want to automatically get every episode of the Stream Life podcast, you can subscribe on your favorite podcast app.
In this episode of The Stream Life Podcast, Perry Correll and Nick Tankersley join the show to talk about all the latest enhancements coming to Cribl Stream, Cribl Edge, and Cribl Search!   Resources Turning Up the Heat: Cribl’s Summer Product Launch If you want to automatically get every episode of the Stream Life podcast, you can subscribe on your favorite podcast app.
In this live stream, Cjapi's James Curtis joins Ed Bailey to discuss the challenges of building a distributed global security team. Talent is hard to find, and companies are hiring from all over the world to build the best teams possible, but this trend has a price. Traditional management processes don’t always transfer over to remote management — everything from building a culture to the basics around assigning, tracking, and measuring work needs adjustment.
In this episode of The Stream Life Podcast, Nick Heudecker comes on the show to look at the major trends defining the observability market in 2023 Resources Learn more about CriblCon Register for CriblCon When Stream Meets Lake: Cribl’s Integration With Amazon Security Lake Helps Customers Address Data Interoperability If you want to automatically get every episode of the Stream Life podcast, you can subscribe on your favorite podcast app.
The Evils of Data Debt

The Evils of Data Debt

2023-06-0828:59

Join Cribl's Ed Bailey and Jackie McGuire as they discuss the harmful effects of data debt on observability and security teams. Data debt is a pervasive problem that increases costs and produces poor results across observability and security. Simply put, garbage in equals garbage out. Ed and Jackie will delve into what data debt is and how to solve it in the long term. They will explore the complex nature of observability and security data, which is highly volatile and requires a different approach from typical analytics use cases. To get the best results, data standards must be established with high-level buy-in from leadership. Additionally, teams must have access to a high-quality observability pipeline that allows them to manipulate data in real-time. It's also important to build a strong relationship with your GRC team, so you can track issues with standards and gain the right visibility in the enterprise. With the right strategies, data debt can be overcome, and Ed and Jackie will help you get started on the road to success.  
In this episode of The Stream Life Podcast, I chat with Mike Dupuis about CriblCon! At Cribl, we understand there is power in getting together IN PERSON to share ideas, best practices, and swap battle stories with friends new and old. That’s what CriblCon, on July 17th, at the Mirage in Las Vegas, is all about. We’re bringing together a group of remarkable people–that’s YOU!–to solve problems, talk architecture, figure out how to route, optimize, and enrich data to get more value from your SIEM, AI Ops, and analytics tools and do more with less. Resources Learn more about CriblCon Register for CriblCon If you want to automatically get every episode of the Stream Life podcast, you can subscribe on your favorite podcast app.
In this live stream discussion, angel investor Ross Haleliuk joins Cribl's Ed Bailey to make a big announcement about his new fund to shape the future of the cybersecurity industry. Ross is a big believer in focusing on the security practitioner to provide practical solutions to common issues by making early investments in companies that will promote these values. Ed and Ross also discuss trends in the industry and common struggles that both Cribl and his new fund seek to address by adding value and giving security practitioners choice and control over how they run their security program. Read more about the discussion on Cribl's blog post.
loading
Comments 
Download from Google Play
Download from App Store