How Do You Use eBPF for Security and Observability in Enterprise Kubernetes?

eBPF (extended Berkeley Packet Filter) has changed the economics of Kubernetes observability and security more than any infrastructure technology in the past five years. The traditional approach — injecting sidecar containers into every pod, running a DaemonSet agent per node, or deploying a full service mesh like Istio — adds 50–150 MB of memory per pod, introduces latency on every network packet, and requires instrumentation code in every application. eBPF eliminates these costs by running sandboxed programs directly in the Linux kernel, capturing the same observability data and enforcing the same security policies at the kernel level, with under 1% CPU overhead per node and zero application code changes. If your team handles cloud infrastructure and DevOps engineering for production Kubernetes workloads, understanding eBPF is now a baseline competency, not an advanced elective.

The CNCF Observability Technical Advisory Group's 2026 survey shows that 67% of teams running Kubernetes at scale have adopted at least one eBPF-based observability tool. The shift is happening because the production numbers are unambiguous: for a 500-pod cluster, sidecar-based observability consumes over 75 GB of RAM while an equivalent eBPF stack consumes roughly 12 GB. At high throughput, traditional sidecar-heavy monitoring uses up to 249% more CPU to process the same volume of telemetry. These are not marginal wins — they are the difference between an infrastructure budget that scales with the business and one that compounds into an existential cost problem.

This guide covers what eBPF is at the kernel level, why traditional observability approaches hit a scale wall, and how the three core production tools — Cilium/Hubble (networking and network observability), Tetragon (runtime security and enforcement), and Beyla (zero-instrumentation application tracing) — work together as a unified eBPF stack. It is the infrastructure companion to our OpenTelemetry observability guide, which covers the telemetry pipeline that receives and routes the data this stack produces.

What Is eBPF and How Does It Work at the Kernel Level

eBPF is a Linux kernel feature that lets you load small, sandboxed programs into the kernel without modifying kernel source code or loading kernel modules. The name is historical — it descends from BSD Packet Filter — but modern eBPF has little in common with its packet-filtering origins. Today it is a general-purpose in-kernel programmability layer: you write a program in a restricted subset of C, compile it to eBPF bytecode, and the kernel's verifier checks that the program is safe before it runs. The kernel guarantees the program cannot crash the kernel, cannot loop infinitely, and cannot access memory outside permitted bounds.

• →Event-driven execution: eBPF programs execute in response to specific events — a syscall entry or exit, a network packet arrival, a kernel function call, a file open, a process fork. The program runs to completion in kernel context and returns immediately; it does not block normal kernel execution
• →Maps for state and data export: eBPF programs pass data to user space through eBPF maps — shared memory structures that both kernel programs and user-space processes can read and write. This is how metrics are aggregated, events are exported, and enforcement decisions are communicated across the kernel-user boundary
• →Verifier safety guarantee: before any eBPF program loads, the kernel verifier performs static analysis — checking all code paths, bounding loops, validating memory accesses. A program that fails verification never runs. This safety model is what makes eBPF safe to deploy in production kernels without a module-signing framework or kernel recompilation
• →Minimum kernel requirement: for the full production eBPF stack (Cilium, Tetragon, Beyla), the minimum supported kernel is 5.10 LTS; the recommended version for 2026 production environments is 6.1+, which ships BTF (BPF Type Format) natively for portable kernel introspection. Check every node pool's kernel version before planning an eBPF adoption roadmap

Why Traditional Kubernetes Observability Approaches Hit a Scale Wall

The sidecar pattern — injecting Envoy or a language-specific tracing agent into every pod — made sense when Kubernetes clusters were small and polyglot instrumentation was the only way to get distributed traces. At scale, the economics collapse. A 1,000-pod cluster with sidecar proxies carries 50–150 GB of RAM dedicated to observability infrastructure alone, before any application workloads run. Every network hop passes through two user-space processes (the sending pod's sidecar and the receiving pod's sidecar), adding latency per hop and creating connection state that multiplies with cluster size.

• →Agent proliferation: most teams end up running three to five separate DaemonSet agents for logging, metrics, APM traces, and security monitoring. Each agent has its own memory footprint, its own CPU budget, its own configuration language, and its own failure mode — and they all read from the same kernel data sources independently, paying the collection cost multiple times for the same underlying events
• →Application code coupling: language-specific instrumentation (OpenTelemetry SDKs, APM agents injected into JVM startup, Python bytecode manipulators) ties observability to the application build process. When a service changes language, the instrumentation changes. When instrumentation breaks, debugging requires understanding both the application and the instrumentation framework simultaneously
• →Coverage gaps: sidecar and agent instrumentation covers only what was explicitly instrumented. A database driver that does not export spans, a kernel-level file operation, or a network packet below the application layer does not appear in the trace. eBPF captures everything at the kernel boundary — the application cannot opt out and cannot miss coverage

Cilium and Hubble: eBPF-Based Networking and Network Observability

Cilium replaces iptables-based kube-proxy and traditional CNI plugins with eBPF programs loaded directly into the Linux kernel. Where iptables implements network policy as a series of sequential rules that the kernel evaluates for every packet, Cilium compiles network policies to eBPF bytecode that executes at the XDP (eXpress Data Path) layer — before a packet even reaches the kernel's networking stack. The result is 2–10x faster network policy enforcement and richer observability primitives than iptables provides.

• →Hubble for network observability: Hubble is Cilium's observability layer — a DaemonSet on each node that exposes an API providing real-time visibility into all network flows between pods, without any application instrumentation. Every HTTP request, DNS query, and TCP connection between services appears in Hubble's flow log with full pod identity context (namespace, label selectors), enabling network forensics and security auditing at the flow level
• →Identity-based network policy: Cilium enforces network policy based on pod identity (Kubernetes labels and service accounts), not IP addresses. This matters in Kubernetes because pod IPs change constantly on rescheduling — IP-based policy requires continuous reconfiguration. Identity-based policy holds across restarts and scales to thousands of pods without policy table bloat
• →Cilium as a service mesh replacement: Cilium 1.14+ implements Kubernetes services, load balancing, and mutual TLS at the eBPF layer without requiring sidecar proxies. For teams evaluating Istio for mTLS and service-to-service policy, Cilium covers most use cases at significantly lower overhead and with no sidecar injection requirement
• →Prometheus integration: Cilium exports metrics via a Prometheus endpoint on each DaemonSet pod — flow rates, drop rates, policy verdicts, DNS resolution latency. These plug directly into existing Prometheus and Grafana stacks with minimal configuration and no additional scrape infrastructure

Tetragon: Runtime Security Enforcement at the Kernel Layer

Tetragon is a CNCF project from the Cilium team that extends eBPF into security observability and runtime enforcement. Where Falco reads kernel audit events and sends alerts to user space, Tetragon attaches eBPF programs directly to kernel functions and can enforce policy decisions — block a process from opening a network connection, prevent a file write to a sensitive path, kill a process executing from a temporary directory — in the same kernel context where the event occurs, before it completes. The enforcement is not advisory; it is a synchronous kernel-level block that the application cannot bypass from user space.

• →TracingPolicy as code: Tetragon security policies are Kubernetes custom resources (TracingPolicy CRDs) — defined in YAML and applied with kubectl. Policies are stored in etcd and deployed to all nodes via the Tetragon DaemonSet, which means security policies follow the same GitOps workflow as application manifests — version-controlled, code-reviewed, auditable, and environment-specific
• →Process, network, and file observability: Tetragon tracks the full process execution tree on each node — which processes spawned which children, what files they opened, what network connections they made. This is the kernel-level audit trail that SIEM tools historically needed endpoint agents to collect. Tetragon provides it natively at no additional agent cost
• →Production readiness in 2026: Tetragon 1.4, released in February 2026, resolved the remaining rough edges in policy authoring and added full support for CGroups v2 environments. It is now the production default for runtime security in Cilium-based Kubernetes clusters across the CNCF ecosystem
• →SIEM and alerting integration: Tetragon exports events in JSON via a gRPC API and a file-based sink. Export to Splunk, Elastic, or Chronicle via the OpenTelemetry Collector. Alert on specific TracingPolicy violations using Grafana AlertManager or PagerDuty rules against the Prometheus metrics endpoint

The combination of Cilium/Hubble for network-layer visibility and Tetragon for process, file, and syscall visibility gives the same threat detection surface that traditionally required both a network detection tool and an endpoint detection and response (EDR) agent — but at a fraction of the overhead and with native Kubernetes pod identity context, which EDR agents cannot understand. For teams building on security and scalability engineering, this changes the economics of runtime security significantly.

Beyla: Zero-Instrumentation Application Tracing via eBPF

Beyla (donated to the OpenTelemetry project in 2025) captures distributed traces from applications running in Kubernetes without any application code changes, library additions, or agent injection. It attaches eBPF uprobes to user-space functions in the application binary — HTTP/2 framing functions in Go's net/http package, SSL_read and SSL_write in OpenSSL — and reconstructs full distributed traces at the kernel level, including trace context propagation across service boundaries. The result is zero-instrumentation tracing with OpenTelemetry-compatible output, covering Go, Python, Java, .NET, Node.js, and Ruby via their standard HTTP and gRPC libraries.

• →Coverage scope: Beyla covers standard service-to-service HTTP and gRPC communication, including TLS-encrypted traffic via SSL/TLS interception probes. Edge cases — custom HTTP clients, non-standard database drivers, binary protocols — may not be captured. Most teams run Beyla alongside existing SDK instrumentation during a transition period to validate coverage before removing SDK-level agents
• →Deployment model: Beyla runs as a DaemonSet, one per node, and discovers pods via namespace annotations or label selectors. The required elevated capabilities (BPF, PERFMON, SYS_PTRACE) mean it needs a privileged PodSecurity exception or namespace-level policy override — the main deployment constraint that differentiates it from standard DaemonSet workloads
• →OTel Collector integration: Beyla exports OTLP traces and metrics directly to an OpenTelemetry Collector, which routes to Tempo, Jaeger, Grafana Cloud, or any OTLP-compatible backend. This makes Beyla's traces directly comparable with SDK-instrumented traces in the same Grafana/Tempo dashboard, which simplifies the transition period

Building the Production eBPF Stack: Architecture and Rollout Sequence

The recommended production stack in 2026 consists of Cilium + Hubble for networking and network observability, Tetragon for runtime security, and Beyla for application tracing — all exporting telemetry to an OpenTelemetry Collector that routes to your existing Grafana, Prometheus, and SIEM backends. This covers the three dimensions of production observability — network, security, application — with a unified kernel-level data collection model. The Kubernetes cost optimization enabled by this consolidation is significant: replacing five DaemonSet agents with two (Tetragon, Beyla) and eliminating sidecar proxies from all pods reduces per-pod memory consumption by 60–80%.

• →Weeks 1–2: audit kernel versions across all node pools (minimum 5.10, target 6.1+). Deploy Cilium as a CNI replacement on a staging cluster and run Hubble in observe-only mode. Validate that all existing service-to-service traffic appears in Hubble flow logs before touching production. Document the CNI migration runbook and rollback procedure
• →Weeks 3–4: deploy Tetragon DaemonSet on staging. Start with audit-only TracingPolicies (no enforcement) to catalog process execution patterns and build baselines. Feed Tetragon JSON events into your SIEM to validate signal-to-noise ratios and write alert rules before enabling any enforcement policies in production
• →Weeks 5–6: deploy Beyla in parallel with existing APM agents. Compare trace coverage between Beyla and existing SDK instrumentation. Resolve coverage gaps. Build Grafana dashboards from Beyla OTLP traces, then verify they match SDK-sourced traces for the same request flows
• →Weeks 7–8: enable Tetragon enforcement policies in production — starting with low-risk rules (block network connections from batch processes that should not have external access, alert on processes writing to /etc). Validate false positive rates. Begin DaemonSet agent consolidation as coverage is verified. Roll Cilium CNI to production node pools in a rolling node drain sequence

eBPF vs Sidecar Proxies: The Honest Trade-off Analysis

eBPF is not a universal replacement for service mesh sidecar proxies in every scenario. The trade-offs are concrete and worth understanding before committing to a migration.

• →Where eBPF wins decisively: memory footprint (60–80% reduction per pod), CPU overhead at high throughput (2–3x less CPU for equivalent telemetry volume), coverage of non-instrumented code paths that sidecars miss, and the elimination of sidecar injection lifecycle complexity — no more broken pod startup due to a misconfigured mutating webhook
• →Where sidecar proxies still lead: Istio and Linkerd provide richer traffic shaping capabilities — circuit breaking, weighted canary routing, fault injection for chaos testing, and retry budgets at the per-route level — that Cilium's eBPF-based service mesh does not yet fully replicate. If advanced traffic management is a hard requirement for specific services, sidecar proxies may remain justified in those namespaces
• →The hybrid architecture: the practical 2026 enterprise deployment is Cilium for networking, Tetragon for security, and Beyla for tracing across the cluster, with Istio sidecars retained only in the subset of namespaces that genuinely need its advanced traffic management features. The sidecars are not everywhere — they are where traffic policy complexity justifies the overhead
• →Operational maturity requirement: replacing iptables with Cilium on a production cluster requires kernel version validation, CNI migration planning, and team familiarity with eBPF debugging tools (bpftool, bpftrace, Hubble CLI). The migration operational risk is real, even if steady-state operations are simpler than managing sidecar injection. Start with a new node pool in staging and validate before touching production nodes

Frequently Asked Questions

Is eBPF safe to run in production Kubernetes clusters?

Yes — with kernel version ≥ 5.10 and validated CNCF tooling (Cilium, Tetragon, and Beyla are all CNCF projects with production deployments at hyperscaler scale). The eBPF verifier enforces safety guarantees at load time — no infinite loops, no out-of-bounds memory access, no kernel crash risk. Meta, Google, Netflix, Cloudflare, and LinkedIn all run eBPF at massive production scale. The operational risk is in the migration itself (CNI replacement, kernel version requirements), not in the eBPF execution model.

Do I need to modify my application code to use eBPF observability?

No — that is the central value proposition. Beyla captures distributed traces, Hubble captures network flows, and Tetragon captures process and file events, all at the kernel layer without any changes to application code, build pipelines, or deployment manifests. The trade-off is that eBPF instrumentation covers standard library code paths but may miss application-specific business logic that an explicit SDK would capture. Most teams run eBPF alongside existing SDK instrumentation during a transition period and remove SDK agents after validating coverage equivalence.

What is the difference between Falco and Tetragon for Kubernetes runtime security?

Both use eBPF to capture kernel-level security events, but they differ fundamentally in enforcement capability. Falco detects and alerts — it observes syscalls and generates events for your SIEM or alerting system, but it does not block actions. Tetragon can detect and enforce — it can prevent a process from opening a network connection, kill a process executing from a sensitive path, or block a file write, synchronously at the kernel level. For compliance use cases requiring real-time blocking (not just alerting), Tetragon is the more capable tool. Falco has a larger community rule library if broad detection coverage without enforcement is the goal.

How does Cilium compare to kube-proxy for Kubernetes service routing?

kube-proxy implements Kubernetes service routing via iptables rules. At large pod counts, iptables chains grow linearly — a cluster with 10,000 services can have 40,000+ iptables rules that every packet must traverse sequentially. Cilium replaces this with eBPF maps and programs that perform direct hash-table lookups in O(1) time regardless of cluster size. Benchmark results consistently show 2–10x faster service routing with sub-millisecond latency maintained at scale. Cilium also implements NetworkPolicy with richer primitives — L7 HTTP policy, FQDN-based egress controls, DNS visibility — that iptables-based plugins cannot support.

What kernel version is required for eBPF in production?

The minimum supported kernel for the full Cilium + Tetragon + Beyla stack is Linux 5.10 LTS. The recommended version for 2026 production environments is 6.1+, which ships BTF (BPF Type Format) natively — a requirement for portable kernel introspection that Tetragon and Beyla use without architecture-specific kernel headers. EKS, GKE, and AKS all support kernel 6.1 on their default managed node images as of mid-2026. If you run self-managed nodes on Ubuntu 22.04 LTS (kernel 5.15 by default), you meet the minimum requirement. Audit every node pool before beginning a Cilium CNI migration.

How Belsoft Helps With eBPF Infrastructure and Cloud-Native Security

Belsoft designs and implements production eBPF stacks for enterprise Kubernetes environments — from kernel version audits and CNI migration planning through Tetragon TracingPolicy authoring, Beyla trace coverage validation, and Grafana dashboard and alert configuration. We have run Cilium CNI migrations on production clusters with hundreds of nodes and authored TracingPolicy libraries covering common compliance use cases — PCI DSS process allowlisting, HIPAA file access auditing, SOC 2 network egress monitoring — ready for deployment alongside your application workloads. See our cloud infrastructure and DevOps engineering service for the infrastructure layer and our security and scalability engineering service for compliance-aligned security policy design.

If you are evaluating eBPF adoption or planning a Cilium CNI migration and want a concrete technical roadmap — kernel requirements, migration sequencing, rollback procedures, and cost-benefit analysis for your specific cluster size — schedule a technical review with our infrastructure team.

“eBPF does not add observability to your kernel. It reveals the observability that was always there — you were just paying three times the cost to collect it through user space.”