Introduction
In this series of blogs, we have explored how eBPF can be used to implement various features of service mesh such as traffic management, security, and observability. In this blog, we will compare eBPF-based service mesh with the traditional sidecar proxy model and highlight the benefits and challenges of each approach.
Sidecar proxy model of service mesh
The sidecar proxy model is the most common way of implementing service mesh today. It involves running a layer 4 proxy such as Envoy or NGINX in each pod or workload that intercepts and forwards all the traffic between services. The proxy is responsible for applying the service mesh logic such as routing, load balancing, encryption, authentication, authorization, etc. The proxy also collects and exports metrics and traces about the traffic.
Pros and Cons of the Sidecar Proxy Model
The sidecar proxy model has some advantages such as:
- Compatibility: The proxy can work with any application protocol or language without requiring any changes to the application code or network configuration.
- Flexibility: The proxy can be configured and updated dynamically by a central control plane such as Istio or Linkerd that manages the service mesh policies and rules.
- Extensibility: The proxy can be extended with custom filters or plugins to support new features or integrations.
However, the sidecar proxy model also has some drawbacks such as:
- Overhead: The proxy adds additional CPU, memory, and network overhead to each pod or workload. It also introduces latency and jitter due to context switches and data copies between user space and kernel space.
- Complexity: The proxy adds another layer of abstraction and indirection to the application communication. It also increases the attack surface and the potential for bugs or vulnerabilities.
- Inconsistency: The proxy may not always reflect the true state of the application or the network. It may also have different behaviors or versions across different pods or workloads.
eBPF-based service mesh model
The eBPF-based service mesh model is an alternative way of implementing service mesh that leverages eBPF, a technology that allows user code to run in the kernel with high performance and safety. eBPF can enable a transparent service mesh that does not require any proxies or libraries in each pod or workload. It can also provide L7-aware processing, identity verification, and observability at the kernel level.
The eBPF-based service mesh model works by attaching eBPF programs to various network hooks or events that intercept and process the traffic between services. For example, an eBPF program can be attached to a socket connect() call to redirect the traffic to a local port where another eBPF program is listening. This way, the application thinks that it is connecting to a remote service, but in fact it is connecting to a local eBPF program that handles the service mesh logic.
The eBPF program that listens on the local port can then perform various tasks such as:
- L7 traffic management: The eBPF program can parse and modify the application layer protocols such as HTTP or gRPC to implement features such as load balancing, rate limiting, routing, retrying, etc.
- Identity-based security: The eBPF program can verify the identity of both the sender and receiver of the traffic using certificates or tokens. It can also encrypt and decrypt the traffic using TLS or other mechanisms.
- Observability and tracing: The eBPF program can collect and export metrics and traces about the traffic such as latency, throughput, errors, etc. It can also inject headers or tags into the traffic to enable distributed tracing across services.
The eBPF program can also communicate with other eBPF programs running on different nodes or pods using shared data structures or maps. This way, it can exchange information such as service discovery data, configuration data, policy data, etc.
Read more about How eBPF Can Be Used to Implement Traffic Management Features of Service Mesh
Pros and Cons of Service mesh model
The eBPF-based service mesh model has some advantages such as:
- Performance: Since eBPF runs in the kernel space with direct access to hardware resources such as CPU and memory, it can achieve higher performance than user space proxies or libraries. It can also avoid context switches and data copies between user space and kernel space.
- Efficiency: Since eBPF does not require any additional processes or containers to run in each pod or workload, it can reduce resource consumption such as CPU cycles, memory usage, disk space, etc.
- Security: Since eBPF runs in the kernel space with strict verification and isolation, it can prevent malicious or buggy code from compromising the system or the application. It can also provide stronger security guarantees than user space proxies or libraries that may have vulnerabilities or bugs.
- Simplicity: Since eBPF does not require any changes to the application code or network configuration, it can simplify the deployment and management of the service mesh. It can also avoid compatibility issues or conflicts between different versions or implementations of proxies or libraries.
However, the eBPF-based service mesh model also has some challenges such as:
- Compatibility: eBPF requires a recent version of the Linux kernel that supports the latest features and capabilities. It may not work on older or customized kernels that are used by some cloud providers or operating systems.
- Flexibility: eBPF programs are written in a restricted subset of C language and compiled into bytecode that is verified by a verifier before being loaded into the kernel. This may limit the expressiveness and portability of the service mesh logic. It may also make it harder to update or debug the eBPF programs dynamically.
- Extensibility: eBPF programs are attached to predefined hooks or events that are exposed by the kernel. This may limit the scope and granularity of the service mesh logic. It may also make it harder to integrate with external systems or services.
Conclusion
eBPF and sidecar proxy are two different models for implementing service mesh with different trade-offs and challenges. eBPF can provide a transparent service mesh with high performance, efficiency, security, and simplicity, but it may also face compatibility, flexibility, and extensibility issues. Sidecar proxy can provide a compatible, flexible, and extensible service mesh, but it may also add overhead, complexity, and inconsistency issues.
There is no one-size-fits-all solution for service mesh. Depending on the use case and the environment, one model may be more suitable than the other. It is also possible to combine both models to achieve the best of both worlds. For example, one can use eBPF for L4 traffic management and security, and use sidecar proxy for L7 traffic management and observability.
Top resources on eBPF and Service Mesh
If you are interested in learning more about eBPF and service mesh, you can check out some of the following resources:
- eBPF and the Service Mesh: Don’t Dismiss the Sidecar Yet – An article by William Morgan from Linkerd that explains why eBPF is not a silver bullet for service mesh and why sidecar proxies still have advantages.
- How eBPF will solve Service Mesh – Goodbye Sidecars – An article by Thomas Graf from Isovalent that argues why eBPF is a better solution for service mesh than sidecar proxies and how Cilium implements it.
- eBPF or Not, Sidecars are the Future of the Service Mesh – An article by Christian Posta from Solo.io that compares eBPF and sidecar models for service mesh and why they are not mutually exclusive.
- Tanzu Service Mesh Acceleration using eBPF – An article by VMware that describes how Tanzu Service Mesh uses eBPF to achieve network acceleration by bypassing the TCP/IP networking stack in the Linux kernel.
Let’s Build Something Great Together
If you’re an engineer or decision-maker looking to harness the power of eBPF for complex system monitoring, security, or networking tasks, we’re here to help. Our team at Benison is deeply experienced in eBPF development. We can collaborate with you to create bespoke solutions tailored to your specific challenges.
Ready to take your eBPF projects to the next level? Contact us today to discuss your ideas and how we can bring them to life.
