Augur - AI-native Engineering Operations

Platform Engineering in Financial Institutions: The Practitioner Panel ???? Key Innovations:?? ??- Integration of platform engineering in financial institutions for rapid shipping of services.?? ??- Adoption of cloud-native technologies for improved scalability and reliability.?? ??- Innovative use of CI/CD practices in a regulated environment.?? ??? ???? Notable Features:?? ??- Introduction of standardization and automation in software delivery.?? ??- Utilization of building blocks and frameworks to enhance developer experience.?? ??- Ongoing measurement of developer happiness and engagement as a performance metric.?? ??? ????? Perfect for:?? ??- Platform engineers in financial services.?? ??- Software developers aiming to work in regulated environments.?? ??- Leaders in technology looking to modernize infrastructure in financial institutions.?? ??? ???? Impact:?? ??- Enhanced operational efficiency and reduced delivery times through technology integration.?? ??- Better balance between regulatory compliance and rapid innovation.?? ??- Improved developer satisfaction, potentially reducing turnover and enhancing product quality.?? ??? ???? Preview of the Talk:?? ??Join seasoned practitioners Paula Kennedy, Chris Plank ?, Suhail Patel Jinhong Brejnholt, Rachael Wonnacott from leading banks as they share their insights on platform engineering in financial institutions. Delve into the successes and challenges faced by these organizations in implementing cloud-native strategies while meeting stringent regulatory requirements. The panel discusses practical approaches to enhancing developer experience and driving innovation within the constraints of compliance, offering valuable lessons and strategies for fellow professionals in the field.? Watch the full session here: https://lnkd.in/gdZ3ajPE

Rogue No More: Securing Kubernetes with Node-Specific Restrictions ???? Key Innovations:?? ??- Introduction of node-specific restrictions for service account tokens.?? ??- KEP 4193 to enhance service account tokens for node-level access control.?? ??- Extension and generalization of the node restriction admission plugin.?? ?? ???? Notable Features:?? ??- Validation admission policies for enhanced security measures.?? ??- Support for complex authorization checks using bound tokens.?? ??- Robust auditing capabilities with JWT credential identifiers.?? ?? ????? Perfect for:?? ??- Kubernetes Administrators?? ??- Security Engineers?? ??- Cloud Infrastructure Teams?? ?? ???? Impact:?? ??- Enhanced security and risk mitigation for Kubernetes clusters.?? ??- Reduction of potential escalation attacks across nodes.?? ??- Improved management of service account permissions.?? ?? ???? Preview of the Talk:?? ??Anish Ramasekar and James Munnelly share insights on securing Kubernetes with innovative node-specific restrictions. They discuss significant vulnerabilities associated with daemonsets, and introduce KEP 4193 for improved service account token management. The session emphasizes practical implementation strategies, validating admission policies, and highlights the importance of node isolation to prevent privilege escalation attacks.?? ?? Watch the full talk here: https://lnkd.in/gE5adj8j

Share the Ride: Robust Multi-Tenancy in Kubernetes at Uber ???? Key Innovations: ??- Robust multi-tenancy architecture leveraging a single Kubernetes cluster to provide data plane, access, and control plane isolation. ??- Utilizes node pools mapped to namespaces ensuring dedicated resources for each tenant. ???? Notable Features: ??- Custom controllers for node lifecycle management and resource quota monitoring. ??- API rate limiting via flow schemas for tenant-specific resource management. ??- Native Kubernetes support for RBAC and network policies to ensure tenant isolation at multiple layers. ????? Perfect for: ??- Kubernetes engineers seeking multi-tenancy solutions. ??- DevOps teams managing resource allocation across various teams. ??- Organizations in industries with diverse workload requirements needing secure isolation. ???? Impact: ??- Reduced operational overhead by 30% through fewer clusters and simplification of configurations. ??- Enhanced scalability and performance, currently managing over 100 tenants with plans for continued growth. ??- Improved user experience with streamlined workload submission and automatic resource allocation. ???? Preview of the Talk: ??In this session, Sashank Reddy Appireddy and Apoorva Jindal from Uber discuss their innovative model of multi-tenancy in Kubernetes, which allows multiple tenants to coexist securely on a single cluster. They delve into unique challenges Uber faced and solutions implemented, highlighting their architecture’s efficiency and scalability. Key takeaways include incorporating node pools for isolation, handling operational complexity, and ensuring robust performance amidst varied workloads. ??Watch the full session here: https://lnkd.in/g7VWiuMh

Tutorial: Stop Kubernetes' Revolving Door: A Hands-on Tutorial to Secure a Kubernetes Cluster ???? Key Innovations: ??- Detailed walkthrough of the Kubernetes Security Checklist to enhance cluster security. ??- Hands-on exercises that provide practical experience with Kubernetes security practices. ??? ???? Notable Features: ??- Topics covered include authentication, authorization, network policies, pod security, and more. ??- Use of local development tools like Kind and integrations with security tools like Cilium. ??? ????? Perfect for: ??- DevOps Engineers ??- Kubernetes Administrators ??- Security Professionals ??- Anyone looking to enhance their Kubernetes security knowledge. ??? ???? Impact: ??- Participants will leave equipped with knowledge to better secure their Kubernetes clusters. ??- Provides a solid foundation towards achieving a secure Kubernetes environment, potentially mitigating risks from vulnerabilities. ??? ???? Preview of the Talk: ??In this comprehensive tutorial led by Savitha Raghunathan, Rey Lejano, and Mahé Tardy, attendees explore essential security measures necessary for securing Kubernetes clusters. Key topics range from managing authentication to implementing network policies that restrict pod communications. The session emphasizes practical, hands-on exercises utilizing Kind and Cilium, ensuring participants not only learn theoretical aspects but also apply best practices in real-time. This session is ideal for both newcomers and seasoned professionals looking to brush up on Kubernetes security.? ??? ??Watch the full session here: https://lnkd.in/gwrQumT6

Load-Aware GPU Fractioning for LLM Inference on Kubernetes ???? Key Innovations: ??- Analytical relationship between request load and GPU requirements for LLM inference. ??- Introduction of an open-source controller for automatic GPU fraction allocation using MIG (Multi-Instance GPU) slices. ??- Enhanced GPU sharing techniques to improve overall resource utilization on Kubernetes. ???? Notable Features: ??- Predictive performance estimation for varying loads. ??- Support for multiple concurrent models using fractional GPU allocations. ??- Integration with Kubernetes via a webhook to manage pod specifications dynamically. ????? Perfect for: ??- Data scientists working with large language models (LLMs). ??- DevOps teams managing GPU resources in Kubernetes environments. ??- AI researchers focused on GPU optimization for model serving. ???? Impact: ??- Increased GPU utilization leading to lower costs and enhanced sustainability.? ??- Improved performance metrics on throughput and latency for deployed models. ??- Simplified deployment process for developers, eliminating the need for manual GPU specifications. ???? Preview of the Talk: ??In this enlightening session, Olivier Tardieu and Yue Zhu from IBM explore the challenges of efficiently utilizing GPU resources for Large Language Models in Kubernetes environments. They introduce a framework for understanding GPU compute and memory requirements and present a live demo of an innovative controller that automates GPU fractioning. Attendees will glean insights into the interplay between workload characteristics and GPU performance, and learn about the potential for increased operational efficiency. Don't miss out on this game-changing conversation about enhancing LLM inference systems! ??Watch the full session here: https://lnkd.in/gr2kHNzB

Achieving and Maintaining a Healthy CI with Zero Test Flakes ???? Key Innovations:?? ??- Implementation of solid CI policies to eliminate flaky tests.?? ??- Adoption of effective collaboration models among diverse special interest groups (SIGs) within Kubernetes.?? ??- Development of infrastructure tools like Kind for reliable CI operations.?? ?? ???? Notable Features:?? ??- **Test Grid**: Visual history and analysis of job failures and flaky tests over time.?? ??- **Triage Tool**: Clustering failure by message for easier debugging and identification of test issues.?? ??- **Configurable Alerts**: Notifications for critical flaky tests or job failures, ensuring better tracking and response.?? ?? ????? Perfect for:?? ??- Software Engineers looking to improve CI processes.?? ??- DevOps teams aiming for lower test flakiness.?? ??- Project maintainers who manage CI/CD pipelines in their organizations.?? ?? ???? Impact:?? ??- Enhanced reliability of CI pipelines leading to faster development cycles.?? ??- Reduced frustration and increased developer confidence in code quality.?? ??- Establishment of a culture focused on building high-quality software.?? ?? ???? Preview of the Talk:?? ??In this enlightening session, Google engineers Antonio Ojea, Michelle Shepardson, and Benjamin Elder delve into strategies for achieving zero test flakes. They explore shared responsibilities across Kubernetes SIGs, highlight essential tools like Test Grid and Triage that enhance debuggability, and present infrastructures that lead to reliable CI operations. The session encourages a collaborative culture, harnesses best practices, and ultimately drives software quality upwards.? ??Watch the full session video here: https://lnkd.in/gXxVg7Rm

Faster Containerized LLM Serving via Knowledge Sharing ???? Key Innovations: ??- **Knowledge-Sharing System**: Allows LLMs to share digested knowledge (KV caches) from single document processing, avoiding redundant reads and enhancing efficiency. ??- **Cost-Efficient Storage**: Efficient storage mechanisms for KV caches using lower-cost hardware rather than traditional GPU/CPU memory. ???? Notable Features: ??- **Dynamic KV Management**: The system dynamically retrieves and manages KV caches to optimize for speed and reduce latency. ??- **Seamless Integration with Kubernetes**: Demonstrates deployment and management on Kubernetes ecosystems for efficient scaling. ????? Perfect for: ??- AI Researchers ??- Data Engineers ??- DevOps Teams focusing on scalable LLM applications ??- Product Managers looking for advanced AI solutions ???? Impact: ??- Improved serving time of LLMs with a reduction in delay by up to 10x during inference. ??- Significant cost savings (5 to 10 times) for cloud-based services compared to traditional on-demand solutions. ??? ???? Preview of the Talk: ??In this enlightening session, Junchen Jiang of University of Chicago and Zhou Sun of Mooncake Labs discuss a transformative approach to LLM error margin through innovative knowledge-sharing that vastly improves processing time and cost-efficiency. They delve deeply into their robust system architecture, demonstrating how efficient KV cache storage and management enables LLMs to deliver responses faster while substantially lowering operational costs. Audiences gain insights into real-world applications, impressive benchmarks, and upcoming features poised to revolutionize LLM serving in a cloud-native environment. Watch the full session here: https://lnkd.in/g_J3GUqe

What Containerd 2.0 Means for You - Samuel Karp, Google ???? Key Innovations: ??- Major version upgrade to 2.0, introducing new features and extension points for enhanced flexibility. ??- Stabilized previously experimental features from containerd 1.7. ??- Improved performance in image operations and CRI. ??? ???? Notable Features: ??- Node Source Interface (NRI): Middleware for custom container configurations beyond Kubernetes defaults. ??- Image Verifier Plugins: Set policies for image pulls, allowing for signature enforcement and denial of vulnerable images. ??- Transfer Service: A new stable mechanism for efficient image transfers. ??? ????? Perfect for: ??- Software Engineers focusing on container technology. ??- DevOps teams managing Kubernetes environments. ??- IT professionals utilizing containerd in production environments. ??? ???? Impact: ??- Increased efficiency and flexibility in workload management. ??- Enhanced image security with new verification policies. ??- Simplified upgrading process with an emphasis on backward compatibility. ??? ???? Preview of the Talk: ?Samuel Karp, a software engineer and maintainer at Google, dives deep into Containerd 2.0's significant advancements and improvements. He elaborates on the new features, removal of deprecated functionalities, and provides actionable strategies for transitioning to Containerd 2.0 in production environments. The session culminates with insights on future developments and the roadmap ahead. Watch the full session here: https://lnkd.in/gnnjjHeA

Micro-Segmentation and Multi-Tenancy: The Brown M&Ms of Platform Engineering ???? Key Innovations: ??- **Micro-segmentation:** Enhances network security by dividing larger networks into smaller segments. ??- **Multi-tenancy:** Allows multiple workloads to share infrastructure securely. ??- **Automation:** Utilizes GitOps and Policy as Code for streamlined operational processes. ???? Notable Features: ??- **Namespaces-as-a-Service:** Facilitates isolation for different teams using shared clusters. ??- **Policy Automation:** Tools like Cilium and Kerno manage network policy to ensure security and compliance. ??- **Standardization:** Reduces cognitive load for developers by implementing common practices across teams. ????? Perfect for: ??- Platform engineers seeking to implement robust security in Kubernetes environments. ??- Enterprises looking to balance multi-tenancy with performance and cost efficiency. ??- DevOps teams needing a structured approach to application development and deployment. ???? Impact: ??- Improved application security through micro-segmentation. ??- Enhanced operational efficiency and reduced cognitive load on development teams. ??- Greater predictability in cost and resource management through standardization. ???? Preview of the Talk: ??In this insightful session, Jim Bugwadia and Rachael Wonnacott explore the pressing need for micro-segmentation and multi-tenancy in platform engineering. Using the metaphor of Van Halen's infamous 'no brown M&Ms' clause, they underline the importance of meticulous detail in managing shared infrastructure. The speakers present various approaches to secure Kubernetes deployments, showcasing real-world examples, and demonstrating how automation reduces cognitive load for app developers. Key insights into tools like Cilium and Kerno reveal how organizations can improve both security and efficiency within their cloud environments. ??Watch the full session here: https://lnkd.in/gp3i3UC2 #kubeconNA

Enhancing Security Visibility in Kubernetes ???? Key Innovations: ??- Enhanced CVE response process: Streamlined reporting and fixing vulnerabilities through collaboration among multiple SIGs. ??- Official CVE Feed: Automated feed established for keeping track of security issues in Kubernetes releases. ??- HackerOne bug bounty program: Encourages reporting vulnerabilities with financial incentives to security researchers. ???? Notable Features: ??- Collaboration between SIGs (Security Response Committee, SIG Release, SIG Security) to manage vulnerabilities effectively. ??- Introduction of vulnerability classification through severity ratings and tools for visualization. ??- Mechanism for distributors to receive advance notice of vulnerabilities for timely patching. ????? Perfect for: ??- Kubernetes security professionals ??- Software engineers focused on security ??- DevOps teams looking to enhance deployment security ??- Security researchers interested in vulnerability reporting ???? Impact: ??- Improved visibility and response to security vulnerabilities leads to greater trust in Kubernetes deployments. ??- Established a systematic approach for vulnerability disclosure that mitigates risks from malicious actors. ??- Strengthened collaboration fosters a more secure ecosystem and empowers users to manage security more effectively. ???? Preview of the Talk: ??In this insightful session, Rita Zhang and Jeremy Rickard from Microsoft discuss the essential roles of different SIGs in enhancing security visibility within Kubernetes. They explore the intricacies of coordinating responses to security vulnerabilities, introduce the official CVE feed, and outline future enhancements to their approaches. The talk emphasizes the importance of private vulnerability reporting and engaging with the Kubernetes community to maintain secure practices. For more in-depth insights, watch the full session on YouTube: https://lnkd.in/gb--FMJ7.