Give your Edge an Adrenaline Boost: Using Kubernetes to Orchestrate FPGAs and GPU
Over the past year, we’ve been experimenting with field-programmable gate arrays (FPGAs) and graphics processing units (GPUs) to improve edge compute performance and reduce the overall cost of edge deployments.
Unless you’ve been under a rock for the past 2 years, you’ve heard all the excitement about edge computing. For the uninitiated, edge computing allows for applications that previously required special hardware to be on customer premises to run on systems located near customers. These workloads require either very low latency or very high bandwidth, which means they don’t do well in the cloud. With many of these low-latency applications, microseconds matter. At CableLabs, we’ve been defining a reference architecture and adapting Kubernetes to better meet the low-latency needs of edge computing workloads.
CableLabs engineer Omkar Dharmadhikari wrote a blog post in May 2019 called Moving Beyond Cloud Computing to Edge Computing, outlining many of the opportunities for edge computing. If you aren’t familiar with the benefits of edge computing, I’d suggest reading that post before you read further.
As part of our efforts around Project Adrenaline, we’ve shared tools to ease the management of hardware accelerators in Kubernetes. These tools are available in the SNAPS-Kubernetes GitHub repository.
- Field-programmable gate array (FPGA) accelerator integration
- Graphics processing unit (GPU) accelerator integration
FPGAs and GPUs can be used as hardware accelerators. There are three advantages that we consider when moving a workload to an accelerator:
- Time requirements
- Power requirements
- Space requirements
Time, space and power are all critical for edge deployments. You have limited space and power for each location. The time needed to complete the operation must fall within the desired limits, and certain operations can be much faster running on an accelerator than on a CPU.
Writing applications for accelerators can be more difficult because there are fewer language options than general-purpose CPUs have. Frameworks such as OpenCL attempt to bridge this gap and allow a single program to work on CPUs, GPUs and FPGAs. Unfortunately, this interoperability comes with a performance cost that makes these frameworks a poor choice for certain edge workloads. The good news is that several major accelerator hardware manufacturers are targeting the edge, releasing frameworks and pre-built libraries that will bridge this performance gap over time.
Although we don’t have any hard-and-fast rules today for what workloads should be accelerated and on which platform, we have some general guidelines. Integer (whole number) operations are typically faster on a general-purpose CPU. Floating point (decimal number) are typically faster on GPUs. Bitwise operations, manipulating ones and zeros, are typically faster on FPGAs.
Another thing to keep in mind when deciding where to deploy a workload is the cost of transitioning that workload from one compute platform to another. There’s a penalty for every memory copy, even within the same server. This means that running consecutive tasks within a pipeline on one platform can be faster than running each task on the platform that is best for that task.
Accelerator Installation Challenges
When you use accelerators such as FPGAs and GPUs, managing the low-level software (drivers) to run them can be a challenge. Additional hooks to install these drivers during the OS deployment have been added to SNAPS-Boot, including examples for installing drivers for some accelerators. We encourage you to share your experiences and help us add support for a broader set of accelerators.
These features were developed in a co-innovation partnership with Altran. We jointly developed the software and collaborated on the proof of concepts. You can discover more about our co-innovation program on our website, which includes information about how to contact CableLabs with a co-innovation opportunity.
Extending Project Adrenaline
Project Adrenaline only scratches the surface of what’s possible with accelerated edge computing. The uses for edge compute are vast and rapidly evolving. As you plan your edge strategy, be sure to include the capability to manage programmable accelerators and reduce your dependence on single-purpose ASICs. Deploying redundant and flexible platforms is a great way to reduce the time and expense of managing components at thousands or even millions of edge locations.
As part of Project Adrenaline, SNAPS-Kubernetes ties together all these components to make it easy to try in your lab. With the continuing certification of SNAPS-Kubernetes, we’re staying current with releases of Kubernetes as they stabilize. SNAPS-Boot has additional features to easily prepare your servers for Kubernetes. As always, you can find the latest information about SNAPS on the CableLabs SNAPS page.
Contact Randy to get your adrenaline fix at Mobile World Congress in Barcelona, February 24-27 2020.
Vaccinate Your Network to Prevent the Spread of DDoS Attacks
CableLabs has developed a method to mitigate Distributed Denial of Service (DDoS) attacks at the source, before they become a problem. By blocking these devices at the source, service providers can help customers identify and fix compromised devices on their network.
DDoS Is a Growing Threat
DDoS attacks and other cyberattacks cost operators billions of dollars, and the impact of these attacks continues to grow in size and scale, with some exceeding 1 Tbps. The number of Internet of Things (IoT) devices also continues to grow rapidly, many have poor security, and upstream bandwidth is ever increasing; this perfect storm has led to exponential increases in IoT attacks, by over 600 percent between 2016 and 2017 alone. With an estimated increase in the number of IoT devices from 5 billion in 2016 to more than 20 billion in 2020, we can expect the number of attacks to continue this upward trend.
As applications and services are moved to the cloud and the reliance on connected devices grows, the impact of DDoS attacks can continue to worsen.
Enabled by the Programmable Data Plane
Don’t despair! New technology brings new solutions. Instead of mitigating a DDoS attack at the target, where it’s at full strength, we can stop the attack at the source. With the use of P4, a programing language designed for managing traffic on the network, the functionality of switches and routers can be updated to provide capabilities that aren’t available in current switches. By coupling P4 programs with ASICs built to run these programs at high speed, we can do this without sacrificing network performance.
As service providers update their networks with customizable switches and edge compute capabilities, they can roll out these new features with a software update.
Comparison Against Traditional DDoS Mitigation Solutions
|Feature||Transparent Security||Typical DDoS solution|
|Mitigates ingress traffic||X||X|
|Mitigates egress traffic||X|
|Deployed at network peering points||X||X|
|Deployed at hub/head end||X|
|Deployed at customer premises||X|
|Requires specialized hardware||X|
|Mitigates with white box switches||X|
|Works with customer gateways||X|
|Identifies attacking device||X|
|Time to mitigate attack||Seconds||Minutes|
|Packet header sample rate||100%||< 0.1%|
Transparent Security can mitigate ingress and egress traffic at every point in the network, from the customer premises to the core of the network. To mitigate ingress attacks, typical DDoS mitigation solutions are deployed only at the edge of the network. This means that they don’t protect the network from internal DDoS attacks and can allow their networks to be weaponized.
Transparent Security runs on white box switches and software at the gateway. This provides a wide variety of vendor options and is compatible with open standards, such as P4. Typical solutions frequently rely on the purchase of specialized hardware called scrubbers. It isn’t feasible to deploy these at the customer premises. Finally, Transparent Security can look at the header for every egress packet to quickly identify attacks originating on the service providers network. Typical solutions sample only 1 in 5,000 packets.
Just the Beginning
Transparent Security is just the beginning, and one of many solutions that can be deployed to improve broadband services. Through the programmable data plane, network management will become vastly smarter, and new services will benefit, from Micronets to firewall and managed router as a service.
Join the Project
CableLabs is engaging members and vendors to define the interfaces between the transparent security components. This should create an interoperable solution with a broad vendor ecosystem. The SDNC-Dashboard, AE-SDNC, SDNC-Switch and Switch-AE interfaces in the diagram below have been identified for the initial iteration. Section 6 of the white paper describes these interfaces in detail.
The Transparent Security architecture and interface definitions will expand over time to support additional use cases. These interfaces leverage existing industry standards when possible.
CableLabs Announces SNAPS-Kubernetes
Today, I’m pleased to announce the availability of SNAPS-Kubernetes. The latest in CableLabs’ portfolio of open source projects to accelerate the adoption of Network Functions Virtualization (NFV), SNAPS-Kubernetes provides easy-to-install infrastructure software for lab and development projects. SNAPS-Kubernetes was developed with Aricent and you can read more about this release on their blog here.
In my blog 6 months ago, I announced the release of SNAPS-OpenStack and SNAPS-Boot, and I highlighted Kubernetes as a future development area. As with the SNAPS-OpenStack release, we’re making this installer available while it's still early in the development cycle. We welcome contributions and feedback from anyone to help make this an easy-to-use installer for a pure open source and freely available environment. We’re also releasing the support for the Queens release of OpenStack—the latest OpenStack release.
The use of cloud-native technologies, including Kubernetes, should provide for even lower overhead and an even better-performing network virtualization layer than existing virtual machine (VM)-based solutions. It should also improve total cost of ownership (TCO) and quality of experience for end users. A few operators have started to evaluate Kubernetes, and we hope with SNAPS-Kubernetes that even more members will be able to begin this journey.
Our initial total cost of ownership (TCO) analysis with a virtual Converged Cable Access Platform (CCAP) core distributed access architecture (DAA) and Remote PHY technology has shown the following improvements:
- Approximately 89% savings in OpEx costs (power and cooling)
- 16% decrease in rack space footprint
- 1015% increase in throughput
We anticipate that Kubernetes will only increase these numbers.
Three Waves of NFV
SNAPS-Kubernetes will help deliver Virtual Network Functions (VNFs) that use fewer resources, are more fault-tolerant and quickly scale to meet demand. This is a part of a movement coined “cloud native.” This the second of the waves of NFV maturity that we are observing.
With the adoption of NFV, we have identified three overarching trends:
- Lift & Shift
- Cloud Native
- Autonomous Networks
Lift & Shift
Service providers and vendors typically support the Lift & Shift model today. These are large VMs running on an OpenStack-type Virtualized Infrastructure Manager (VIM). This is a mature technology, and many of the gaps in this area have closed.
VNF vendors often brag that their VNF solution runs the same version of software that runs on their appliances in this space. Although achieving feature parity with their existing product line is admirable, these solutions don’t take advantage of the flexibility and versatility that can be achieved by fully leveraging virtualization.
There can be a high degree of separation between the underlying hardware and operating system from the VM. This separation is great for portability, but it comes at a cost. Without some level of hardware awareness, it isn’t possible to take full advantage of acceleration capabilities. An extra layer of indirection is included, which can add latency.
Containers and Kubernetes excel in this quickly evolving section of the market. These solutions aren’t yet as mature as OpenStack and other virtualization solutions, but they are lighter weight and integrate software and infrastructure management. This means that Kubernetes will scale and fail over applications, and the software updates are also managed.
Cloud native is well suited for edge and customer-premises solutions where compute resources are limited by space and power.
Autonomous networks are the desired future in which every element of the network is automated. High-resolution data is being evaluated to continually optimize the network for current and projected conditions. The 3–6-year projection for this technology is probably a bit optimistic, but we need to start implementing monitoring and automation tools in preparation for this shift.
This release is based on Kubernetes 1.10. We will update Kubernetes as new releases stabilize and we have time to validate these releases. As with SNAPS-OpenStack, we believe it’s important to adopt the latest stable releases for lab and evaluation work. Doing so will prepare you for future features that help you get the most out of your infrastructure.
This initial release supports Docker containers. Docker is one of the most popular types of containers, and we want to take advantage of the rich ecosystem of build and management tools. If we later find other container technologies that are better suited to specific cable use cases, this support may change in future releases.
Because Kubernetes and containers are so lightweight, you can run SNAPS-Kubernetes on an existing virtual platform. Our Continuous Integration (CI) scripts use SNAPS-OO to completely automate the installation on almost any OpenStack platform. This should work with most OpenStack versions from Liberty to Queens.
SNAPS-Kubernetes supports the following six solutions for cluster-wide networking:
- Single Root I/O Virtualization (SRIOV)
- Dynamic Host Configuration Protocol (DHCP)
Weave, Calico and Flannel provide cluster-wide networking and can be used as the default networking solution for the cluster. Macvlan and SRIOV, however, are specific to individual nodes and are installed only on specified nodes.
SNAPS-Kubernetes uses Container Network Interface (CNI) plug-ins to orchestrate these networking solutions.
As we highlighted before, serverless infrastructure and orchestration continue to be future areas of interest and research. In addition to extending the scope of our infrastructure, we are focusing on using and refining the tools.
Multiple CMTS vendors have announced and demonstrated virtual CCAP cores, so this will be an important workload for our members.
Try It Today
Like other SNAPS releases, SNAPS-Kubernetes is available on GitHub under the Apache Version 2 license. SNAPS-Kubernetes follows the same installation process as SNAPS-OpenStack. The servers are prepared with SNAPS-Boot, and then SNAPS-Kubernetes is installed.
Have Questions? We’d Love to Hear from You
- Reach out on IRC: Server: Freenode Channel #cablelabs-snaps
- Contribute to the documentation, backlog and code on GitHub
- Send an email message directly to email@example.com
- Tweet to @RandyLevensalor
Subscribe to our blog to learn more about SNAPS in the future.
CableLabs Announces SNAPS-Boot and SNAPS-OpenStack Installers
After living and breathing open source since experimenting in high school, there is nothing as sweet as sharing your latest project with the world! Today, CableLabs is thrilled to announce the extension of our SNAPS-OO initiative with two new projects: SNAPS-Boot and SNAPS-OpenStack installers. SNAPS-Boot and SNAPS-OpenStack are based on requirements generated by CableLabs to meet our member needs and drive interoperability. The software was developed by CableLabs and Aricent.
SNAPS-Boot will prepare your servers for OpenStack. With a single command, you can install Linux on your servers and prepare them for your OpenStack installation using IPMI, PXE and other standard technologies to automate the installation.
The SNAPS-OpenStack installer will bring up OpenStack on your running servers. We are using a containerized version of the OpenStack software. SNAPS-OpenStack is based on the OpenStack Pike release, as this is the most recent stable release of OpenStack. You can find an updated version of the platform that we used for the virtual CCAP core and mobile convergence demo here.
How you can participate:
We encourage you to go to GitHub and try for yourself:
SNAPS (SDN & NFV Application Platform and Stack) is the overarching program to provide the foundation for virtualization projects and deployment leveraging SDN and NFV. CableLabs spearheaded the SNAPS project to fill in gaps in the open source community to ease the adoption of SDN/NFV with our cable members by:
Encouraging interoperability for both traditional and prevailing software-based network services: As cable networks evolve and add more capabilities, SNAPS seeks to organize and unify the industry around distributed architectures and virtualization on a stable open source platform to develop baseline OpenStack and NFV installations and configurations.
Network virtualization requires an open platform. Rather than basing our platform on a vendor-specific version, or being over 6 months behind the latest OpenStack release, we added a lightweight wrapper on top of upstream OpenStack to instantiate virtual network functions (VNFs) in a real-time dynamic way.
Seeding a group of knowledgeable developers that will help build a rich and strong open source community, driving developers to cable: SNAPS is aimed at developers who want to experiment with building apps that require low latency (gaming, virtual reality and augmented reality) at the edge. Developers are able to share information in the open source community on how they optimize their application. This not only helps other app developers, but helps the cable industry understand how to implement SDN/NFV in their networks and gain easy access to these new apps.
At CableLabs, we pursue a “release early” principle to enable contributions to improve and guide the development of new features and encourage others to participate in our projects. This enables us to continuously optimize the software, extend features and improve the ease of use. Our subsidiary, Kyrio, is also handling the integration and testing on the platform at their NFV Interoperability lab.
You can find more information about SNAPS in my previous blog posts “SNAPS-OO is an Open Sourced Collaborative Development” and “NFV for Cable Matures with SNAPS”
Who benefits from SNAPS?
- App Developers will have access to a virtual sandbox that allows them to test how their app will run in a cable scenario, saving them time and money.
- Service providers, vendors and enterprises will be able to build more exciting applications, on a pure open source NFV platform focused on stability and performance, on top of the cable architecture.
How we developed SNAPS:
We leverage containers which have been built and tested by the OpenStack Kolla project. If you are not familiar with Kolla, it is an OpenStack project that maintains a set of Docker containers for many of the OpenStack components. We use these scripts to deploy the containers because the Kolla-Ansible scripts are the most mature and include a broad set of features which can be used in a low latency edge data center. By using containers, we are improving the installation process and updating.
To maximize the usefulness of the SNAPS platform, we included many of the most popular OpenStack projects:
Additional services we included:
Where the future of SNAPS is headed:
- We plan to continue to make the platform more robust and stable.
- Because of the capabilities we have developed in SNAPS, we have started discussions with the OPNFV Cross Community Continuous Integration (XCI) project to use SNAPS OpenStack as a stable platform for testing test tools and VNFs with a goal to pilot the project in early 2018.
- Aricent is a strong participant in the open source community and has co-created the SNAPS-Boot and SNAPS-OpenStack installer project. Aricent will be one of the first companies to join our open source community contributing code and thought leadership, as well as helping others to create powerful applications that will be valuable to cable.
- As an open source project, we encourage other cable vendors and our members to join the project, contribute code and utilize the open source work products.
There are three general areas where we want to enhance the SNAPS project:
- Integration with NFV orchestrators: We are including the OpenStack NFV orchestrator (Tacker) with this release and we want to extend this to work with other orchestrators in the future.
- Containers and Kubernetes support: We already have some support for Kubernetes running in VMs. We would like to evaluate the benefit of running Kubernetes with or without the benefit/overhead of VMs.
- Serverless computing: We believe that Serverless computing will be a powerful new paradigm that will be important to the cable industry and will be exploring how best to use SNAPS as a Serverless computing platform.
Interactive SNAPS portfolio overview:
Have Questions? We’d love to hear from you
- Reach out on IRC: Server: Freenode Channel #cablelabs-snaps
- Contribute to the documentation, backlog and code on GitHub
- Send an e-mail directly to firstname.lastname@example.org
- Tweet to @RandyLevensalor
Don’t forget to subscribe to our blog to read more about NFV and SNAPS in our upcoming in-depth SNAPS series. Members can join our NFV Workshop February 13-15, 2018. You can find more information about the workshop and the schedule here.
NFV for Cable Matures with SNAPS
SNAPS is improving the quality of open source projects associated with the Network Functions Virtualization (NFV) infrastructure and Virtualization Infrastructure Managers (VIM) that many of our members use today. In my posts, SNAPS is an Open Source Collaborative Development Resource and Snapping Together a Carrier Grade Cloud, I talk about building tools to test the NFV infrastructure. Today, I’m thrilled to announce that we are deploying end-to-end applications on our SNAPS platform.
To demonstrate this technology, we recently held a webinar “Virtualizing the Headend: A SNAPS Proof of Concept” introducing the benefits and challenges of the SNAPS platform. Below, I’ll describe the background and technical details of the webinar. You can skip this information and go straight to the webinar by clicking here.
CableLabs’ SDN/ NFV Application Development Platform and OpenStack project (SNAPS for short) is an initiative that attempts to accelerate the adoption of network virtualization.
Network virtualization gives us the ability to simulate a hardware platform in software. All the functionality is separated from the hardware and runs as a “virtual instance.” For example, in software development, a developer can write an application and test it on a virtual network to make sure the application works as expected.
Why is network virtualization so important? It gives us the ability to create, modify, move and terminate functions across the network.
Why SNAPS is unique
- Creates a stable, replicable and cost-effective platform: SNAPS allows operators and vendors to efficiently develop new automation capabilities to meet the growing consumer demand for self-service provisioning. Much like signing up for Netflix, self-service provisioning allows customers to add and change services on their own, as opposed to setting-up a cable box at home.
- Provides transparent API’s for various kinds of infrastructure
- Reduces the complexity of integration testing
- Only uses upstream OpenStack components to ensure the broadest support: SNAPS is open source software which is available directly from the public OpenStack project. This means we do not deviate from the common source.
With SNAPS, we are pushing the limits of open source and commodity hardware because members can run their entire Virtualized Infrastructure Manager (VIM) on the platform. This is important because the VIM is responsible for managing the virtualized infrastructure of a NFV solution.
Webinar: Proof of Concepts
We collaborated with Aricent, Intel and Casa Systems to deploy two proof of concepts that are reviewed in the webinar. We chose these partners because they are leading the charge to create dynamic cable and mobile networks to keep up with world’s increasing hunger for faster, more intelligent networks tailored to meet customers' needs.
Casa and Intel: Virtual CCAP and Mobile Cores
CableLabs successfully deployed a virtual CCAP (converged cable access platform) core on OpenStack. Eliminating the physical CCAP provides numerous benefits to service providers, including power and cost savings.
Casa and Intel provided hardware and Casa Systems provided the Virtualized Network Function (VNFs) which ran on the SNAPS platform. The virtual CCAP core controls the cable plant and moves every packet to and from the customer sites. You can find more information about CCAP core in Jon Schnoor’s blog post “Remote PHY is Real.”
Advantages of Kyrio’s NFV Interop Lab
For the virtual CCAP demo, the Kyrio NFV Interop Lab provided a collaborative environment for Intel and Casa to leverage the Kyrio lab and staff to build and demonstrate the key building blocks for virtualizing the cable access network.
The Kyrio NFV Interop Lab is unique. It provides an opportunity for developers to test interoperability in a network environment against certified cable access network technology. You can think of the Kyrio lab as a sandbox for engineers to work and build in, enabling:
- Shorter development times
- Operator resources savings
- Faster tests, field trials and live deployments
Aricent: Low Latency and Backhaul Optimization
With Aricent we had two different proof of concepts. Both demos highlighted the benefits of having a cloud (or servers) at the service provider edge (less than 100 miles from a customer’s home):
- Low latency: We simulated two smart cars connected to a cellular network. The cars used an application running on a cloud to calculate their speed. If the cloud was too far away, a faster car would rear end a slower car before it was told to slow down. If the cloud was close, the faster car would slow down in time to prevent rear-ending the slower car.
- Bandwidth savings: Saving data that will be used by several people in a closer location can reduce the amount of traffic on the core network. For example, when someone in the same neighborhood watches the same video, they will see a local copy of the video, rather than downloading the original from the other side of the country.
The SNAPS platform continues CableLabs’ tradition of bringing leading technology to the cable industry. The collaborations with Intel, Aricent and Casa Systems were very successful because:
- We demonstrated end-to-end use cases from different vendors on the same version of OpenStack.
- We identified additional core capabilities that should be a part of every VIM. We have already incorporated new features in the SNAPS platform to better support layer 2 networking, including increasing the maximum frame size (or MTU) to comply with the DOCSIS® 3.1 specification.
In addition to evolving these applications, we are interested in collaborating with other developers to demonstrate the SNAPS platform. Please contact Randy Levensalor at email@example.com for more information.
Don’t forget to subscribe to our blog to read more about how we utilize open source to develop quickly, securely, and cost-effectively.
5 Things I Learned at OpenStack Summit Boston 2017
Recently, I attended OpenStack Summit in Boston with more than 5,000 other IT business leaders, cloud operators and developers from around the world. OpenStack is the leading open source software run by enterprises and public cloud providers and is increasingly being used by service providers for their NFV infrastructure. Many of the attendees are operators and vendors who collaboratively develop the platform to meet an ever-expanding set of use cases.
With over 750 sessions, it was impossible to see them all. Here are my top five takeaways and highlights of the event:
1. Edward Snowden's Opinions on Security and Open Source
In the biggest surprise of the event, Edward Snowden, former US NSA employee and self-declared liberator, joined us over a live video feed from an undisclosed location. He talked about the ethics and importance of the open source movement and how open source can be used to improve security and privacy.
Unlike vulnerabilities in proprietary software, those in open source are transparent. As a result, the entire community can learn from these exploits and how to prevent them in the future. Though not mentioned by Snowden, his rhetoric brought to mind the work done to secure OpenSSL after the heart bleed vulnerability was made public. This changed the way that core projects are managed. Snowden mentioned Apple’s iPhone as an example where vulnerabilities were found and the solution was not transparent:
“When Apple or Google has a bug, not only can we have no influence over the cure, but we don’t know anything about the cause and we don’t know what they have learned in effecting a cure. So, it’s not possible for everyone to use that knowledge to help build a better world for everyone.”
His talk brought applause from the audience and was a call to action as much as it was informative.
2. OpenStack is Helping Make the World Safer
The U.S. Army is using OpenStack to rapidly deliver the required curriculum for cyber command training and saving millions of dollars in the process. Using software development as an example, they created an agile development process where the instructors can improve the course rapidly and presented an example of their deployment of different virtual machines with malware and threat detection software. Instructors are able to create new content by submitting code to a source code repository and have it approved in less than a day. The new content is also available to graduates of the course in support of ongoing training. As a taxpayer, I can only hope that the other branches of the military will follow the Army’s lead in delivering the same innovative philosophy and process. These processes employed by the Army can be leveraged by service provides to deliver new services, apply security patches, and remedy service disruptions.
You can watch the keynote here and the in-depth talk below:
3. Lightweight OpenStack Control Planes for Edge Computing
OpenStack was designed to run large clouds managing thousands of servers in traditional data centers. Running OpenStack on a single local server allows service and OTT providers to manage CPEs using the same toolchain for managing VMs in their hosted cloud solutions.
Verizon’s keynote highlighting their uCPE is available here.
4. Aligning Container and Virtual Machine Technologies
My favorite forum session was a discussion to align VMs and containers. Containers address the application configuration and management challenges that are not as easily addressed with virtual machines. OpenStack can be used to manage the dependencies that containers need to run. In addition to the general summit proceeding, OpenStack has a forum format. You can learn more about the format here.
Leaders from both the OpenStack Nova team and the Linux Foundation’s Kubernetes were on the panel. Kubernetes performs many complementary and some overlapping tasks as OpenStack. Because Kubernetes was developed several years after Nova, they improved on some of the similar features.
CableLabs hosted an OpenStack Users Group meeting recently on the same subject called "OpenStack & Containers: Better Together".
5. Data Plane Acceleration
With the growth of OpenStack in the service provider space, the focus to move packets from point A to point B is as critical as ever. Open vSwitch continues to be a popular choice, and with the addition of DPDK support, they are reducing the latency involved with process packets in a virtualized network. Tapio Tallgren, the chair of OPNFV’s Technical Steering Committee, provides some results of testing DPDK with OPNFV. As many of you may know, CableLabs SNAPS project leverages OPNFV as a foundation. The Yardstick performance testing project, which Tapio discusses in his blog post Snaps-OO Open Sourced Collaborative Development Resource, is in the process of migrating many of their scenarios to leverage our SNAPS-OO utility.
FD.io is the newest player for accelerating the data plane. Their testing results in the lab are remarkable, and we are beginning to see some adoption for use in production. There was even a 1-day training session dedicated solely to FD.io.
With demos, product launches, and informative talks, OpenStack Summit Boston 2017 was a huge success. I hope to see you at the next one! If you have any questions about OpenStack don’t hesitate to leave a comment below.
SNAPS-OO is an Open Sourced Collaborative Development Resource
In a previous blog, I have provided an overview of the SNAPS platform which is CableLabs’ SDN/NFV Application development Platform and Stack project. The key objectives for SNAPS are to make it much easier for NFV vendors to onboard their applications, provide transparent APIs for various kinds of infrastructure and reduce the complexity of integration testing.
I am thrilled to share our latest SNAPS success. We have written an OpenStack API abstraction library that also contains many automated tests and we have contributed it to the Open Platform for NFV (OPNFV) project at the Linux Foundation. OPNFV is a project where service providers and network vendors collaborate to improve the capabilities and adoption of open source Network Functions Virtualization (NFV). Our results have also been shared at NFV World Congress, SDN World Congress, OPNFV Summit [video], Open Networking Summit (ONS) [video] [pdf] and the Big Communications Event (BCE).
The Rationale for our Approach
CableLabs has deep expertise developing specifications by following a collaborative, iterative approach. In many ways, the open source software development process mirrors many of these specification development processes. In the open source communities, CableLabs provides source code and feedback coming from our integration and debugging activity. In fact, CableLabs contributions are included in key open source projects such as OpenStack and OpenDaylight. In this way, we are making it easier for vendors to use open source projects to build solutions for the benefit of the entire ecosystem.
We have generated practical knowledge and insights through our hands-on experience of building and operating an active SDN/NFV application development lab. And we took vendor neutrality to the next level by basing our software stack on purely open source solutions and based on the OPNFV reference configuration. We did not use versions of OpenStack, OpenDaylight, etc. that have been tested and customized by a vendor. This allowed us to interact with a much larger community for new features and fixes.
The CableLabs team supported by vendors and services providers has moved our project into OPNFV as “SNAPS-OO”, based on the idea that it is an Object Oriented way to work with our SDN/NFV Application development Platform and Stack. The project was quickly accepted and is now being used by the release testing team to verify each OPNFV build. With the integration of SNAPS-OO into the OPNFV FuncTest project, our contributions are now part of the release criteria and suite of tests that will be used at the upcoming OPNFV PlugFest next month.
Some of the benefits that SNAPS-OO delivers are:
- Ease of use for new developers
- A rich library of example applications and test suites
- Support for accessing multiple secured clouds
- Automated cleanup of the NFVI when updates are applied
- Quick identification of component failure(s)
As a result of this open source approach, and in just a few weeks since SNAPS-OO was released, we have seen a significant increase in the level of contributions and adoption.
- Continue to expand the capabilities supported by SNAPS-OO.
- Encourage additional OPNFV projects to use SNAPS-OO.
- Use SNAPS-OO and other tools to run much more sophisticated SDN/NFV workloads.
- Share SNAPS-OO with more open source communities.
How SNAPS-OO Benefits Our Membership
SNAPS-OO is helping to improve the quality of the open source projects associated with the NFV infrastructure and Virtualization Infrastructure Managers that many members are using today and plan to use in the future. SNAPS-OO can be used to validate that the infrastructure is installed properly and it will be playing a key role in the Kyrio NFV Interoperability lab. Future NFV development provided by vendors will benefit from the use of SNAPS-OO. With the variety of workloads that we will be running on our SNAPS platform, we will be able to specify a single configuration that can run future NFV workloads alongside other cloud hosted applications.
Snapping Together a Carrier Grade Cloud
Today's enterprise and hyper-scale cloud solutions will not deliver everything needed to virtualize the service providers’ networks. However, cloud solutions do provide many of the building blocks as a great starting point.
Service providers are evolving their networks and services to better meet customer needs and expectations. Hosted applications are continuously updated with new features and consumers are starting to demand a similar frequency of change with services innovation. This rate of change and innovation in service provider networks will not be achieved by rolling more and more specialized hardware boxes to tens of millions of customers. Delivering software-based network solutions that reduce dependency on specialized hardware boxes is the only way to meet these customer expectations.
End users' expectation for service quality continues to increase, and they are typically not willing to accept a tradeoff between performance and capabilities. They want both increased performance and increased capabilities. Service Level Agreements (SLAs) are typically required for enterprise customers, but simply over-provisioning dedicated resources to meet these needs is neither economic, nor sustainable. High performance and network proximity are key to delivering interactive voice and video solutions with high bandwidth and low latency. No one wants to be misunderstood when delivering nuanced details during a videoconference with their stakeholders!
Currently, network services are delivered on several specialty devices located at customer sites or hosted by operators. Today, these specialty devices only provide a subset of needed capabilities and physical upgrades are both expensive and time consuming.
Critical Success Factors
In addition to being consistent and predictable, the network must be fast. There are no milliseconds to spare while moving across the network. For time sensitive applications such as cellular networks, there is no tolerance for physically routing packets inappropriately. They need to traverse the quickest route to their ultimate destination. To use a reference from "Smokey and the Bandit," one of my favorite movies, Bandit (Burt Reynolds' character) didn't drive through New York City to win the race from Texas to Georgia. He took the shortest and fastest route possible. Network traffic needs to do the same thing. Stick to the fastest and most direct route and only deviate when absolutely necessary,
This is not the natural mode for software running in an interrupt-driven multi-tasking environment. Much like humans trying to multi-task, tasks tend to take much longer if we are very busy. Software needs to be configured to prevent or bound interference when multiple workloads are running on the same computer.
"Location, location, location" is as important to network virtualization as it is to real-estate. Virtual Network Functions (VNFs) are the software components that replace the current Physical Network Functions (PNFs). VNFs need to be strategically placed, including positioning at the customer site or even other service provider nodes. Managing Wi-Fi networks requires access to devices at customer sites. Even when offloading the majority of the work to a hosted cloud, there are still physical accesses, routing and local security workloads that are best hosted on the customer site.
Low latency services, such as Content Delivery Networks need caching instances to be located relatively close to the customer site to reduce latency and core network bandwidth. Storage of data should not be on the other side of a busy or slow network connection. The path the data takes over the network needs to provide a consistent user experience. The network also needs to be flexible, as it must adapt to varying network loads and outages. Typically, enterprise cloud applications are designed for high availability and low cost. Speedier customer use is not always a consideration. The ability to easily manage service delivery locations by automatically placing and moving workloads within a data center, or geographically is a must for virtualizing network services.
VNFs must work with the deployed Network Function Virtualization (NFV) infrastructure and hardware. Should each VNF require a different infrastructure, it would be nearly impossible to manage and would cost much more to deploy. Interoperability can enable more competition and a broader set of vendors to deliver network services. Competition drives innovation. Standards and interoperability drive economies of scale.
ETSI-NFV is leading the way in developing the foundational standards for NFV based on a set of use cases and requirements coming from industry. Other standards bodies are referencing the ETSI-NFV work to address application-specific needs. These standards are becoming the basis for defining interoperability. But as with any standards effort, there will be many interpretations and implementations that follow these guidelines.
All of the independent components will need to be validated at key touch points to ensure interoperability and there is still no single test suite available today that will guarantee interoperability between VNFs or between VNFs and the infrastructure that hosts them. To help address this issue, ETSI-NFV is developing test specifications that are being referenced by OPNFV which itself was initiated by the ETSI NFV co-founders to accelerate implementation and feedback on the NFV specifications.
Over the next two to three years, we should see NFV being incorporated in mainstream cloud platforms. The expected performance and interoperability enhancements will increase the efficiency of compute and networking resources while requiring less power and space to run the same work. The improved, distributed nature of a trusted cloud will simplify managing applications running on or near the customers’ locations.
What CableLabs is Doing
CableLabs’ SDN/NFV Application development Platform and Stack project (SNAPS for short) is just one of the initiatives at CableLabs that attempts to accelerate and ease the adoption of network virtualization.
We are identifying the performance needs for network virtualization by evaluating the best open source software components and commercially available servers in order to build a stable and replicable platform for developing and demonstrating virtualized network capabilities and to validate interoperability and repeatability. Currently, the SNAPS project leverages a specific configuration of OPNFV which is being tested and hardened. Many of our enhancements have been included in the OPNFV "Colorado" release of the Apex installer.
Sharing our Expertise
While trying out different OpenStack installers, we soon ran into the dilemma of how to quickly use and validate our cloud in a repeatable manner. In response, we created a Python library whose responsibility is to deploy and provision OpenStack tenants from which we built a set of test suites to perform this validation. While the test suite tools are still under development, we have already made them available under the Apache v2 open source license in CableLabs' C3 collaborative software environment.
Additional contributors are always welcome. The source repository is located here: https://gerrit.cablelabs.com/#/admin/projects/snaps-provisioning
Accelerating NFV Adoption
The SNAPS project team, consisting of CableLabs member companies and vendors, is currently generating requirements and defining use cases to be shared publicly. These requirements include both performance and interoperability guidelines.
CableLabs wholly owned subsidiary Kyrio is using the lessons learned through this R&D process to drive evolution of the Kyrio SDN/NFV Interoperability lab.
We are actively involved in OPNFV and OpenDaylight, and we actively contribute to ETSI NFV.
Making Network Virtualization and Carrier Ethernet Easier
A CableLabs Demonstration of SDN with OpenSource Software
Seeing is believing, especially with all of the hype and limitless possibilities that new technologies promise. It is refreshing to quickly build a prototype for a real customer use case without all of the smoke and mirrors. Through an open and collaborative effort, CableLabs and BSS/OSS solutions provider Intraway developed a proof-of-concept and demonstration using open source software. This demonstration manages virtualized customer premise equipment (vCPE) and is compliant with Metro Ethernet Forum (MEF) defined Carrier Ethernet services. Details on the demonstration and how you run and build upon this work will be shared later in this post.
Software-Defined Networks and Network Function Virtualization, commonly known as SDN and NFV, are terms that are present in every telecommunications engineer’s mind today. It is hard to look at a technical newsletter or site without seeing at least one article with those acronyms.
Most Communications Service Provider (CSP) executives know what these letters mean in essence, and that they represent a shift in the networking paradigm. The real question is not how trendy this technological approach is, but rather if it is still just hype or if there are genuine business opportunities for service providers. In this article, we intend to answer that question. And, here’s a spoiler: this technology is closer than you think.
But, why would any CSP want to adopt this technology so early?
CableLabs sees three key benefits for leveraging SDN and NFV technologies:
- Reduce time to market
SDN and NFV allow for more flexible and agile deployment of networks and network services
- Reduce operational expense
SDN and NFV enable greater automation of network configuration by exposing network functions through application programming interfaces (APIs)
- Facilitate rapid innovation
Networking functions implemented in software can be developed, verified, and deployed - or rejected - far faster than hardware development cycles allow.
For instance, what happens in a non-virtualized environment when a customer requests a bandwidth upgrade for all its branches? Normally, it would take weeks to manually gather the network configuration. Then there are the planning, designing and delivery stages, which can take months. And what if that upgrade request was made to handle a high-demand period? Too often, by the time the service operator is ready to deliver, the high demand is over. But SDN/NFV, along with an orchestration layer, delivers new services and change requests in a matter of days or even hours. With SDN and NFV adoption, rapid innovation is the new network paradigm. It opens so many doors to new sources of revenue that we are just starting to imagine the possibilities. This technology represents such a shift that while not yet fully realized, its promise is already changing the business playbook for MSOs.
The Enabling Technology
SDN is commonly characterized as the separation of the networking equipment’s control capability from the data manipulation and forwarding processes. By separating the control of networked equipment from the hardware, SDN provides an opportunity to programmatically configure the network. By exposing control of networked elements through a software programming interface, configuration and control of the network can be automated through software scripts or applications.
Software controlling operation of networking equipment can be modified and tested easily, and deployed much faster than the design-build-test cycle of the hardware. Thus, SDN provides the opportunity to greatly streamline not only the initial deployment of services on the network, but also later modification and maintenance.
NFV is the modularization of functionality in networking equipment so that it can be separated from the network hardware and potentially executed outside the equipment in a sequence determined by the network engineer. NFV creates the opportunity to both simplify network hardware and introduce flexibility in the network architecture. Examples include network functions such as routing, load balancing, packet inspection, and firewall. When implemented in software as independent virtual network functions (VNF), data handling processes such as these can be invoked and chained together as needed.
CableLabs established early leadership in virtualization technology for deploying business services by exploring the use of open source SDN controller platforms to configure networking equipment for delivering business/commercial services. Open source refers to the software development by a community of talented developers that are not necessarily all working for the same company. Open source development allows for progress to be made at a pace that is not necessarily tied to a particular organization’s product schedule or hardware specifics.
But if SDN and NFV are to be more than a collection of proprietary, semi-closed ecologies, there needs to be a working reference model that the greater communications community can look to for ideas, guidance, and benchmarks.
CableLabs initiated a proof-of-concept project called the Virtualized Business CPE demonstration. The goal of the project is to demonstrate a properly implemented, open-source based SDN solution to support existing services. Intraway volunteered to join CableLabs (with consultant Inocybe Technologies) in this open-source effort.
Service activation and live modification of business services typically have the longest lead time and often require the most manual intervention. These use cases were selected for the initial demonstration. A point-to-point Ethernet circuit between two User Network Interfaces (UNIs) was chosen as the service. To demonstrate that SDN functions need not be proprietary, we used OpenDaylight as the control plane, OVSDB for the communications interface, and Open vSwitch to do the actual switching in the CPE devices.
For the hardware, we chose to use two Raspberry Pi devices to host the UNIs, an eight port Ethernet switch (to act as the network) and a laptop running Ubuntu Linux acting as the controller. Connected to each of the Raspberry Pi UNIs were laptops acting as the clients or end-devices. (The operating system of the two ‘client’ laptops is not important.)
This demonstration shows the creation of an Ethernet Private Line (EPL) between the two Raspberry Pi devices at a set bandwidth rate. After connectivity and traffic are established, we instruct the network to modify the existing rate through the controller software, which then issues instructions to the Raspberry Pi devices. The bandwidth of the EPL is immediately changed without having to tear down and rebuild the circuit!
Conclusion / Next Steps
With the success of the proof-of-concept, the source code is now publicly available in the open source project’s code repositories. CableLabs members and the vendor community can operationalize these concepts for many business services.
git clone https://git.opendaylight.org/gerrit/unimgr
git clone https://gerrit.opnfv.org/gerrit/lsoapi
This open collaboration with participation and feedback from CableLabs members, along with the active participation of vendors such as Intraway and Inocbye in open source projects, is one way that CableLabs helps ease the path forward with the adoption of new technology.
The Virtualized Business CPE demo, developed by CableLabs with participation from Inocybe Technologies and Intraway, will be featured at CableLabs 2016 Winter Conference in Orlando, Florida. If you are attending the conference, stop by to catch a glimpse of the future!
Randy Levensalor is a Lead Architect in the Wired Networks group at CableLabs.
OpenStack: Pushing the Cloud
Software defined, programmable networks are becoming ubiquitous. They need an underlying software orchestration and infrastructure layer to accelerate their adoption. Without reimaging network management to run at scale without constant human intervention, operational savings will never be realized. After attending The OpenStack Summit in early November, and seeing what the NFV working group has planned, I am of the opinion that OpenStack will be the leading platform for managing large-scale virtualized network infrastructure.
Virtualizing Networks is Only the First Step
Simply moving to a Network Functions Virtualization (NFV) infrastructure without changing the lifecycle management will save on capital investment and have some operational savings. In order to fully realize the operational savings of this transition, the virtual machines and the infrastructure that they run on need to execute at a massive “cloud” scale. One administrator will need to manage 1,000s or 10,000s of Virtual Network Functions (VNFs) and the underlying infrastructure, much like the hyper-scale providers do today. VNFs are the software equivalent of physical network devices.
OpenStack is an open source software platform commonly used to create public and private clouds. I have been working with OpenStack since 2011 as a component of HP’s public cloud and now as an open source orchestration and infrastructure management solution. As a platform, OpenStack has been gaining momentum with private clouds despite minimal penetration in the public cloud space.
Comcast and Time Warner Cable both shared how they are leaders in this area by using OpenStack to host portions of their internal infrastructure and customer facing services. Other cable operators can follow their leads to reduce operational and licensing (VMware for virtualization) costs by moving large portions of infrastructure to OpenStack based private clouds. CableLabs will work with the cable industry and vendors to create the infrastructure for demand driven networks of the future.
OpenStack Based Public Clouds
As noted in OpenStack Foundation’s COO’s keynote at the Open Stack Summit, it would be natural to see cloud computing becoming much more distributed across several providers rather than remain centralized. With the commoditization of compute and storage along with the standardization of OpenStack installations, the network is positioned to become the differentiator for cloud-based services. With direct access to customer premises, cable providers are well positioned to take advantage of this opportunity.
Managing Network Functions with OpenStack
With OpenStack’s core functionality to manage compute, storage and networking resources, it is capable of providing the infrastructure for large-scale virtualized networks. You can read more about Network Function Virtualization in Don Clark’s post about CableLabs joining OPNFV. Basically, NFV moves networking functionality off of physical, purpose-built hardware and onto virtual machines on cheaper commodity off-the-self servers.
Improvements in OpenStack networking, such as IPv6 support and performance (SR-IOV) added in the recent Juno release provide capabilities to start managing NFV today. The OpenStack NFV working group is driving additional OpenStack features to expand the number of supported NFV use cases and improve performance in the next release of OpenStack (Kilo), scheduled for an April 30, 2015 release.
There is a great niche for an open source orchestration and infrastructure management solution. OpenStack is well positioned to be that solution. However, with the growing number of users and special purpose use cases, there is also a risk that the community could fracture. Are you interested in getting involved with the OpenStack initiative? As a suggestion, install an OpenStack proof of concept or contact an OpenStack vendor to evaluate the current Juno release for your infrastructure needs. Many OpenStack distribution providers have versions available to evaluate with no fees or support contracts.