A “101” on DOCSIS® Technology: The Heart of Cable Broadband
Welcome to the first installment of our CableLabs 101 series about a suite of breakthrough technologies that are instrumental in the path toward the cable industry’s 10G vision—a new era of connectivity that will revolutionize the way we live, work, learn and play. These technologies work together to further expand the capabilities of cable’s hybrid fiber coaxial (HFC) network by increasing connection speeds and capacity, lowering latency and enhancing network reliability and security to meet cable customers’ needs for many years to come.
What Is DOCSIS?
Initially released by CableLabs in 1997, DOCSIS—or Data Over Cable Service Interface Specification—is the technology that enables broadband internet service over an HFC network, now used by hundreds of millions of residential and business customers around the globe. It is essentially the set of specifications that allows different cable industry vendors to design interoperable cable modems (the piece of network equipment that sits in the home) and cable modem termination systems (CMTSs—the network equipment that sits in the cable operator’s hub site). The CMTS is a head-end traffic controller that routes data between the modem in the home and the internet.
DOCSIS technology helped usher in the era of broadband and “always on” internet connections, enabling a wave of innovation that continues to this day. With DOCSIS technology, internet customers were no longer forced to use dial-up solutions that tied up home phone lines and probably caused a significant spike in family feuds. The DOCSIS solution changed everything. Not only did it allow for an “always-on” cable connection (no dial-up required!), it was also significantly faster than dial up. We’ll talk about connection speed—along with capacity, latency and other network performance metrics—and how they affect you a little later in this article.
How Does It Work?
DOCSIS technology governs how data is transmitted over the HFC network. To understand how it works, we need to start with the HFC network—the physical infrastructure that most cable companies use to provide high-speed internet connectivity to their customers. As the name suggests, the HFC network is composed of two parts: the fiber optical network and the coaxial network. HFC networks are predominantly fiber, as illustrated in our recent blog post. The remaining portion of the HFC network is coaxial cable. The coaxial network is connected to the optical fiber network at a “fiber node,” where the (fiber) optical signals are converted to radio frequency electrical signals for transmission over the coaxial network to the subscriber’s home. The HFC network seamlessly transmits data from the CMTS to your cable modem (we call this “downstream” or “download” traffic) or from your modem back to the CMTS (“upstream” or “upload”). In turn, the CMTS is connected to the internet via a set of routers in the service provider’s network.
Think of the HFC network as a “highway” and the data as traffic moving in “lanes” in either direction. In the downstream direction, DOCSIS devices translate the data from the internet into signals carried on the fiber optic portion of the HFC network and then down the coaxial network to your modem. On the upstream, the data that you upload is sent back up the network on a separate upstream “lane.” Traditionally, this “highway” has had more lanes dedicated to the downstream traffic than upstream, which matches current customer traffic patterns. All of this is about to change with the 10G vision, which strives toward symmetrical upstream and downstream service speeds.
How Has This Technology Evolved?
DOCSIS technology has come a long way since 1997. Over the years, it has undergone a few iterations, through versions 1.0, 1.1, 2.0 and 3.0 to 3.1. As DOCSIS has evolved, it has gotten faster by adding more lanes in each direction and it has become more energy-efficient as well. Along the way, several additions to the base technology have been continuously added. These include enabling lower latencies, increased security of the traffic, and tools to make the network more reliable. Today’s cable networks leverage DOCSIS 3.1 technology, which has enabled the widespread availability of 1 Gbps cable broadband services, allowing us to easily enjoy services like 4K video, faster downloads, seamless online gaming and video calls.
DOCSIS 4.0, released in March 2020, is another stepping stone toward that 10G vision. It will quadruple the upstream capacity to 6 Gbps, to match changing data traffic patterns and open doors to even more gigabit services, such as innovative videoconferencing applications and more. DOCSIS 4.0 equipment is still in the process of being developed and is seeing great progress each day toward device certification. Once certification is complete, cable vendors will start mass-producing DOCSIS 4.0-compatible equipment. With the widespread deployment of DOCSIS 4.0 technology, cable operators will have the ability to offer symmetrical multigigabit broadband services over their HFC networks.
How Does This Technology Affect Me and My Future?
All this talk about connection speeds, low latency, reliability and other performance metrics matter to us technologists because it’s how we gauge progress. But it’s so much more than giga-this and giga-that. These metrics will directly impact your future in a real, tangible way.
Over the past two decades, high-speed internet connectivity went from an obscure tech geek novelty to an important part of modern life. We are now streaming in 4K, collaborating on video chat, playing online games with people around the world, driving connected cars and so on. Continuous advancements in DOCSIS technologies are helping make this reality possible by increasing download and upload speeds, lowering latency—or lag—for a more seamless experience, and improving reliability and security to protect our online information.
DOCSIS 4.0 technology will enable symmetrical multigigabit services, ushering in a new wave of innovation across industries and applications, including healthcare, education, entertainment, collaboration technologies, autonomous vehicles and many more. In the near future, we will see advanced health monitoring services, immersive learning and work applications, visually rich VR/AR, holodecks, omnipresent AI assistants and other game-changing innovations that we haven’t even thought of yet. In many ways, the reach and flexibility of cable’s HFC infrastructure is the backbone of our 10G future, and DOCSIS—in combination with other advanced network technologies—is key to helping us reach this Near Future.
CableLabs Releases DOCSIS® Simulation Model
When it comes to technology innovation, one of the most powerful tools in an engineer’s toolbox is the ability to rapidly test hypotheses through simulations. Simulation frameworks are used in nearly all engineering disciplines as a way to understand complex system behaviors that would be difficult to predict analytically. Simulations also allow the researcher to control variables, explore a wide range of conditions and look deeply into emergent behaviors in ways that are either impossible or extremely challenging to accomplish in real-world testbeds or prototype implementations.
For some of our innovations, CableLabs uses the “ns” family of discrete-event network simulators (widely used in academic networking research) to investigate sophisticated techniques for making substantial improvements in broadband network performance. The ns family originated at Lawrence Berkeley National Laboratory in the mid-1990s, and has evolved over three versions, with “ns-3” being the current iteration that is actively developed and maintained. The open-source ns-3 is managed by a consortium of academic and industry members, of which CableLabs is a member. Examples of features developed with the help of ns include the Active Queue Management feature of the DOCSIS 3.1 specifications, which was developed by CableLabs using ns-2, and more recently, the Low Latency DOCSIS technology, which was created using models that we built in ns-3. In both cases, the simulation models were used to explore technology options and guide our decision making. In the end, these models were able to predict system behavior accurately enough to be used as the reference against which cable modems are compared to assess implementation compliance.
As a contribution to the global networking research community, CableLabs recently published its DOCSIS simulation model on the ns-3 “App Store,” thus enabling academic and industry researchers to easily include cable broadband links in their network simulations. This is expected to greatly enhance the ability of DOCSIS equipment vendors, operators and academic researchers to explore “what-if” scenarios for improvements in the core technology that underpins many of the services being delivered by cable operators worldwide. For example, a vCMTS developer could easily plug in an experimental new scheduler design and investigate its performance using high-fidelity simulations of real application traffic mixes. Because this DOCSIS model is open source, anyone can modify it for their own purposes and contribute enhancements that can then be published to the community.
If you’ve ever been interested in exploring DOCSIS performance in a particular scenario, or if you have had an idea about a new feature or capability to improve the way data is forwarded in the network, have a look at the new DOCSIS ns-3 module and let us know what you think!
The Cable Security Experience
We’ve all adjusted the ways we work and play and socialize in response to COVID. This has increased awareness that our broadband networks are critical – and they need to be secure. The cable industry has long focused on delivering best-in-class network security and we continue to innovate as we move on towards a 10G experience for subscribers.
CableLabs® participates in both hybrid fiber coaxial (HFC) and passive optical network (PON) technology development. This includes the development and maintenance of the Data Over Cable Service Interface Specification (DOCSIS®) technology that enables broadband internet service over HFC networks. We work closely with network operators and network equipment vendors to ensure the security of both types of networks. Let’s review these two network architectures and then discuss the threats that HFC and PON networks face. We’ll see that the physical media (fiber or coax) doesn’t matter much to the security of the wired network. We’ll discuss the two architectures and conclude by briefly discussing the security of the DOCSIS HFC networks.
A Review of HFC and PON Architectures
The following diagram illustrates the similarities and differences between HFC and PON.
Both HFC and PON-based FTTH are point-to-multipoint network architectures, which means that in both architectures the total capacity of the network is shared among all subscribers on the network. Most critically, from a security perspective, all downlink subscriber communications in both architectures are present at the terminating network element at the subscriber – the cable modem (CM) or optical network unit (ONU). This necessitates protections for these communications to ensure confidentiality.
In an HFC network, the fiber portion is between a hub or headend that serves a metro area (or portion thereof) and a fiber node that serves a neighborhood. The fiber node converts the optical signal to radio frequency, and the signal is then sent on to each home in the neighborhood over coaxial cable. This hybrid architecture enables continued broadband performance improvements to support higher user bandwidths without the need to replace the coaxial cable throughout the neighborhood. It’s important to note that the communication channels to end users in the DOCSIS HFC network are protected, through encryption, on both the coaxial (radio) and fiber portions of the network.
FTTH is most commonly deployed using a passive optical networking (PON) architecture, which uses a shared fiber down to a point in the access network where the optical signal is split using one or more passive optical splitters and transmitted over fiber to each home. The network element on the network side of this connection is an Optical Line Terminal (OLT) and at the subscriber side is an ONU. There are many standards for PON. The two most common are Gigabit Passive Optical Networks (GPON) and Ethernet Passive Optical Networks (EPON). An interesting architecture option to note is that CableLabs developed a mechanism that allows cable operators to manage EPON technology the same way they manage services over the DOCSIS HFC network – DOCSIS Provisioning of EPON.
In both HFC and PON architectures, encryption is used to ensure the confidentiality of the downlink communications. In DOCSIS HFC networks, encryption is used bi-directionally by encrypting both the communications to the subscriber’s cable modem (downlink) and communications from the subscriber’s cable modem (uplink). In PON, bi-directional encryption is also available.
How might an adversary (a hacker) look at these networks? There are four attack vectors available to adversaries in exploiting access networks:
- Adversaries can directly attack the access network (e.g., tapping the coax or fiber cable).
- They may attack a customer premises equipment (CPE) device from the network side of the service, typically referred to as the wide area network (WAN) side.
- They may attack the CPE device from the home network side, or the local area network (LAN) side.
- And they may attack the network operator’s infrastructure.
Tapping fiber or coaxial cables are both practical. In fact, tools to allow legitimate troubleshooting and management by authorized technicians abound for both fiber and coaxial cables. An incorrect assumption is to believe that fiber tapping is difficult or highly technical, relative to tapping a coaxial cable. You can easily find several examples on the internet of how this is simply done. Depending where the media is accessed, all user communications may be available on both the uplink and downlink side. However, both HFC and PON networks support having those communications encrypted, as highlighted above. Of course, that doesn’t mean adversaries can’t disrupt the communications. They can do so in both cases. Doing so, however, is relegated only to houses passed on that specific fiber or coaxial cable; the attack is local and doesn’t scale.
For the other attack vectors, the risks to HFC or PON networks are equivalent. CPE and network infrastructure (such as OLTs or CMTSs) must be hardened against both local and remote attacks regardless of transport media (e.g., fiber, coax).
Security Tools Available to Operators
In both HFC and PON architectures, the network operator can provide the subscriber with an equivalent level of network security. The three primary tools to secure both architectures rely on cryptography. These tools are authentication, encryption, and message hashing.
- Authentication is conducted using a secret of some sort. In the case of HFC, challenge and response are used based on asymmetric cryptography as supported by public key infrastructure (PKI). In FTTH deployments, mechanisms may rely on pre-shared keys, PKI, EAP-TLS (IETF RFC 5216) or some other scheme. The authentication of endpoints should be repeated regularly, which is supported in the CableLabs DOCSIS specification. Regular re-authentication increases the assurance that all endpoints attached to the network are legitimate and known to the network operator.
- Encryption provides the primary tool for keeping communications private. User communications in HFC are encrypted using cryptographic keys negotiated during the authentication step, using the DOCSIS Baseline Privacy Interface Plus (BPI+) specifications. Encryption implementation for FTTH varies. In both HFC and PON, the most common encryption algorithm used today is AES-128.
- Message hashing ensures the integrity of messages in the system, meaning that a message cannot be changed without detection once it has been sent. Sometimes this capability is built into the encryption algorithm. In DOCSIS networks, all subscriber communications to and from the cable modem are hashed to ensure integrity, and some network control messages receive additional hashing.
It is important to understand where in the network these cryptography tools are applied. In DOCSIS HFC networks, user communications are protected between the cable modem and the CMTS. If the CMTS functionality is provided by another device such as a Remote PHY Device (RPD) or Remote MACPHY Device (RMD), DOCSIS terminates there. However, the DOCSIS HFC architecture provides authentication and encryption capabilities to secure the link to the hub as well. In FTTH, the cryptographic tools provide protection between the ONU and the OLT. If the OLT is deployed remotely as may be the case with RPDs or RMDs, the backhaul link should also be secured in a similar manner.
The Reality – Security in Cable
The specifications and standards that outline how HFC and PON should be deployed provide good cryptography-based tools to authenticate network access and keep both network and subscriber information confidential. The security of the components of the architecture at the management layer may vary per operator. However, operators are very adept at securing both cable modems and ONUs. And, as our adversaries innovate new attacks, we work on incorporating new capabilities to address those attacks – cybersecurity innovation is a cultural necessity of security engineering!
Building on more than two-decades of experience, CableLabs continues to advance the security features available in the DOCSIS specification, soon enabling new or updated HFC deployments to be even more secure and ready for 10G. The DOCSIS 4.0 specification has introduced several advanced security controls, including mutual authentication, perfect forward secrecy, and improved security for network credentials such as private keys. Given our strong interest in both optical and HFC network technologies, CableLabs will ensure its own specifications for PON architectures adopt these new security capabilities and will continue to work with other standards bodies to do the same.
10G Integrity: The DOCSIS® 4.0 Specification and Its New Authentication and Authorization Framework
One of the pillars of the 10G platform is security. Simplicity, integrity, confidentiality and availability are all different aspects of Cable’s 10G security platform. In this work, we want to talk about the integrity (authentication) enhancements, that have been developing for the next generation of DOCSIS® networks, and how they update the security profiles of cable broadband services.
DOCSIS (Data Over Cable Service Interface Specifications) defines how networks and devices are created to provide broadband for the cable industry and its customers. Specifically, DOCSIS comprises a set of technical documents that are at the core of the cable broadband services. CableLabs manufacturers for the cable industry, and cable broadband operators continuously collaborate to improve their efficiency, reliability and security.
With regards to security, DOCSIS networks have pioneered the use of public key cryptography on a mass scale – the DOCSIS Public Key Infrastructure (PKIs) are among the largest PKIs in the world with half billion active certificates issued and actively used every day around the world.
Following, we introduce a brief history of DOCSIS security and look into the limitations of the current authorization framework and subsequently provide a description of the security properties introduced with the new version of the authorization (and authentication) framework which addresses current limitations.
A Journey Through DOCSIS Security
The DOCSIS protocol, which is used in cable’s network to provide connectivity and services to users, has undergone a series of security-related updates in its latest version DOCSIS 4.0, to help meet the 10G platform requirements.
In the first DOCSIS 1.0 specification, the radio frequency (RF) interface included three security specifications: Security System, Removable Security Module and Baseline Privacy Interface. Combined, the Security System plus the Removable Security Module Specification became Full Security (FS).
Soon after the adoption of public key cryptography that occurred in the authorization process, the cable industry realized that a secure way to authenticate devices was needed; a DOCSIS PKI was established for DOCSIS 1.1-3.0 devices to provide cable modems with verifiable identities.
With the DOCSIS 3.0 specification, the major security feature was the ability to perform the authentication and encryption earlier in the device registration process, thus providing protection for important configuration and setup data (e.g., the configuration file for the CM or the DHCP traffic) that was otherwise not protected. The new feature was called Early Authorization and Encryption (EAE), it allows to start Baseline Privacy Interface Plus (BPI) even before the device is provisioned with IP connectivity.
The DOCSIS 3.1 specifications created a new Public Key Infrastructure *(PKI) to handle the authentication needs for the new class of devices. This new PKI introduced several improvements over the original PKI when it comes to cryptography – a newer set of algorithms and increased key sizes were the major changes over the legacy PKI. The same new PKI that is used today to secure DOCSIS 3.1 devices will also provide the certificates for the newer DOCSIS 4.0 ones.
The DOCSIS 4.0 version of the specification introduces, among the numerous innovations, an improved authentication framework (BPI Plus V2) that addresses the current limitations of BPI Plus and implements new security properties such as full algorithm agility, Perfect Forward Secrecy (PFS), Mutual Message Authentication (MMA or MA) and Downgrade Attacks Protection.
Baseline Privacy Plus V1 and Its Limitations
In DOCSIS 1.0-3.1 specifications, when Baseline Privacy Plus (BPI+ V1) is enabled, the CMTS directly authorizes a CM by providing it with an Authorization Key, which is then used to derive all the authorization and encryption key material. These secrets are then used to secure the communication between the CM and the CMTS. In this security model, the CMTS is assumed trusted and its identity is not validated.
The design of BPI+ V1 dates back more than just few years and in this period of time, the security and cryptography landscapes have drastically changed; especially in regards to cryptography. At the time when BPI+ was designed, the crypto community was set on the use of the RSA public key algorithm, while today, the use of elliptic-curve cryptography and ECDSA signing algorithm is predominant because of its efficiency, especially when RSA 3072 or larger keys are required.
A missing feature in BPI+ is the lack of authentication for the authorization messages. In particular, CMs and CMTS-es are not required to authenticate (i.e., sign) their own messages, making them vulnerable to unauthorized manipulation.
In recent years, there has been a lot of discussion around authentication and how to make sure that compromises of long-term credentials (e.g., the private key associated with an X.509 certificate) do not provide access to all the sessions from that user in the clear (i.e., enables the decryption of all recorded sessions by breaking a single key) – because BPI+ V1 directly encrypts the Authorization Key by using the RSA public key that is in the CM’s device certificate, it does not support Perfect Forward Secrecy.
To address these issues, the cable industry worked on a new version of its authorization protocol, namely BPI Plus Version 2. With this update, a protection mechanism was required to prevent downgrade attacks, where attackers to force the use of the older, and possibly weaker, version of the protocol. In order to address this possible issue, the DOCSIS community decided that a specific protection mechanism was needed and introduced the Trust On First Use (TOFU) mechanism to address it.
The New Baseline Privacy Plus V2
The DOCSIS 4.0 specification introduces a new version of the authentication framework, namely Baseline Privacy Plus Version 2, that addresses the limitations of BPI+ V1 by providing support for the identified new security needs. Following is a summary of the new security properties provided by BPI+ V2 and how they address the current limitations:
- Message Authentication. BPI+ V2 Authorization messages are fully authenticated. For CMs this means that they need to digitally sign the Authorization Requests messages, thus eliminating the possibility for an attacker to substitute the CM certificate with another one. For CMTS-es, BPI+ V2 requires them to authenticate their own Authorization Reply messages this change adds an explicit authentication step to the current authorization mechanism. While recognizing the need for deploying mutual message authentication, DOCSIS 4.0 specification allows for a transitioning period where devices are still allowed to use BPI+ V1. The main reason for this choice is related to the new requirements imposed on DOCSIS networks that are now required to procure and renew their DOCSIS credentials when enabling BPI+ V2 (Mutual Authentication).
- Perfect Forward Secrecy. Differently from BPI+ V1, the new authentication framework requires both parties to participate in the derivation of the Authorization Key from authenticated public parameters. In particular, the introduction of Message Authentication on both sides of the communication (i.e., the CM and the CMTS) enables BPI+ V2 to use the Elliptic-Curves Diffie-Hellman Ephemeral (ECDHE) algorithm instead of the CMTS directly generating and encrypting the key for the different CMs.Because of the authentication on the Authorization messages, the use of ECDHE is safe against MITM attacks.
- Algorithm Agility. As the advancement in classical and quantum computing provides users with incredible computational power at their fingertips, it also provides the same ever-increasing capabilities to malicious users. BPI+ V2 removes the protocol dependencies on specific public-key algorithms that are present in BPI+ V1. , By introducing the use of the standard CMS format for message authentication (i.e., signatures) combined with the use of ECDHE, DOCSIS 4.0 security protocol effectively decouples the public key algorithm used in the X.509 certificates from the key exchange algorithm. This enables the use of new public key algorithms when needed for security or operational needs.
- Downgrade Attacks Protection. A new Trust On First Use (TOFU) mechanism is introduced to provide protection against downgrade attacks – although the principles behind TOFU mechanisms are not new, its use to protect against downgrade attacks is. It leverages the security parameters used during a first successful authorization as a baseline for future ones, unless indicated otherwise. By establishing the minimum required version of the authentication protocol, DOCSIS 4.0 cable modems actively prevent unauthorized use of a weaker version of the DOCSIS authentication framework (BPI+). During the transitioning period for the adoption of the new version of the protocol, cable operators can allow “planned” downgrades – for example, when a node split occurs or when a faulty equipment is replaced and BPI+ V2 is not enabled there. In other words, a successfully validated CMTS can set, on the CM, the allowed minimum version (and other CM-CMTS binding parameters) to be used for subsequent authentications.
In this work we provided a short history of DOCSIS security and reviewed the limitations of the current authorization framework. As CMTS functionality moves into the untrusted domain, these limitations could potentially be translated into security threats, especially in new distributed architectures like Remote PHY. Although in their final stage of approval, the proposed changes to the DOCSIS 4.0 are currently being addressed in the Security Working Group.
Member organizations and DOCSIS equipment vendors are always encouraged to participate in our DOCSIS working groups – if you qualify, please contact us and participate in our weekly DOCSIS 4.0 security meeting where these, and other security-related topics, are addressed.
CoMP over DOCSIS: Femtocells in the Age of vRAN
As promised in the last couple blogs discussing DOCSIS based femtocells, we’ve saved the best for last. So far in the series, we’ve made the case for femtocells over DOCSIS networks and laid out the total cost of ownership (TCO) benefits of this deployment model. In this final blog post, I’ll share the results of some testing we’ve been doing at CableLabs on using Coordinated Multipoint (CoMP) to optimize femtocell performance in dense deployments.
Decluttering the Radio Signal
Let’s step back and look at a key issue that has limited the benefit of femtocells in the past: intercell interference. When femtocells (or any cells, for that matter) are placed in close proximity, the radio signals each cell site produces can bleed into its neighbor’s territory and negatively affect network performance.
With CoMP, neighboring cells can coordinate their transmissions in a variety of ways to work collaboratively and prevent interference. They can share scheduling and beamforming data to avoid creating interference. Or, they can use joint processing, which allows multiple cells to talk to a single cell phone at the same time, increasing the signal quality.
Although it’s not a perfect analogy, it’s a bit like trying to listen to a bunch of people singing their favorite song at the top of their lungs versus listening to a choir following a conductor, as you see in the following figure. The former is old femtocells, and the latter is virtualized RAN (vRAN) femtocells using CoMP.
Since its inception, CoMP has been largely believed to require fiber transport links to work. For example, in TR 36.819, there’s a whole section devoted to the impact of “higher latency communication between points,” where “higher” refers to 5ms, 10ms or 15ms of latency. In that text, gains decrease as latency increases, ultimately going negative (i.e., losses in performance).
However, with the increase in attention on vRAN, particularly lower-layer splits like the work going on in Telecom Infra Project (TIP) vRAN Fronthaul and O-RAN Alliance WG4, latency takes on new meanings with respect to CoMP.
For example, what matters more, the latency from one radio unit to another or the latency from one virtualized baseband unit (vBBU) to another? And if it’s the latter, does that mean CoMP can provide benefit even over long-latency non-ideal vRAN fronthaul like DOCSIS?
To find out the answers to these questions, we set up a test bed at CableLabs in collaboration with Phluido to explore CoMP over DOCSIS. We used the hardware from the TIP vRAN Fronthaul project, with an LTE SW stack provided by Phluido that supports CoMP. We installed two radio units in different rooms, each radio connected via a DOCSIS® 3.0 network to the vBBU. We designated two test points, one with a phone located at the cell center, the other with both phone in the cell edge/cell overlap region.
Notably in our setup, the latency from radio unit to vBBU and radio unit to radio unit were both about 10ms. However, the latency between vBBUs was essentially zero as both radios shared the same vBBU. This setup is specifically designed to test whether vBBU-to-radio latency or vBBU-to-vBBU latency is more important for CoMP gains.
What we found is that radio-to-radio latency and radio-to-vBBU latency can be quite large in absolute terms, and we can still get good CoMP performance provided that latency is low between the vBBUs and that vBBU-to-radio unit latency is similar for the radios in the CoMP cluster, as you see below.
In other words, to realize CoMP gains, the relative latency between a set of cells is more important than the absolute latency from vBBU to each radio.
We tested four configurations of phones at the cell center versus the cell edge, or some mix thereof, as the following figure shows.
In case 1, we see full cell throughput at each phone with CoMP enabled or disabled. This is great; this result shows that we haven’t lost any system capacity at the cell center by combining the cells into a single physical cell ID (PCI) and enabling CoMP.
In case 2, the phone throughput jumped from 55 Mbps to 78 Mbps when we enabled CoMP, showing a CoMP gain of almost 50 percent.
In case 3, when we enabled CoMP, the phone at the cell edge saw a throughput gain of 84 percent. In this scenario, the throughput of the cell center phone saw a decrease in throughput. This illustrates a tradeoff of CoMP when using legacy transmission modes (TM4, in this case) where the operator must choose whether it wants to favor cell edge users or cell center users. With more advanced transmission modes (e.g., TM10), this tradeoff is no longer an issue. Note that this is true of any CoMP deployment and not related to our use of DOCSIS network fronthaul.
In case 4, we expected to see significant gains from CoMP, but so far we haven’t. This is an area of further investigation for our team.
vRAN Femtocell CoMP in MDUs
Let’s look at an example use case. Cell service in multi-dwelling units (MDUs) can be challenging. A combination of factors, such as commercial construction materials, glazing and elevation, affect the indoor signal quality. As discussed in my previous blog, serving those indoor users can be very resource intensive.
As an operator, it would be great to have a low-cost way to deploy indoor cells. With vRAN over DOCSIS networks supporting CoMP, the operator can target femtocell deployments at heavy users, then build CoMP clusters (i.e., the set of radios that collaborate) as needed to optimize the deployment.
Putting It All Together
The testing described here has shown that CoMP gains can be realized even when using long-latency fronthaul over DOCSIS networks. As these solutions mature and become commercial-ready, deployments of this type will provide the following for operators:
- Low-Cost Hardware: vRAN radios, particularly for femtocells, are low-complexity devices because the majority of the signal processing has been removed and put in the cloud. These radios can be built into the gateway customer premises equipment (CPE) already deployed by operators.
- Low-OPEX Self Installs: With vRAN radios built into DOCSIS CPEs, operators can leverage the simplicity of self-installation. The ability to dynamically reconfigure CoMP clusters means that detailed RF planning and professional installation aren’t necessary.
- High-Performing System: As shown in our testing results, CoMP gains can be realized over DOCSIS network–based vRAN femtocells. This eliminates another of the previous stumbling blocks encountered by earlier femtocell deployments.
Enabling 5G with 10G Low Latency Xhaul (LLX) Over DOCSIS® Technology
I am a GenXer, and I am addicted to my iPhone. But it’s not just me, today’s consumers, millennials and baby boomers and everyone in between, are increasingly spending more and more time on their mobile devices. Have you ever wondered what happens to your traffic when you interact with your iPhone or Android devices? The traffic reaches a radio tower, but it doesn’t just stop there – it needs to reach the internet via a connection between the cellular base station and a distant data center.
Traditionally, that connection (a.k.a., “xhaul”) is mostly provided by fiber. Fiber has great speed and latency performance but is costly to build. With advancements in LTE and 5G, mobile operators are increasingly deploying more and more radios deeper into the neighborhoods. They will need a more scalable solution to provide that xhaul without sacrificing the performance. This is where the hybrid fiber coaxial (HFC) network can help.
With ubiquitous cable infrastructures that are already in place, the cable operators have the scalability to support today’s LTE and tomorrow’s 5G networks without the cost of building new fiber networks. With DOCSIS 3.0+ as well as Low Latency Xhaul (LLX) technology, the DOCSIS network has performance that is virtually indistinguishable from fiber. The CableLabs 10G technologies make the HFC network a better xhaul network, which is a win-win for the consumers, mobile operators, and cable operators.
How Low Latency Xhaul (LLX) Works
Today’s DOCSIS technology provides a good starting point for mobile xhaul but may not be enough to support the ultimate latency requirements needed for future mobile traffic. DOCSIS upstream latency can range from a typical of 8-12 milliseconds to around a maximum of 50 milliseconds under heavy load. We want to see that latency down to 1 to 2 milliseconds range in order to support 5G.
The LLX technology is specifically designed to reduce the latency experienced by mobile traffic while traversing the DOCSIS transport network on its way to the internet. The LLX technology development started about 3 years ago as a joint innovation project between CableLabs and Cisco. I wrote about it here and here.
So, how does LLX work? Let’s look at the case of LTE backhauled over a DOCSIS network as an example. Today, LTE and DOCSIS are two independent systems – their operations occur in serial, and the overall latency is the sum of the two system latencies. But from an engineer’s point of view, both technologies have a similar request and grant-based mechanism to access the channel. If the two processes can be pipelined, then LTE and DOCSIS operations can take place in parallel, removing the “sum” from the latency equation. To enable pipelining, we designed a protocol that utilizes a message called the bandwidth report (BWR) that allows the LTE network to share information with the DOCSIS network. Pipelining is a unique and inventive aspect of LLX and is the heart of what creates a low latency transport.
So, just how well does LLX work? We have recently teamed up with Shaw, one of our Canadian members, as well as our technology development partners Cisco and Sercomm to perform a series of lab trials. The detail of the trials will be published in the upcoming SCTE Cable-Tec Expo in October. But as a preview, we demonstrated that even when the DOCSIS network is heavily loaded, LLX consistently reduced the DOCSIS upstream latency down to 1 to 2 milliseconds, all without adversely affecting other traffic.
Deploying LLX Technology
The LLX specification was published a few months ago, the result of collaborative efforts from key cable and mobile equipment vendors in the CableLabs-led LLX working group.
LLX technology is designed to work for a variety of deployment models, including backhaul and fronthaul, over DOCSIS as well as over PON networks. To this end, we have taken the technology to mobile industry standardization organizations such as the O-RAN Alliance whose current focus is fronthaul.
LLX works in the DOCSIS 3.0 and later networks as a software upgrade to the CMTS. It has been implemented on commercial DOCSIS and mobile equipment. More information on LLX is available here.
For those attending the SCTE Cable-Tec Expo in New Orleans, we will be discussing the innovation on the Innovation Stage at 12:45pm local time with my industry partners from Shaw, Cisco, and Sercomm. I will also dive deep into the technology and the Shaw trial results in my SCTE panel “Mobile X-haul and DOCSIS”, Wednesday October 2nd at 9am local time. Hope to see you there.
CableLabs Low Latency DOCSIS® Technology Launches 10G Broadband into a New Era of Rapid Communication
Remember the last time you waited (and waited) for a page to load? Or when you “died” on a virtual battlefield because your connection couldn’t catch up with your heroic ambitions? Many internet users chalk those moments up to insufficient bandwidth, not realizing that latency is to blame. Bandwidth and latency are two very different things and adding more bandwidth won’t fix the internet lag problem for latency-sensitive applications. Let’s take a closer look at the difference:
- Bandwidth (sometimes referred to as throughput or speed) is the amount of data that can be delivered across a network over a period of time (Mbps or Gbps). It is very important, particularly when your application is trying to send or receive a lot of data. For example, when you’re streaming a video, downloading music, syncing shared files, uploading videos or downloading system updates, your applications are using a lot of bandwidth.
- Latency is the time that it takes for a “packet” of data to be sent from the sender to the receiver and for a response to come back to the sender. For example, when you are playing an online game, your device sends packets to the game server to update the global game state based on your actions, and it receives update packets from the game server that reflect the current state of all the other players. The round-trip time (measured in milliseconds) between your device and the server is sometimes referred to as “ping time.” The faster it is, the lower the latency, and the better the experience.
Interactive applications, where real-time responsiveness is required, can be more sensitive to latency than bandwidth. These applications really stand to benefit from technology that can deliver consistent low latency.
As we’ve alluded, one good example is online gaming. In a recent survey we conducted with power users within the gaming community, network latency continually came up as one of the top issues. That’s because coordinating the actions of players in different network locations is very difficult if you have “laggy” connections. The emergence of Cloud gaming makes this even more important because even the responsiveness of local game controller actions depends on a full round-trip across the network.
Queue Building or Not?
When multiple applications share the broadband connection of one household (e.g. several users performing different activities at the same time), each of those applications can have an impact on the performance of the others. They all share the total bandwidth of the connection, and they can all inflate the latency of the connection.
It turns out that applications that want to send a lot of data all at once do a reasonably good job of sharing the bandwidth in a fair manner, but they actually cause latency in the network when they do it, because they send data too quickly and expect the network to queue it up. We call these “queue-building” applications. Examples are video streaming and large downloads, and they are designed to work this way. There are also plenty of other applications that aren’t trying to send a lot of data all at once, and so don’t cause latency. We call these “non-queue-building” applications. Interactive applications like online gaming and voice connections work this way.
The queue-building applications, like video streaming or downloading apps, get best performance when the broadband connection allows them to send their data in big bursts, storing that data in a buffer as it is being delivered. These applications benefit from the substantial upgrades the cable industry has made to its networks already, which are now gigabit-ready. These applications are also latency-tolerant – user experiences are generally not impacted by latency.
Non-queue-building applications like online gaming, on the other hand, get the best performance when their packets don’t have to sit and wait in a big buffer along with the queue-building applications. That’s where Low Latency DOCSIS comes in.
What is Low Latency DOCSIS 3.1 and how does it work?
The latest generation of DOCSIS that has been deployed in the field—DOCSIS 3.1—experiences typical latency performance of around 10 milliseconds on the access network link. However, under heavy load, the link can experience delay spikes of 100 milliseconds or more.
Low Latency DOCSIS (LLD) technology is a set of new features, developed by CableLabs, for DOCSIS 3.1 (and future) equipment. LLD can provide consistent low latency (as low as 1 millisecond) on the access network for the applications that need it. The user experience will be more consistent with much smaller delay variation.
In LLD, the non-queue-building applications (the ones that aren’t causing latency) can take a different path through the DOCSIS network and not get hung up behind the queue-building applications. This mechanism doesn’t interfere with the way that applications go about sharing the total bandwidth of the connection. Nor does this reduce one application's latency at the expense of others. It is not a zero-sum game; rather, it is just a way of making the internet experience better for all applications.
So, LLD gives both types of applications what they want and optimizes the performance of both. Any application that wants to be able to send big bursts of data can use the default “classic” service, and any application that can ensure that it isn’t causing queue build-up and latency can identify its packets so they use the “low latency” service. Both then share the bandwidth of the broadband connection without one getting preference over the other.
Incorporating LLD Technology
Deploying Low Latency DOCSIS in a cable operator’s network can be accomplished by field-upgrading existing DOCSIS 3.1 CMs and CMTSs with new software. Some of the low latency features are even available to customers with older (pre-DOCSIS 3.1) CMs.
The technology includes tools that enable automatic provisioning of these new services, and it also introduces new tools to report statistics of latency performance to the operator.
DOCSIS equipment manufacturers are beginning to develop and integrate LLD features into software updates for CMTSs and CMs, and CableLabs is hosting Interoperability Events this year and next year to bring manufacturers together to help iron out the technology kinks.
We expect these features to become available to cable operators in the next year as they prepare their network to support low latency services.
LLD provides a cost-effective means of leveraging the existing hybrid fiber-coaxial (HFC) network to provide a high-performance network for latency-sensitive services. These services will help address customers’ requirements for many years into the future, maximizing the investments that cable operators have made in their networks. The cable industry is provisioning the network with substantial bandwidth and low latency to take another leap forward with its 10G networks.
For those attending the SCTE Cable-Tec Expo in New Orleans, Greg will be presenting the details of this technology on a SCTE panel “Low Latency DOCSIS: Current State and Future Vision”
No doubt our cable industry has a unique culture of working and innovating together to solve technical issues. But there are best practices from other communities which we can build from; these practices inform how we can continue to develop toward more reliable services. By “reliable,” as it relates to service, I mean reliable, available, and resilient services, which result from reliable, available, resilient, repairable, maintainable, and highly performing cable networks, not to mention operations focused on the customers’ needs. On the other hand, specifically used, reliability refers to the probability of not experiencing failure, whereas availability refers to the expected proportion of time that something is working as intended. These are very related, but very different things. You can read more here. But when we speak generally about reliability, often many of these like concepts are relevant.
What is Unique About Cable Relating to Reliability Concepts?
For one thing, DOCSIS® networking is unique. Each version of DOCSIS technologies improved performance, but also increased the robustness of the services it supports. Error correction, profile management, pre-equalization, echo cancelers, and other technologies have enabled this performance extension, but also these advantages create separation from the impairment and service failures, allowing for maintenance before service is impacted.
Another unique advantage is Proactive Network Maintenance (PNM). The advantages of DOCSIS technologies are what make PNM possible. We use data to find impairments in the network that, left untreated, will eventually impact service. This capability affords operators the opportunity to find and remove impairments early, before the network is further damaged by degradation, and service is impacted severely. Networks can be maintained well, but also services remain available while the network is experiencing failure.
Cable operators and vendors in cable have analog radio frequency (RF) expertise with a digital mindset. The cable industry knows RF, and that knowledge has helped it get the most out of the physical layer of the network. That deep understanding of the network’s physical layer is why mitigating network failure modes is second nature, and the industry has the needed skills.
Then there’s the industry’s “laser focus.” Pushing fiber out deeper into the network can improve reliability and availability, but current technology does lack some of the PNM advantages. There is work to do, but the capabilities are there for us to develop.
What Are the Best Practices We Can Re-use?
Designing communication networks for reliability carries many best practices and experience.
- The ability to understand and mitigate failures before deployment – We have defined PNM use cases based on the measurements we’ve been able to define in the DOCSIS specifications. Now, we must extend that work to link to failure modes, effects, and criticality analysis, and root cause analysis, to inform technology choices, measurements for management, and design for reliability.
- Condition based maintenance – Maintenance optimization research is clear that in any practical situation it is almost always more cost efficient to base maintenance on condition information rather than age information.
- Prognostics and Health Management (PHM) – A newer field of reliability, PHM is a lot like our PNM. PHM is a research field of study using data sources (e.g., vibration in mechanical systems, or charge time in batteries) to determine the remaining useful life of a component or system. PNM is a clear cousin to that field, so we can certainly share and gain benefit from that work.
- Certification testing – Certifying cable modems (CMs) has improved the PNM responsiveness of CMs, and the same can be true about cable modem termination systems (CMTSs) as that part of the network begins to align.
- Maintenance optimization – Service reliability and availability, in addition to network reliability and availability and robustness, are important focuses for the industry; they relate, but are distinct and important in their own. The network can fail while service continues to perform at a high level, so maintenance can be better planned in this situation.
Thoughts for the Future of Cable
- More options mean more standardization – Adding more options to the technology choices allows operators to better meet the unique needs of their customer base. However, keeping it all standardized increases operability and repairability so that service is highly reliable and available.
- Each feature needs measurements – As we add options and features to cable technologies, each option needs special measurements to assure that the feature can be managed properly. DOCSIS 4.0 technology is full of options, so we’ll need a critical eye on each to make sure those options can be operated reliably.
- Pushing the limits of technology requires more diligence on PNM – As we rely on tighter tolerances and more complexity on issues like upstream noise, echo cancelation, and error correction, we need more information about how those perform, and more diligent PNM practice relating to them.
- Impairments relate to capacity and network resilience – As capacity becomes a stronger focus, the impact of impairments on that capacity becomes more important, so cable network reliability is entwined.
- As we push higher capacity to the edge, redundancy must come with it – With more capacity comes more critical services, and more impact to the lives of customers. A failure becomes more impactful as a result. Then, as the cost of a failure increases, large failures become more expensive, driving the need for more network resiliency, and thus more redundancy.
Strong Foundation, Strong Future
Building on a strong foundation of PNM and DOCSIS technologies, the cable industry has the right culture and technology foundation to take communications to a reliable future. We have lots of work to do, but we’re on the right path to do it. Here we go!
The 10g platform is going to provide reliable service. As the cable industry embarks on the development of 10G services, there is a lot of work ahead, but we already have a strong foundation of experience and technology to build upon.
The 10 Gbps goal is about performance. But it must come with low cost, high quality, and sufficient reliability. 10G services have to be easy to install reliably, remain stable and robust against cable plant variations and conditions, and provide a wealth of service flexibility so that services remain reliable under a broad set of use cases.
The Road to 10G…
At CableLabs, we’ve taken big leaps toward 10G with DOCSIS® 4.0, including Full Duplex DOCSIS, and with cable modems (CMs) which will be capable of 5 Gbps symmetrical service in the near future. To fully arrive at 10G, we need to enable 10 Gbps downstream speeds. To accomplish that, we’ll need to expand our use of available spectrum, and we’ll likely need to use that spectrum in a highly efficient manner. Pushing higher bandwidth solutions deeper into the network and closer to the edge customers will be required, too. We have a lot of innovation ahead of us to get to the 10G future.
…Is Paved with Innovation
Invention often begins with an initial solution that is later repeated for verification, then validated further. That initial solution then needs to be scaled; in other words, it needs to be made repeatable, at a low cost, and with sufficient reliability.
Fortunately, DOCSIS networking is a technology with many reliability traits integrated. Data are delivered reliably due to Forward Error Correction. Profile management can control the data rate to allow the best performance possible, but not push performance to low reliability. Adjustments to the connection between the cable modem termination system (CMTS) and CM assure reliable transmission continues under constant environmental and network changes. And Proactive Network Maintenance (PNM) assures that plant conditions are discoverable, and that they can be translated into maintenance activities that can further assure services stay reliable at low cost. The cable industry is starting on a solid foundation.
Consider one possible direction we could take on the road to 10G. As we begin to expand the frequencies that DOCSIS uses, we may need improved error correction, better profile management, or better CMTS-to-CM coordination to assure reliable services continue at expected levels. However, pushing these limits might also mean new failure modes in the plant, or greater service sensitivity to existing failure modes, thus increasing the importance of PNM. Operators should up their PNM game now, understanding that it will be an even more important element to assure a reliable 10G future.
A Super Highway in Many Directions
Because of this strong reliability foundation in cable technologies, particularly DOCSIS, we can build our 10G future with reliability in mind. Rather than simply extending our boundaries and hoping that our existing methods to assure reliable services will be sufficient, we can define solutions that bring reliability with them. By focusing simultaneously on increased performance, lower operational costs, and reliable services, we can evolve into an effective, desirable 10G future for the world.
Also, by thoughtfully choosing the technologies to develop, we can create degrees of freedom and opportunities to enhance reliability while developing 10G. This is the right approach for the industry to take because reliability can only be built into a service, not added later. By choosing to develop solutions now that expand our options for reliable services, we can enable operators to have full control of their services. To make it work reliably, PNM will be there, and so will a few other advantages to come.
Innovation Journeys: 10G is new. We have been working on it for years.
You may have noticed that CableLabs is focused on innovation. One of our goals is to be recognized as the leading industry innovation lab in the world but talking about our innovation can be a bit tricky. Our job is to deliver innovation for the worldwide cable industry, but we can’t really talk about what we are working on now. We need to keep that secret for our member companies (cable operators) until the technology is ready to launch.
Our CEO, Phil McKinney has talked about how innovation is messy. Where you start may not be where you end up. I want to tell you about the path that led to one of our most important innovations--and part of our 10G platform. Low latency.
Our Low Latency Journey
We started on this journey over four years ago, with a challenge question (Focus in the FIRE methodology): What applications will drive a need for 60Mbps+ of sustained Internet bandwidth? That led to ideation sessions that unearthed the usual suspects: Internet of Things (billions of sensors, but each with such low bandwidth that they still don’t add up to much), 4K streaming video (good try, but still only 15Mbps or less), “Big Data” (sorry, not really a candidate for consumer households). Those applications didn’t quite answer the question.
But the emergence of 360° immersive video looked promising. Experiencing some of the earliest 360° video at the beginning of 2014 (shot on 6 Go-Pro’s, manually stitched) on a low-resolution Oculus Development Kit Virtual Reality headset got us thinking about where the technology might lead. Six 4K videos, streamed to the headset met the challenge of over 60Mbps, although compression gains would reduce the bandwidth and resolution increases would increase it.
Rather than “geeking out” on the technical possibilities, we followed advice from Phil: “Talk to consumers!” In February of 2015, we did primary research, bringing 50 varied members of the public into CableLabs to try out “immersive video content.” Rather than just focusing on virtual reality (VR) headsets, we constructed some other ways of consuming the content, such as immersive multi- 4K TV displays, ultra-wide projectors, tablets and regular TVs. We needed to understand whether “regular humans” (not geeks) would like these technologies.
The consumer research was massively informative. We shared the insights with our member companies at the time and realized that this ecosystem was likely to take off. We stepped back and tried to work out other mass-market use cases for VR.
We pivoted. We started to look at the possibilities of transforming how people communicate, and the ability to have holographic telepresence using digital human technology to perform digital headset removal. We don’t really want to talk to another person and see that person with a headset on; we want to see other people eye to eye and have them see us eye to eye. To prove the point, later in 2015 and into early 2016 we developed eye and mouth tracking capabilities that we added to a wireless VR headset and developed a digital human avatar of one of our staff.
We linked the head, eye and mouth tracking to real-time control of the digital avatar.
And in May of 2016 we demonstrated this to our board of directors.
We also found that realistic digital human avatars take LOTS of compute to render in real time, and that required a tethered PC. Even as mobile processors get faster and more capable, PC graphics will always be faster and more capable, due to their power budget. Phones get hot when you try to render realistic humans. To get to mass-market adoption, we need to go wireless and move the PC out of the home.
No Less Than a Revolution
VR needs incredibly low latency between head movement and the delivery of new pixels to your eyes, or you start to feel nauseated. To move the PC out of the home, we need to make the communications over the cable network be a millisecond or less round trip. But our DOCSIS® technology at the time could not deliver that.
So, we pivoted again. Since 2016, CableLabs DOCSIS architects Greg White and Karthik Sundaresan have been focused on revolutionizing DOCSIS technology to support sub-1ms latency. Although VR is still struggling to gain widespread adoption, that low and reliable DOCSIS latency will be a boon to gamers in the short term and will enable split rendering of VR and augmented reality (AR) in the longer term. The specifications for Low Latency DOCSIS (as a software upgrade to existing DOCSIS 3.1 equipment) have been released, and we’re working with the equipment suppliers to get this out into the market and to realize the gains of a somewhat torturous innovation journey.
Low latency is a key component of our 10G initiative. You can read more about the importance of latency here, and gain access both to a technical brief (members only) and to a detailed report (members only) on Wi-Fi latency in retail Wi-Fi routers.