A Milestone in Wi-Fi / LTE-U Coexistence
Today is an important milestone for unlicensed spectrum coexistence - the Wi-Fi Alliance (WFA) has released its plan for testing how well LTE-Unlicensed coexists with Wi-Fi.
This culminates many months of work by many expert engineers within the WFA and its membership, including CableLabs staff. The outcome is that we now have a definitive set of tests, based on real-world consumer data, against which to judge LTE-U – and we can move past the competing technical studies that were the hallmark of 2015.
The WFA and its staff are to be commended for bringing all sides to the table on this issue of such importance for broadband consumers everywhere. The test plan, developed in record-time, is a product of compromise by all sides, and LTE-U proponents participated robustly in the process. There are a number of tests that CableLabs supported as important that ultimately were not adopted. But the final product is nevertheless essential – both in validating coexistence performance of any LTE-U device proposed for deployment, and as a sign that diverse industry interests can work toward solutions as wireless access becomes ever more important for consumers.
CableLabs will continue to be engaged as the WFA moves to implement this plan with authorized test labs. We look forward to a transparent process with results reported publicly by the WFA. As we move to this implementation phase, it is worth describing what the test plan does, in order to understand why it is so important.
At a high level, the test plan does the following:
- Checks that LTE-U devices select the most lightly used channel, as LTE-U proponents say they will do;
- Ensures that new Wi-Fi networks can access the channel when LTE-U is active;
- Measures the impact to Wi-Fi throughput and latency from LTE-U; and,
- Ensures that LTE-U adapts its use of the spectrum in response to variation in consumer use of Wi-Fi, as occurs in the real world, in real time.
And it does all of this at signal levels that have been shown with real-world data to be reflective of consumer use of Wi-Fi hotspots. These tests are necessary due to the well-documented shortcomings in the LTE-U Forum coexistence specification, and the lack of standardized test procedures to date, which has yielded vastly different coexistence conclusions. For more information on our views of the test procedures, see Jennifer Andreoli-Fang’s contribution to the August workshop of the WFA, which is available here.
Reasonable compromises have been made by all sides in developing this test plan. It is time to move forward using the outcome of this process, in full, as the sole source of reliable determinations of LTE-U coexistence.
Fair LTE-U Coexistence Far From Proven In CableLabs / Qualcomm Testing
CableLabs has been working hard to ensure that the introduction of LTE into unlicensed spectrum is a win for wireless broadband, and that it does not disrupt the Wi-Fi services that consumers have come to rely on. We have covered in prior posts why that is a challenge, since LTE-U can take advantage of Wi-Fi’s inherent politeness. Here we will review recent technical work we did with a major proponent of LTE-U, Qualcomm Technologies, and explain why that effort only reinforces our concerns.
In brief, we observed that current LTE-U prototype equipment is quite primitive — it is really just in mock-up state at this point — and incapable of demonstrating important coexistence features. The vendor-promised features, most of which are not required or even identified by the LTE-U Forum in its latest specifications, are not yet working to enable fair and reliable coexistence and confidence in testing. In addition, we found that claims of its ability to share fairly rest on a seemingly faulty understanding of how Wi-Fi shares spectrum. For CableLabs, this reinforces the need for a collaborative and open standards development process.
The Wi-Fi community has long sought the same collaborative standards development process for LTE-U that LAA has enjoyed (License Assisted Access LTE is the flavor of unlicensed LTE being developed in the mobile standards body, 3GPP). But in the absence of LTE-U standards development, CableLabs has engaged directly with the promoters of LTE-U in an attempt to do the fundamental research required to find coexistence solutions. Far from converging on solutions, however, our work to date on LTE-U has raised more questions than answers.
Lessons Learned with Qualcomm’s LTE-U
CableLabs recently concluded a brief technical engagement at Qualcomm’s campus in San Diego, which was preceded by a lengthy negotiation of what we would be allowed to test on site. Ultimately, the scope of the plan was much narrower than our guidance and focused on a limited set of basic coexistence tests. It certainly was not the fulsome research that we recommended and is required to address the concerns of Wi-Fi technologists, which we summarized in a presentation to the Wi-Fi Alliance last week.
We began this limited test hoping that we would be able to independently validate the definitive statements of LTE-U proponents that it has been ‘proven to coexist’, and is ‘more friendly to Wi-Fi than Wi-Fi is to itself’.
Unfortunately, our main conclusion from the three weeks we spent on site at Qualcomm is that there is no basis for definitive technical statements about LTE-U coexistence. The reason for this is surprisingly simple: LTE-U is in a prototype phase of development, and does not possess the features that its proponents have noted are important to coexistence.
For instance, Qualcomm has noted that their LTE-U solution will sense the spectrum for Wi-Fi activity and adjust its duty cycle ‘on’ time for rapidly changing congestion conditions. But that is not what we were shown. What we saw was an LTE-U prototype that must have its duty cycle manually programmed; it has no adaptation capabilities at all. There were other issues as well: For example, the equipment did not natively use the 5 GHz band that is targeted for LTE-U, and it only supported a single user device. We will refrain from going on at length here, but as is apparent in the photo below, LTE-U requires substantial further development. In short, we were surprised to see that the state of the art plainly won’t work in the real world, despite assurances to the contrary and claims of comprehensive testing.
Importance of a Common Research Framework
Since LTE-U equipment is not mature, it should come as no surprise that coexistence research leaves much to be desired as well. Statements that LTE-U is ‘more friendly to Wi-Fi than Wi-Fi is to itself’ necessarily rely on a baseline understanding of how Wi-Fi shares the spectrum with other Wi-Fi networks. But, in our three weeks at Qualcomm, engineers spent the majority of the time grappling with that crucial baseline information. The test setup at Qualcomm was uncontrolled and provided strangely imbalanced measurements. Afterwards, a CableLabs engineer replicated the setup at our Colorado facilities with the same make of Wi-Fi equipment, and within a half hour obtained balanced results, suggesting that problematic baseline measurements were somehow endemic to Qualcomm’s research environment. CableLabs and its members regularly test, configure, and operate Wi-Fi networks, and the behavior we observed on-site in San Diego seems quite out of the ordinary. Selected baseline measurements are shown in Figure 2 below, including the expected balanced baseline we observed in our Colorado lab.
This highlights the fundamental problem with the LTE-U coexistence research done to date: There is no common technical framework in which stakeholders are working, which makes it very difficult, if not impossible, to interpret research results across studies. This is apparent in our limited work with Qualcomm, and in Qualcomm’s prior studies, which also reflect baseline imbalances and call into question the research conclusions of LTE-U proponents.
Since we spent most of our time at Qualcomm working to diagnose apparent problems with the research environment, we did not come close to executing against the already modest test plan developed at the outset. We did however take some limited measurements of Wi-Fi behavior in the presence of LTE-U (which was tuned to 50% duty cycle ‘on’ time, so is only a narrow representation of possible real-world LTE-U configurations).
As seen in Figure 3 below, the impact of LTE-U depends on what your comparison point is: To conclude that LTE-U coexists better than Wi-Fi, one would need to lower the bar as much as possible — using an imbalanced baseline and referencing only the lower end in the analysis. This would clearly be a skewed approach, and even when doing so, it still doesn’t tell a conclusive story — reference the first case in the figure below, where the presence of LTE-U degrades Wi-Fi more than either baseline case. We would submit that the better approach is to diagnose the problems with the baseline measurements, rather than using unexplained results to justify definitive conclusions.
Furthermore, the in-home research detailed in a recent blog post by our principal architect, Jennifer Andreoli-Fang, made it clear that LTE-U is likely to have a disproportionately negative impact to Wi-Fi when the baseline is properly calibrated. An open standards process with common research methods is clearly needed to drive greater consistency and confidence in results.
We certainly hope that Qualcomm’s LTE-U solution will move from prototype to product in the near future, so that the Wi-Fi community can attempt to validate its coexistence efficacy in an open and comprehensive fashion. But that would only be one necessary step on the path toward equitable spectrum sharing. As we have detailed before, the LTE-U Forum coexistence specification leaves substantial room for different vendor and carrier approaches, which are likely to do disproportionate harm to Wi-Fi.
While this quite limited testing at Qualcomm’s facilities raised more questions than answers for us, CableLabs remains fully committed to rectifying the shortcomings in unlicensed LTE coexistence. Indeed, we have seen more hopeful progress in the development of the global standard form of the technology, LAA-LTE, which we are cautiously optimistic is on a path to coexist well. The more aggressive approach taken by LTE-U clearly poses significant challenges, but we see promise in the open standards process and the particular technical choices of LAA as the basis for a more effective and comprehensive solution.
Reliable coexistence in unlicensed spectrum requires a broadly supported agreement on specific solutions. That is why a collaborative, open standards development process is so important — that is how Wi-Fi is developed. Its success is self-evident in the marketplace.
Rob Alderfer is Vice President of Technology Policy at CableLabs.
Wi-Fi vs. Duty Cycled LTE-U: In-Home Testing Reveals Coexistence Challenges
Rob Alderfer, VP Technology Policy, CableLabs and Nadia Yoza-Mitsuishi, Wireless Architect Intern, CableLabs also contributed to this article.
In our last blog on Wi-Fi / LTE coexistence, we laid out the dangers attending the apparent decision of a few large carriers to go forward with the carrier scale deployment of a non-standard form of unlicensed LTE in shared spectrum. This time, we will review some of the testing conducted by CableLabs recently to explain why we are worried. We covered this material at a recent Wi-Fi Alliance workshop on LTE-U coexistence, along with the broader roadmap of research that we see as needed to get to solutions that are broadly supported.
Before we begin, let’s recap: there are different approaches to enabling LTE in unlicensed spectrum: LAA-LTE, and LTE-U. LAA is the version that the cellular industry standards body (3GPP) has been working on for the past year, and its coexistence measures appear to be on a path similar to what is used in Wi-Fi: “listen before talk,” or LBT. In contrast, LTE-U, the technology being developed for the US market, is taking a wholly different approach. LTE-U has not been submitted for consideration to a collaborative standards-setting body like 3GPP, instead it is being developed by a small group of companies through a closed process. LTE-U uses a carrier-controlled on/off switch that “duty cycles” the LTE signal. It turns on to transmit for some time determined by the wireless carrier, then it switches off (again, at the discretion of the carrier) to allow other technologies such as Wi-Fi the opportunity to access the channel.
CableLabs has tested duty-cycled LTE-U in our lab, our office environment, and most recently in a residential environment (test house) to research how technologies are experienced by consumers. Our research to date has raised significant concerns about the impact LTE-U will have on Wi-Fi services. Today we will review our most recent research, which was conducted in our test house.
How We Did Our In-Home Tests
Proponents of LTE-U claim that the technology will be ‘more friendly to Wi-Fi than Wi-Fi is to itself.’ So we decided to test that statement, using our test house to approximate a real-world environment, with technology that would comply with the LTE-U Forum Coexistence Specification v1.2.
(Note that just this week, the LTE-U Forum released a new version of their Coexistence Specification. We’re still looking at it along with the rest of the Wi-Fi community, since it is again a product of their closed process, but we don’t think it changes much for our purposes here. And judging by the discussion at this week’s Wi-Fi Alliance workshop, much work remains to get to reliable coexistence.)
To do our tests, we went to Best Buy and bought two identical off-the-shelf Wi-Fi APs, and we have a LTE signal generator that we can program to cycle the signal on and off – “duty cycling,” in the parlance of the LTE-U Forum.
We first established a baseline of how fair Wi-Fi is to itself. We had our off-the-shelf APs send full buffer downlink traffic (i.e., an AP to a Wi-Fi device), and took throughput, latency and packet loss measurements. We found that the two APs shared the spectrum about equally – throughput was roughly 50:50, for instance.
Next, we replaced one of the APs with LTE-U and repeated our tests, using various duty cycle configurations (LTE ‘on’ times). To make things simpler, we made sure that all of our measurements were done at the relatively strong signal level (-62 dBm) that is used by the LTE-U Forum as the level for detecting Wi-Fi.
What Our Research Found
If our Wi-Fi AP performed better in the presence of LTE-U, that would validate the claims made about it being a better neighbor.
But, we unfortunately did not find that. Quite the opposite, in fact. Our results showed that Wi-Fi performance suffered disproportionately in the presence of LTE-U. Wi-Fi throughput (“speed”) is degraded by 70% when LTE-U is on only 50% of the time, for instance. And more aggressive LTE-U configurations (longer ‘on’ times) would do even more damage, as seen in Figure 1.
Why does LTE-U do disproportionate damage to Wi-Fi? The primary reason is that it interrupts Wi-Fi mid-stream, instead of waiting its turn. This causes errors in Wi-Fi transmissions, ratcheting down its performance. We ran tests across a range of duty cycles to explore this effect. In our test case of two networks sharing with each other, to maintain Wi-Fi at 50% throughput, LTE-U could be on for no more than 35% of the time, as seen in Figure 2.
Does this mean that if LTE-U were to limit its duty cycle to 35% that Wi-Fi would perform acceptably? Unfortunately, it is not that simple. Our test case is admittedly limited: We are showing only two networks sharing the spectrum here, but in reality there can sometimes be hundreds of Wi-Fi APs within range of each other. If LTE-U took 35% of the airtime when sharing with 5 Wi-Fi networks, or 50 for that matter, the Wi-Fi experience would surely suffer significantly. And the problem gets worse if there are multiple LTE-U networks as well, which seems likely.
The effect that LTE-U would have across a range of real-world circumstances is, frankly, unknown. It has not been researched – by anyone. What we do know, based on our work here, is that the 50% LTE-U ‘on’ time considered by the LTE-U Forum when two networks are sharing (see Section 6.2.1 of their coexistence spec) does not yield proportionate throughput results.
That’s the effect of LTE-U on throughput, which is important. But equally important is latency, or the amount of delay on the network, which can have a dramatic impact on real-time applications like voice and video conferencing. We tested Wi-Fi latency across different duty cycles, and the results are seen in Figure 3 below. What’s important to note is that while the two Wi-Fi APs we tested can co-exist while providing smooth operation of real-time communications, that doesn’t appear to be the case if LTE-U is present. Even if LTE-U is only on 50% of the time, it degrades real-time Wi-Fi communications in a way that would likely be irritating to Wi-Fi users. And if LTE-U is on 70% of the time, then latency reaches levels where even latency-tolerant applications, like web page loading, are likely to become irritating.
So, we have seen in our research that LTE-U causes disproportionate harm to Wi-Fi, even when it is configured to share roughly 50:50 in time – which is not a given, as we have noted before. That’s because LTE-U interrupts Wi-Fi signals instead of waiting its turn through the listen-before-talk approach that Wi-Fi uses. We discussed the importance of using listen-before-talk in our last blog, and now you can see why we think it is important for LTE-U.
This research shows that LTE-U is not, in fact, a better neighbor to Wi-Fi than Wi-Fi is to itself, as its proponents claim. What should we make of these competing results? Is one claim right, and the other wrong? It’s not really that simple – the answer depends on how LTE-U is configured and deployed, and what coexistence features it actually adopts.
This highlights the need for the open and collaborative R&D that we have long been urging, so that we can find solutions that actually work for everyone. That has been happening with LAA, the 3GPP standard form of unlicensed LTE, where there is room for cautious optimism on its ability to coexist well with Wi-Fi. Hopefully LTE-U proponents will move toward actual collaboration.
Jennifer Andreoli-Fang is a Principal Architect in the Network Technologies group at CableLabs.
Rob Alderfer is VP Technology Policy and Nadia Yoza-Mitsuishi is Wireless Architect Intern, both at CableLabs, also contributed to this article.