How We Test VPNs: Methodology
We’ve got testing VPNs down to a science and want to tell you all about our methods. That way, you’ll know exactly how we can confidently rate and rank all the VPNs we look at. We start by taking a close look at all the different features a particular VPN offers (like encryption standards for security, IP addresses for privacy, etc.) to make sure it covers the bases of what a VPN should provide. Then, we put each VPN through our own speed and security tests, which we’ll go into detail about below!
Most of our VPNs get tested out of our Brooklyn office, where we have a private Optimum network; however, we occasionally test VPNs while traveling to the Philippines, as well. Our Internet speed without a VPN serves as an objective control, and we only connect one device at a time. We know our readers use all different devices, so we make sure to test on a Macbook Air and a Windows Vivobook. We use the website SpeedTest.net.4 to test download speed, upload speed and ping (latency). First, we perform tests with the VPN, and then, without. Download and upload speed are measured in megabits per second, while latency is measured in milliseconds. Once we gauge these two measures, we identify the difference in terms of a percentage to account for discrepancies between the greatly varied natural speeds of Macs and Windows computers.
Distance to the server, operating system and device all impact speed, but it’s ideal that VPNs have no more than a 40 percent difference in any of our categories, ping, download speed or upload speed.
A main motivator to use a VPN is to protect our web traffic, like our domain names (website names) and their IP addresses. It’s also important for us to know that users’ IP addresses aren’t being leaked due to WebRTC, which allows browsers to communicate directly with each other. It’s also the default on browsers like Chrome, Microsoft Edge and Opera, and since we’re Chrome-users, this was particularly relevant.
We test for DNS leaks by using DNSLeakTest.com. By comparing our computer’s IP address with the IP addresses that the website shows, we can tell if there was a DNS link while using the VPN or not. Then, we test for WebRTC leaks by leveraging a tool available on ExpressVPN’s website. We use a similar tactic where we look at the local and Public IPv4 IP addresses to see if there were any WebRTC leaks.
We also make sure to take a close look at companies’ privacy jurisdiction and data-logging policies. We take note of where a company is headquartered, because it dictates if they are members of Five Eyes, Nine Eyes and 14 Eyes. Those are international surveillance alliances that have the potential to legally force companies to surrender customer data. It’s the best-case scenario for a company to not be located in a country impacted by such agreements, but it’s also ideal if they log minimal user account information. We find things like name, email and payment information acceptable and don’t see why VPNs would log additional data, like when customers use their VPNs, how long they use it for, how much data they’ve transferred, what servers they use, what websites they visit and more.
We also conduct a full analysis of a VPN’s encryption methods and Internet protocols to make sure they meet the industry standards. You can find that information in our full, individual reviews. We prioritize VPNs with AES-256 encryption and OpenVPN, which are some of the highest standards of security on the market.
We check out how a company handles IP addresses because it impacts our likelihood of being tracked. It’s best if a company offers dynamic addresses that change each time we log on, making it even harder to track what we are doing. Static IP addresses stay the same and that makes them less desirable and easier to track. Finally, we make sure that each VPN has a kill switch. This function shuts down Internet browsers in the unlikely event that a VPN fails.
Torrenting and Netflix
VPNs are also used to watch movies and television. We make sure to keep track of which VPNs offer torrenting and have access to Netflix. The Netflix part can be a bit of a toss-up, as it is always updating its code to block torrenting, a big reason why VPNs are popular. Even though we test our ability to access Netflix, we can’t guarantee that it will always work for everyone.
Split Tunneling and Encryption
We also keep an eye out for split tunneling. This feature allows us to use both the VPN, as well as a public network at the same time. The lower bandwidth can create faster speeds. We prefer double or multi-hop VPNs, because this means that data is encrypted multiple times through multiple servers.
We boil subscription information down to the basics so that our readers can easily compare VPNs to each other. We also spend a lot of time looking at VPNs, so we’re able to spot good value when we see it. Some VPNs are super cost-effective, with monthly fees of less than a dollar if you sign on for a long-term commitment. On the flip side, certain companies offer subscriptions as expensive as $15 per month. We like it when a company offers several options in their pricing structure, like the ability to pay month-to-month or pay less monthly for a longer-term contract. We also look for the option to give a VPN a test run through a free trial or money-back guarantee period. Finally, we specify what actually comes in a subscription in terms of how many servers the company offers and how many devices can be connected simultaneously. Typically, a VPN will allow unlimited server switches and devices.
Once our readers choose a VPN, we want to make sure that they have ongoing support from the company. We check what resources are available in the event that issues pop up. It’s most common for companies to offer a live chat option and an online help center. Lots of companies will also compile a database of FAQs where you can search through past users’ questions. It’s uncommon for a company to offer a phone service, so if a VPN we’re reviewing has one, it’s a big plus.