Assessing build quality and battery life
Update: We've overhauled how we test laptops with 3DMark Skydiver and Middle Earth: Shadow of Mordor
TechRadar believes evaluating laptops based on their value proposition. As such, we review mobile computing devices, including Windows tablets, convertibles and laptops of all sizes against their own price and other options available on the market. Our review process is also underpinned by rigorous tests to determine the quality, performance and battery life of each machine.
We begin by examining a laptop's design and build quality to see how solid it feels to the touch and whether it fulfils the role it was designed for.
The overall build quality is important, so we go to great lengths to test the overall strength and durability of each system. We also assess the functionality of all ports, switches and latches. The quality of the screen is considered, with checks for brightness, evenness of tone, as well as any dead pixels identified.
The final part of our initial tests deals with the weight of the machine and its relative portability. Next, we assess the overall usability of the machine, including the quality of the keyboard, touchpad and overall user interface.
Updating and optimizing
Before we begin testing every laptop by updating the system with the latest patches, firmware updates and drivers. No device stays frozen in time and while this means benchmarking numbers are a constantly moving - and often, rising - target, it's a part of our technological progress.
To give every machine the benefit of the doubt, we also turn on "high performance mode" before testing. This ensures the integrated graphics as well as any other components inside the laptop are operating at their maximum performance. Similarly, we switch laptops to "balanced mode" in the Windows power options before battery tests to ensure they don't run out of juice prematurely in a setting that isn't intended for use on battery.
As every laptop is tested using the same suite of benchmark tests. Once its performance has been measured, the device can easily be compared against its peers. Each review is accompanied by the test results for that machine, as well as its closest competitors.
Before the hands-on part of our testing has been dealt with, the laptop will spend up to 72 hours being run through a series of benchmarks to check overall performance. Each machine is set at the same high performance level. This way, we can judge how effectively it will run at its maximum potential.
We use a number of synthetic tests to measure a laptop's components. The first, PCMark 8 battery life, tests the device's battery endurance, which we then follow up with an anecdotal test with real life usage. Meanwhile, 3DMark is specifically designed to test the strength of the laptop's graphics processor(s).
We then evaluate the CPU's multi-core performance through Cinebench. Finally, if we're reviewing a gaming laptop or desktop, we use benchmark tools found within Middle Earth: Shadow of Mordor and Metro: Last Light, to truly tax those dedicated graphics chips.
PCMark 8 battery life
This software tests mobile performance and battery life, simulating popular general tasks such as video chat, web browsing and document creation while the system is unplugged.
Firstly, all laptops are set to balanced in the Windows power options. We also tweak some advanced settings including telling the screen and hard drives to never sleep, we set the critical battery level to 5%, turn off Bluetooth and all other radios (with the exception of Wi-Fi), disable the keyboard backlight if applicable and tone down the screen brightness to a pragmatic level.
With the laptop's battery fully charged, we disconnect the main power. PCMark 8 then simulates day-to-day use until the battery runs close to dry. Once it's through, PCMark 8 provides an estimate of the battery's total capacity in hours and minutes based on how quickly power drained during the test.
PCMark 8 battery life scores
- 2 hours: This either isn't a very power-efficient machine, or wasn't designed for endurance.
- 3 hours: Generally Ultrabooks come in at around this time, as well as low-resolution laptops.
- 4 hours: Only the longest-lasting laptops can achieve this level of endurance or longer.
In addition to using PCMark8 to create a synthetic measurement of battery life, we also test how long laptops can last through a regular day of use. In this real life test, we use the laptop throughout the day working on documents and photos, chatting with colleagues over the web, watching videos and other general tasks users would perform in the course of a day.
We run the laptop from a full charge until the machine ultimately shuts down after the battery is fully exhausted. Along with marking how long the battery life was, we also list exactly what applications we ran during the test as well as noting screen brightness and volume level.
3DMark and gaming benchmarks
3DMark represents our default GPU benchmark, as it is capable of running on high-end dedicated GPUs, as well as entry-level integrated solutions.
With each laptop optimized for peak performance, we then run the three 3DMark tests: Cloud Gate, Sky Diver and Fire Strike. Unlike PCMark 8, this benchmark is run while connected to a power source to give a reflection of how you are most likely to stress the GPU when gaming.
3D Mark scores
- Cloud Gate: Intended to test slightly more capable machines, this test includes physics rendering and other intense tasks. Scores above 10,000 show an ability to play more 3D-intensive games at lower resolutions.
- Sky Diver: Designed as a middle ground benchmark to sit between Cloud Gate and Fire Strike, Sky Diver is a DirectX 11 benchmark for gaming laptops and mid-range machines. We've seen most laptops and even Windows tablets run this test without a hitch, however, marks of 5,000 and higher represent machines with the capacity to play games on low to medium settings and most graphics intensive applications.
- Fire Strike: Only the most powerful mobile GPUs can score in the thousands on this test. Scores of 3,000 or better translate to an ability to play the latest and greatest 3D without much issue.
This test is designed to isolate the CPU and measure its multi-core performance. The benchmark does this in two ways. First, it measures hyperthreading performance – or how well each processor core works in tandem with the others in a single CPU – through an image rendering test. Scores past 600 points generally point to quite a capable quad-core CPU, while anything less is generally indicative of one with just two cores.
The second test measures the graphics rendering capabilities within the CPU. This is done through an OpenGL graphics rendering test performed in real time. Results are reported in frames rendered per second, or fps. Anything above 20 fps is considered satisfactory, whereas anything above 60 fps indicates a more than capable component.
Middle Earth: Shadow of Mordor Infinite and Metro: Last Light both include benchmarking tools to measure a laptop for desktop's gaming performance. We run both real-time renderings three times, at two graphical quality settings, and record the average frame rates.
For both Middle Earth: Shadow of Mordor and Metro: Last Light, we run the tests three times at the lowest possible detail settings at 1080p or the device's native resolution. After recording the average frame rate at those settings, we then run the benchmark another three times at the highest possible settings and native resolution, and record that average.
Being a less intense game graphically, Middle Earth: Shadow of Mordor rates above 45 fps at the highest possible settings indicate a capable GPU. Metro: Last Light, on the other hand, is far more punishing. Few gaming laptops are able to pass this benchmark at the highest possible settings with a playable frame rate, so anything close to 30 fps should be considered impressive.