Measuring and collecting Wi-Fi performance metrics

This section of the report details how Wi-Fi performance metrics can be collected and analysed using various tools, platforms, and software, such as Wi-Fi scanning tools, network management systems, Wi-Fi surveys, network traffic analysis, and Wi-Fi controllers.

By using a combination of these tools, organisations can gather a more comprehensive and accurate set of Wi-Fi performance metrics, enabling them to identify potential issues, optimise network settings, and improve the overall user experience.

These metrics can be used to establish a baseline for stability and efficiency of the various components and activities of a wireless network. By understanding and correctly measuring these metrics, IT teams can quickly detect and resolve wireless issues.

In this report, automated monitoring systems are defined as specialised solutions developed by vendors. The report aims to describe the purpose and differences of available automated monitoring systems. Most of the systems discussed may already exist within an organisation's Wi-Fi network and can be used to obtain a viewpoint of the performance of the network without additional investment or implementation.

All automated monitoring systems collect Wi-Fi metrics from the perspective of the network rather than the client (user) and may not reflect the actual user experience.

A network management system's (NMS) primary function is to monitor the operational status or uptime of your network hardware using protocols such as Simple Network Management Protocol (SNMP) or Ping, but they can also collect Wi-Fi metrics automatically from Wi-Fi capable devices when those devices are polled.

An NMS can collect and display all relevant Wi-Fi performance metrics, if both the NMS vendor and the Wi-Fi hardware support it and are properly configured.

This means the AP must be set up to scan the RF at regular intervals, either through standard configuration or by enabling its spectrum analysis radio. It then reports the RF status back to the NMS.

NMS can provide deep insights into Wi-Fi network performance by also providing historical analytics to better understand when or where Wi-Fi networks are not performing optimally.

All NMS should be capable of addressing the following:

• customisable alerts: receive automated alerts on multiple channels, such as email, SMS, and push notifications
• real-time notifications: provide real-time notifications to quickly detect, diagnose any poor performing Wi-Fi networks
• threshold-based alerts: allow users to establish predefined thresholds and triggers to promptly receive alerts when specific Wi-Fi performance metrics are not being met
• intelligent alerting: use intelligent alerting to recommend remediation or optimisation strategies

This report recommends use of NMS platforms provided by existing vendors who already provide the Wi-Fi hardware solutions at NHS settings, if possible. The benefit of this is so you maintain familiarity with the vendor and that their NMS platforms are designed to integrate with their network infrastructure products which may give additional features and benefits such as ease of deployment, cost and complexity to maintain.

Benefits of using NMS for collecting and measuring Wi-Fi metrics:

• visibility of all managed network devices and clients
• can be managed in-house by IT/network team
• 24/7 real-time status and alerting of Wi-Fi metrics
• gives detailed metric performance of Wi-Fi infrastructure

Considerations of using NMS:

• may require a dedicated NMS per organisation or site (dependant on network design and security policy)
• can be expensive to license, maintain or outsource
• some NMS solutions provide greater functionality than others. Ensure the NMS system chosen can capture all the data performance metrics required
• does not give any network performance perspective from client devices

A Wi-Fi controller's main function is to manage large scale AP and client connections, but if an organisation manages their Wi-Fi network in-house, they can access controllers’ dashboards to record, extract or observe real-time Wi-Fi performance metrics. Wi-Fi controllers can be either an on-premises appliance or cloud based. 

For example, it may be observed that an AP radio is powered too high due to radio resource management configuration resulting in many users experiencing poor performance. Further investigation of that coverage area may result in excessive co-channel interference between APs on the same channel, therefore reducing the AP transmit power or changing assigned channel may improve client performance.

A controller’s main function is to maintain connectivity, configuration and operation of AP and Wi-Fi clients, acting as the brains of the Wi-Fi network but they can also be a source of statistics and performance metrics due to their direct access to AP logs. This allows for the integration of AI or automation-based programs to be used to analyse data in several ways, such as:

• data collection: can continually build intelligence over time through data collection and analysis by collecting data from every device in real time
• Intent based services: can react to problems or enable faster fixes to network issues, by applying configuration changes to improving performance
• contextual services: can be used to provide contextual services, such as expedited troubleshooting and unprecedented visibility into the user experience. This is particularly important as Wi-Fi becomes increasingly business-critical and harder than ever to troubleshoot due to the number of mobile device types, applications, and operating systems
• Massive data collection: collect massive amounts of network data, which is then analysed to identify trends, foresee future events, and recommend changes to improve system performance and user experience.

Benefits of using Wi-Fi controllers:

• already available within the design
• 24/7 remote access to data
• maintain historical data over long periods of time
• integration with some AI programs for customised metric reporting

Considerations of using Wi-Fi controllers:

• limited dashboard features
• lack of visual data representations
• hard to interpret data
• AI and automation integration can be complex

In the context of this report, Wi-Fi sensors are dedicated devices that collect real-time data on the performance of a Wi-Fi network. They can collect real time Wi-Fi performance metrics, however some vendors provide greater functionality over others so it is important to choose based upon your requirements.

If an organisation has surplus Wi-Fi APs in stock they can deploy them in monitor mode, if supported, which is equivalent to an enterprise sensor. In this deployment type, the AP will monitor the RF spectrum, collecting data on how well the local Wi-Fi network is performing. This deployment means the AP is fixed in one position and does not replicate the mobile behaviour of a client moving around an area.

Non-enterprise grade sensors are also available, but may lack features such as pure spectrum analysis, therefore they are only capable of monitoring and capturing Wi-Fi frames and not potential non-Wi-Fi interferers.

Organisations are advised to engage vendors specialising in integrating sensors into their networks. This is crucial to ensure the integration maintains data security and complies with IT policy. While a vendor sensor is implemented, it can be maintained and supported by in-house IT teams which ensures that the integration aligns with the organisation's security and compliance requirements.

For the purpose of this report, a non-automated process refers to collecting Wi-Fi performance metrics by manually collecting Wi-Fi data using handheld tools.

An on-site wireless survey is the most common method for measuring Wi-Fi performance. It aims to collect data from a user's perspective which can provide greater insight into how a Wi-Fi network is performing.

The correct enterprise tools should be used to collect the data. It is advisable for organisations to incorporate their chosen tool as their standard choice. This ensures consistency in measurements and reporting, allowing for easy interpretation of visual data and familiarity with the reporting format.

Wireless site surveys involve a site visit by a qualified engineer to capture Wi-Fi signals and spectrum data using a specific tool as well as observe AP positions, and other environmental factors that may affect RF propagation. A report is produced showing the current status of the Wi-Fi network and if required, remediation or recommendations for improved performance.

There are 3 types of wireless survey available:

• passive
• active
• AP-on-a-Stick (APOS)

Each survey type can be conducted individually or simultaneously depending on the specific requirements of the survey.

Passive site survey

A passive site survey is the most common method of measuring a Wi-Fi network. During this survey, an engineer does not connect to the network but instead passively scans the 2.4GHz, 5GHz, and 6GHz unlicensed spectrum frequencies to capture data.

Passive surveys will collect the following metrics accurately:

• RSSI
• Secondary RSSI
• SNR
• CCI
• channel utilisation
• spectrum utilisation
• spectrum analysis
• packet loss

Passive surveys do not collect the following accurately:

• latency
• jitter
• capacity

The engineer walks the entire coverage areas which could be a site, building, or floor collecting Wi-Fi data using a monitoring tool. The data collected from a passive site survey can be compiled into a report, enabling an organisation to determine the next steps.

The data set collected during a passive survey is considered a snapshot of a moment in time of the Wi-Fi network. This means that should any network changes occur after a passive survey has been completed, data on these change will not be captured or reflected, therefore, another passive survey may need to be conducted.

A passive survey is recommended as the initial step to measure Wi-Fi performance metrics when required, as it provides a more thorough way than automated monitoring. However, contracting an external supplier to carry out frequent surveys may be cost prohibitive, it is not always viable to regularly conduct this type of survey and using data from automated monitoring is the next best thing.

Passive site surveys should always be conducted by certified individuals, whether internal or outsourced to external companies, that demonstrate a full understanding of site survey requirements.

Use of professional survey tools by internal staff can potentially reduce survey costs in the medium to long term but may have expensive upfront costs. These tools not only include purchase and maintenance of proprietary software and radio adapters but may also require the purchase of additional devices, such as tablets or laptops, to run the software.

Active site surveys

An active site survey is like a passive survey but rather than passively scanning, an engineer will actively connect to the Wi-Fi network using a Wi-Fi survey tool to obtain a subset of Wi-Fi performance metrics.

Active surveys are best suited for measuring the following metrics:

• RSSI
• MCS/throughput
• latency/jitter
• SNR

The most effective use for an active survey is to measure the throughput and latency performance of a Wi-Fi network, particularly in settings such as VoIP or RTLS deployments. During an active survey, the survey tool can conduct an iPerf test. iPerf is a widely used open-source program for measuring throughput, latency, and jitter between 2 systems. The metrics can be evaluated to determine if they are good, fair, or poor.

To conduct an active survey, an engineer will walk to a position in a coverage area they wish to measure and stand still while they run the active survey. An active survey can measure both upstream and downstream and records measurement of download and uploads throughput speeds.

AP-on-a-stick survey

A specialised form of wireless site survey, an AP-on-a-Stick (APOS) survey entails placing an AP at a designated location and conducting a survey in its vicinity. This method is particularly valuable for validating and optimising Wi-Fi network designs. APOS surveys are the most time-consuming. This is attributed to the need for engineers to continuously relocate the APOS to each proposed AP position, survey the surrounding area, make any necessary configuration adjustments to transmit power, and then repeat the survey.

More than one engineer may be required to conduct the survey in busy areas - one to walk and survey, and another to remain by the APOS to ensure its stability and prevent any inadvertent movement or disruption.

Conducting a wireless site survey can be time consuming and expensive, as it may require the surveyor to cover every area of the building or site where Wi-Fi service must be available.

Organisations should consider conducting a site survey under the following scenarios:

• new Wi-Fi design is required for a greenfield site
• environmental changes, such as new floor layout
• hardware upgrade/refresh or relocation of existing Wi-Fi network
• validate/optimise the design deployment of a new Wi-Fi network
• validate/optimise configuration or design changes to an existing Wi-Fi network
• when technology changes occur, such as refresh of client devices
• general assessment of the status, health, or performance of an existing Wi-Fi network
• troubleshooting problematic areas as and when required

This report recommends that organisations consider evaluating or surveying their whole Wi-Fi networks at a minimum of every 3 years.

Organisations should also identify their most critical settings where Wi-Fi performance is paramount. A bi-yearly passive survey should be carried out on those Wi-Fi networks as a general health check, focusing on ensuring that those networks meet targeted metrics.

This is because factors within those areas may change, such as new RF interferers, changes in user behaviour, or changes to the physical buildings, which may impact Wi-Fi performance. This may raise issues not previously captured.

Benefits of a wireless survey:

• wireless surveys provide insights into the performance of an existing wireless network or aids in the design of a new one
• analysing signal strength, a site survey helps optimise the placement and configuration of APs to ensure a strong and consistent signal
• site surveys help identify sources of interference, such as other Wi-Fi networks, electronic devices, or physical obstructions
• understanding the number of devices and users in each area allows for effective capacity planning. 
• active surveys during the site survey process involve testing the network's performance by measuring data rates, latency, and other key performance indicators
• produces granular detail on performance metrics

Considerations of a wireless survey:

• may be expensive to conduct
• surveyors require access to all areas of buildings which may be restricted
• time consuming
• higher skill level of a Wi-Fi surveyor is required to conduct onsite surveys
• only provides a snapshot of Wi-Fi network status

Wi-Fi scanning tools are user-friendly and cost-effective software applications used to scan the Wi-Fi environment and collect data on signal strength. These tools can be used by IT staff to spot-check Wi-Fi network performance, determining if further investigation, a survey, or analysis is necessary. Examples of commonly deployed Wi-Fi scanning tools are Wi-Fi Explorer Pro, Acrylic and NetSpot.

Wi-Fi scanning tools typically run on devices such as  laptops or phones, passively scanning RF and capturing Wi-Fi management frames to display metrics such as RSSI, SNR, and channel utilisation. They offer a quick and easy way to observe the Wi-Fi network, providing valuable insights as a real time snapshot into the network's performance.

Additionally, Wi-Fi scanning tools can be used to improve network proactivity, and combat security vulnerabilities such as detecting if network authentication is enabled for that SSID or if it is using legacy encryption methods such as WEP and WPA.

Wi-Fi scanning tools are limited in their capability and cannot provide information on the following RF metrics:

• latency
• jtter
• spectrum utilisation
• spectrum analysis

To effectively use a Wi-Fi scanning tool, much like survey tools, an engineer must operate locally onsite within the coverage area you wish to measure.

Benefits of Wi-Fi scanning tools:

• inexpensive to procure
• Easy and simple to use
• can give good visual comprehensive data on laptops or phones for portability

Considerations of using Wi-Fi scanning tools:

• limited in their capabilities compared to enterprise Wi-Fi survey tools
• only provides a snapshot of local Wi-Fi network
• limited to the radio quality of the phone or laptop
• no reporting capabilities

Wi-Fi analyser tools are dedicated software applications primarily used for troubleshooting Wi-Fi networks. Their purpose is to capture and decode network traffic transmitted over the RF. While they are regarded as valuable tools for identifying Wi-Fi issues, they are not recommended for measuring Wi-Fi performance metrics. Examples of commonly deployed Wi-Fi analyser tools include: Wireshark, Omnipeek and Ekahau.

Wi-Fi analyser tools should be used as a last resort when all Wi-Fi performance metrics have been measured and optimised, yet users still report issues with their devices on their networks. These tools can help pinpoint potential network issues by analysing data frames, although this type of analysis happens infrequently, mostly during troubleshooting.

One example could be high latency detected when measuring Wi-Fi in a coverage area. However RSSI, CCI and other performance metrics are within acceptable targets, therefore latency issues may reside between the client and application server on the local area network (LAN). Wi-Fi analysers can capture the traffic over the air and record the delta between frames to determine if the latency exists on the LAN not the Wi-Fi network.

Wi-Fi analysis requires experienced engineers to read the captures using relatively expensive hardware or software tools. This is because knowing where to capture data, either closer to the client or closer to the AP and knowing how to read and decode captured frames requires a high-level skill set. If this is required organisations should engage with suitable companies to conduct Wi-Fi analysis.

Spectrum analysers are dedicated tools that have specialised hardware radios that scan a radio frequency range detecting the amplitude of raw signals heard. They are able to locate sources of interference that are non-Wi-Fi.

Spectrum analysis is integrated into the Wi-Fi survey tool set provided by some products, however spectrum analysis can also exist as independent standalone tools, with some vendors offering dedicated spectrum analysers that combine Wi-Fi scanning functionality into their products.

The engineer is required to interpret the resolution bandwidth visualised by the device, to determine whether the type of raw signal observed is coming from a genuine Wi-Fi or non-Wi-Fi interference source.