IoT Testing Approach on Devices

Data has become pivotal to any business today. A huge amount of data is generated at various touch points through myriads of interconnected devices. Collecting this data is crucial; in this blog, we will discuss the Internet of Things (IoT), southbound–embedded devices, and gateways. These two parts of IoT are called southbound. The one from cloud and UI is called northbound.

Sensors and gateways fetch real-time data from multiple entities, including people, vehicles, machines, and drones. With the gateway failing to capture data, data is lost forever. So, robust gateway testing capabilities are paramount in south bound for the proper functioning of gateway.

The importance of gateway testing lies in ensuring that the data is not lost and the gateway devices do not go into a dead state. Also, it helps to ensure that all the possible exceptional cases accounting for the loss of the internet connectivity to the cloud and the data from sensor are considered.

We will go in-depth into the technicalities of gateway testing. Before that, let’s glimpse different types of sensors and the importance of gateways in the IoT.

What are different sensors used across industries?

Embedded devices (sensors) are different for different domains, as we see below:

• Healthcare: Health monitors from pressure, airflow, oxygen, pulse oximetry, temperature, and barcode sensing
• Agriculture: Soil temperature sensors, location sensors, optical sensors, electro-chemical sensors, mechanical sensors, dielectric soil moisture sensors and airflow sensors.
• Logistics: Temperature sensors, humidity sensors, infrared sensors, radar meters, road condition sensors, and visibility sensors.
• Energy: Smart grid, EB reader, current & voltage sensors, position sensors, wind speed sensors, absolute motion and direction sensors, non-contract harsh position sensors
• Mining: Intrusion detector, quality sensors, vibration sensors, temperature sensors, and magnetostrictive and inductive sensors.

Each embedded device is a data source, and there will be data either from a single source or from a large field. This data must be gathered and harnessed without any loss and sent to the cloud by Gateways devices. The information is then used in performing analysis. Hence, Gateways are the heart of the IoT system.

You might be interested in : IOT Testing Challenges and Approach

Gateway devices and their importance

Gateway devices are the central hubs for IoT devices, where data is collected at the edge device locally before sending it to the cloud. Gateways interconnect devices within the IoT. These are important bridges that connect the IoT sensor network and cloud. Reasons why gateway devices are key:

• The data from the source needs to be collected, i.e., without any loss.
• Irrespective of any kind of protocol or means of transport like wired or wireless.
• Data comes from a single source or multiple sources.
• Data can come from a closer location or a distant one.
• Data from a sensor, or another control unit will be collected.

How to test gateway in the IoT?

Some gateways require a lot of synchronization, scheduling, and buffering mechanisms, mostly when huge data is collected. On the other hand, a few require nothing but a simple microcontroller unit with a GSM connection to send data. What is important is how the gateway firmware is managed to run all the time without losing any data. Gateways are different for different domains. Here we will go into the details of gateway testing by understanding different scenarios.

The sole purpose of gateway testing in IoT is to address as many worst cases as possible. Let’s take the example of cellular tower testing.

In the event of AC power loss, the diesel engine source will give power to the tower so that data is not lost. The gateway will monitor the sensor data from the AC power source, which will be collected continuously. Whenever there is no data from the source of AC, the possible scenarios are:

• The sensor is faulty
• The sensor collecting data, lost its connectivity to the gateway
• There is an intrusion.
• The sensor is removed.
• There is no current passing through the sensor.

Let’s look at in detail:

• Faulty sensor: In case of sensor fault, the alert for that fault is raised from the gateway to the cloud. The scenarios that needs to looked for are, it is not supposed to turn on the diesel engine source or should wait for the fault to be resolved as it is a critical alert and run-in diesel engine till then.
• Gateway loses connectivity to the sensor: The controller is supposed to give exact information. The possibilities are that while the gateway was performing other data processing of high priority, it might have lost its regular listening, and so the gateway is supposed to try again to receive the latest data from the sensor. In such cases, there should be a business architecture that states the number of times the gateway can attempt to connect to the sensor and then declare that the communication to sensor is lost, following which the diesel engine turning on or waiting.
• Intrusion: Intrusion occurs when it shows the diesel engine running, but in reality, it’s AC source. These activities take place at a lower field level to steal diesel. Testing becomes a high priority in such a case and is essential to prevent manual turning on of diesel engines. With a good testing procedure, the system sends a notification to the authorized person about possible manual intrusion and prevents unauthorized access.
• Sensor removal: The sensor should be removed/replaced only by an authorized person, and the sensor must have a similar configuration as validated by the business. So, those things should be allowed only for limited users, and the information must be recorded in reports.
• Other field cases: According to the climatic conditions, the sensor’s data collection might differ, which might require configuring accordingly. And there are certain temperature conditions where the sensor will enter a sleep state, maybe due to very cold or hot conditions. So, testing must consider these possibilities prior to designing or customer acceptance planning. The environment in which the sensor will be placed must be considered.

The above-described scenarios are mostly functional and related to data validation. So, the testing of the above involves functional validation and verification. There will be many test cases here in which a suitable automated tool to simulate all kinds of sensor data can be used to ensure smooth functionality.

Testing Gateway for IoT: Some facts to consider

Lately, the testing of IoT is test-driven on live data. There are specific tools used for simulating data as expected from the field and verifying the results and effects the data can cause to the existing system. There might be legacy systems where IoT sensor integration would have been done for value addition. For such cases, these tools shall help for testing and analyzing the results without disturbing the legacy system and live data. These are similar to a replica of the field environment which are also helpful for OEM.

In any case of data loss, the gateway must be intelligent enough to detect the exact reason and raise an alert accordingly. When thousands of data sources are there, the data handling capacity of the system can be ascertained by testing. Testing will be crucial in getting an idea about the maximum number of sensors the gateway can handle.

Apart from regular testing of sending message notifications from the SIM on the device, there are cases where the field person might install a SIM, which also allows for incoming data messages. The SIM will be receiving incoming messages, and the memory will be loaded on the SIM; and hence the actual functionality to send the data notification might not be possible. Such cases require boundary analysis and exploratory testing.

When a gateway has been running for years without any problems and stops working due to the Wi-Fi process, it reboots itself. For such single process cases, the gateway should not restart itself, but the process which stopped alone shall restart softly and ensure that there is a seamless connection. Testing in such cases shall add value to the performance of the gateway.

There are performance conditions which shall lead to gateway crash, like when there is a huge request of data from the cloud, and in case the process fails to manage and send the full information as requested, those things can be simulated by tools and verified. Generally, the gateway functionality is tested by shell scripts and automating with the same.

With respect to configuring the firmware, what is important for the verification team is to ensure that only a valid configuration set is allowed while configuring and proper detailed guidance or information is given to complete the same. Here regression testing helps to verify all sets of configurations. Testing helps verify that the same configuration set has been configured on the cloud. So, there is smooth data transmission from the gateway to the cloud. These configuration sets are those which will help for the segregation of scheduled and alarm data from data sources. The historical data will help to build an AI-based predictive maintenance process. These are the actual IoT testing services which are of business value.

The value gateway testing brings to business

The testing of a gateway is like an endless ocean. But the key is approaching the test by categorizing its requirement according to the domain. However, the main tests are performed to give reports on failure (gateway down). As many exceptions and alerts due to gateway failure are known, it is required for us to reach the device remotely to fix the error. If the gateway runs on the filed for longer period, exceptions will keep adding as seen in the firmware updates.

Finally, what are the uses of reports and insights graphics if the data is not valid? Does invalid or lower performance data make sense to business? Instead of adding value, would it lead to a loss? Testing helps address all these questions. So, gateway in IoT requires robust testing.