IoT Data Analytics – Cross Industry Perspectives

Analyzing IoT data across various industries provides valuable insights into operational efficiency, user behavior, and predictive maintenance strategies. By integrating IoT devices and leveraging data analytics, companies can gain real-time visibility into their processes, enabling proactive decision-making and improved resource allocation. Cross-industry perspectives on IoT data analytics emphasize the importance of data security, scalability, and interoperability to drive innovation and competitiveness. With a focus on extracting actionable insights from IoT-generated data, organizations can optimize performance, enhance customer experiences, and drive business growth across different sectors.

Content

Internet of Things (IoT) brings in a whole new world of possibilities with data from Smart Devices, Application Logs and anything connected to provide a means for analyzing data at the most granular lever, detect patterns, predict outcomes, build systems that are automated, self-learning, self-managed, intelligent and derive deep and valuable insights that make the world Smart and efficient. IOT Data Analytics deals with collection, organization, analysis of a vast collection of time variant data and derive insights through AI techniques and make the world a better place. At Coforge, we specialize in Streaming Data processing and analysis, Statistical analysis Deep data mining using Machine learning, Deep Learning and Cognitive AI techniques to deliver industry leading solutions and services for IOT Data Analytics.

Internet of Things (IoT) broadly refers to a collection of inter-connected devices that are enabled with Sensors to capture a pre-defined set of variables in their eco system, continuously over time and share that data across their network.

A common example, most popular in current times is the smart watch that can detect heart rate, pulse rate and other parameters of the person wearing this on a continuous every second basis and helps in tracking the vital health parameters during exercise, sleep and regular daily activities. Likewise, there are 100s of types of sensors for tracking various parameters, in industries, mines, commercial establishments, vehicles, flights, ships, transportation systems, medical equipment, Utility systems for Electricity/ Gas /Water distribution, sports equipment, weather monitoring systems, wind turbines, smart cities, smart homes and every possible area where a smart device can be embedded.

IoT Data Analytics deals with collection, processing, enriching, analyzing and deriving meaningful and valuable business insights from the data through specialized technology, tools and methods. This paper covers various considerations for technology and implementation of IoT Data Analytics.

Industry Trends and Evolution of IoT Data Analytics

The number of IoT devices in use worldwide is forecasted to almost triple from 8.4 billion devices in 2020 to 25.4 billion devices in 2030 (Source: statista.com). IoT Analytics market has been valued at USD 17.14 Billion in 2020 and expected to grow to USD 81.67 Billion at a CAGR of 29.8% through 2021-2026 (source: mordorintelligence.com). IoT devices are widely in use across all industries and across large variety of use cases for tracking and monitoring, efficiency, product and process quality, cost reduction, predictive maintenance, performance improvement, efficiency, Security, improving customer experience, Environment health and safety.

Sensors and monitoring devices have evolved immensely in the last 3 to 5 years. Some of the important evolution and innovation with the Sensors include:

• Transforming from analog to completely digital devices
• Increasing ability to capture data seamlessly and store data temporarily
• Transforming from wired devices to wireless (Wifi enabled) devices
• High-speed, Real-time capture and streaming of data (even at microsecond latency)
• Drastic reduction in size of the sensors with Micro IC chips and upcoming Nano chips
• Advancement of controllers and monitoring software tools
• Emergence of Industrial IOT (IIOT) with Industry 4.0
• Advancement in medical electronics, robotics aided medical procedures
• Common IOT sensors with usage in Smart homes, personal health, Vehicles, Logistics
• Very low cost and affordable (some IOT sensors are lesser than $1, and most sensors are under $10)
• Evolution of Industry leading IOT platforms
• Wider adoption of IOT Data analytics Software tools and Custom applications for IOT analytics in enterprises

Types of IOT Sensors and Data generated

In a nutshell, IoT devices essentially capture data from sensors and device logs that capture details of Time, location physical parameters (temperature, pressure, density, etc.), mechanical parameters (RPM, distance, speed, force, etc.), weather parameters, State or condition of an object (E.g. On/Off, open/closed), Flow meters (for gases, liquids), data from medical electronic devices and few others. The following table captures a representative collection of such sensors:

IoT Data Analytics across Industries

In 2018, International Data Corporation (IDC) had estimated worldwide spending in Internet of Things to grow to $1.2 Trillion in the year 2022. With the Pandemic situation, there has been a slowdown in Spend in 2020 which IDC forecasts to be at $746 Billion and achieve a CAGR rate of 11.3% through 2021-2024. Internet of Things is the way forward for many verticals with a wide variety of Use cases and applications to derive commercial value, innovation and improve quality of life. Use cases across Industries can be synthesized into a set of Business analytics focus areas as depicted below:

Building the Enterprise IoT Data Analytics Platform itself is a major Business Use case and forms the foundation for enabling various IoT analytics themes and business cases. This section provides an overview of various Business use case scenarios for IoT Data Analytics across verticals, and this is only a small representative list of examples while the potential of IoT Analytics is continuously evolving with new products, startups, research organizations emerging and bringing innovative ideas to real life.

1. Manufacturing (Discrete manufacturing, process manufacturing)

• Production process efficiency and quality analysis
• Product quality enhancement and Product innovation
• Modeling for most best operating conditions (threshold ranges for variables) to maximize production
• Predictive failure of machinery or components and failure prevention
• Safely extending usable life of equipment and cost reduction
• Predictive maintenance of machinery components and minimizing downtime
• Improving safety, Compliance to EHS standards and hazard prevention
• Efficient Inventory tracking, loss/theft prevention and supply chain management
• Minimize wastage, Optimizing usage of labor, raw materials, spare parts
• Enhance efficiencies in reducing emission, pollution, carbon footprint,
• Improving after Sale services, warranty and customer satisfaction

2. Healthcare

The healthcare industry has been a frontrunner in the experimentation and adoption of sensors for diagnostics, treatment, monitoring, surgical procedures, and overall health management. There are a wide variety of proven use cases across all these areas in healthcare, some of the key areas represented here:

• Remote patient monitoring through Sensor real time data and monitoring using connected medical devices
• Internet of Medical Things (IoMT) is a fast-evolving area with various sensor based medical devices or wearables help in capturing vital parameters of patients and in diagnostics, personalized treatment procedures
• Advanced models for analytics of patient data and assisting in disease analysis and derive treatment plan
• Personalized Drug management for patients based on the continuous data collected and altering the specific drugs and dosage levels based on changing health conditions
• Enhanced Patient engagement with collaborative effort, commitment from patients in the treatment process
• Tracking and management of assets and medical equipment
• Enable advancements in medical Research by combining IoT Data along with EMR/EHR and other diagnostic data and deriving insights though advanced analytics

3. Insurance

Insurance industry has been experimenting and some leading enterprises have implemented business case using Telematics and IoT data Analytics mainly for Dynamic pricing of policies, preventing fraudulent claims and providing proactive and preventive services like improving personal health. Few key examples are:

• Usage based personalized Insurance using Telematics data analytics
• Innovative pricing model for Car insurance with attractive Policy premiums based on Pay per mile
• Provide personalized Health and Life insurance based on personal health tracking data
• Prevent Fire accidents and resulting claims using Connected sensors in Homes and commercial establishments
• Analyzing claims data and preventing fraudulent claims in agriculture and farming using sensor data and data from drones
• Optimizing premiums by analyzing sensor data collected over a period and defining innovative pricing models for Insurance for Plant and machinery, Heavy equipment, Aircrafts, Ships and other transport systems

4. Retail

Retail industry, especially the large retailers, have adopted sensor-based technologies to improve supply chain and inventory management and enhance customer experience. While most of the IoT usages are in the in-store scenarios, for online retail as well IoT is especially useful in managing warehouse and distribution analytics and optimization. Common use cases in the Retail Industry are:

• Inventory tracking and management using data from machine readable tags, RFIDs, smart shelves.
• Improving Supply chain efficiency with automated tracking of shipments, vehicles and distribution networks
• Providing an intuitive in-store customer experience through insights delivered on mobile apps for store views, search, recommendations, faster billing, and delivery
• Capture data on footfall to the store, visitors for each store sections and analyze impact on the Sales

Key Technologies and Enablers for IoT Analytics

With millions of devices emitting data every second or milli-second and continuously, IoT Data will involve a combination of high volume, high velocity and variety of data along with moderate to high complexity of the data for acquisition, parsing, processing, Storage, analysis and deriving insights. Technologies that are essential for IoT Data Analytics include:

Considerations for IoT Data processing

Raw IoT Data streams essentially comprise of the data items - Device ID/Sensor ID, Timestamp, Sensor value. IoT System controllers (micro controllers) or Control systems like SCADA or PLC will collect the data and put together data from multiple sensors into a data stream. Based on the type of IoT devices the raw data collected may be captured as binary or a Hexadecimal data stream or as ASCII text data. The control systems in most cases will have the capability to parse the data and convert this into an ASCII data stream. Each record in the data stream will have a specific timestamp and provide values for each of the senor parameters observed at that time. This data can be collected and processed in real-time or in micro batches (with a frequency of every minute, 5 minutes, or every hour, etc.) depending on the latency requirements for the Data Analytics use cases. Processing of IoT Data may involve several steps that are executed in real-time or in micro batches and these can include:

• Data acquisition from sensors directly or from IoT system controllers using a messaging service or APIs to support real time stream data capture
• Parsing of data to breakdown complex columns into individual values (e.g.: a byte stream of binary 8 bits may indicate on/off state of 8 different switches or valves) and convert them into a delimited data set like a CSV
• Perform data quality checks and corrections as may be necessary
• Processing the raw data and loading to a raw data layer (a file system or relational DB or NoSQL data store
• Associate the variables captured in the raw data to the related entities involved in the business-like People, Processes, Plant/machinery, Location, Date/Time, Products and every entity in the ecosystem that can be related to the IoT Data to provide a comprehensive view for analysis
• Create and update the metadata about the sensor variables and the values collected from the data streams
• Perform window functions like Min/Max/Median values in a window of 1 minute or 5 minutes
• Calculate statistics in real time – averages, standard deviation, variances, peak/trough values, correlations, frequency distributions
• Perform real-time aggregations of data for different time windows (data stream received at milli-second latency is aggregated to provide values at 100 or 200 milli-second level aggregation)
• Perform aggregations of Process event-based grouping of data (e.g. if a process involved 10 steps with each step running anywhere from a minute to a few hours, to calculate the aggregate values of the variables within each step like max temperature observed in step 1, total power consumed in execution of step2)
• Identify any critical threshold breaches identified in real time streaming data (e.g. temperature reading closer to the Max threshold)
• Split the data streams to multiple data sets to provide different views of data for Reporting and analysis
• Create analytical data sets with historical data for enabling analytics and deep data mining
• Build aggregates on historical data with various possible metrics and scores for long term data analysis

The technology platform and tools considered for IoT Analytics implementations needs to support all the above aspects well and provide for high-performance real-time data processing and integration.

Considerations for IoT Data Analytics

IoT Data comprises of a continuous stream of readings from sensors and coupled with data coming from application and device logs that may be generated. When we combine this data with Master data about all the entities (or objects or things) that are involved in the entire ecosystem, this data provides a complete view of enterprise data with respect to IoT analytics. This combined data set can provide deep insights into almost every context of the analysis that can be perceived though IoT Data analytics. To accomplish such deep analysis, several techniques and algorithms may be adopted. Given the vast extent of possibility with IoT Analytics, this list can extend to various other approaches as well depending on the specific industry need, complexity of the data and the analytics use case to be implemented. A representative list of methods and/or algorithms are given below as examples for the most common scenarios.

• Exploratory Data analysis to query and Visualize data using BI Charts to understand the data, make inferences
• Correlation Analysis to identify relationships or dependencies between different variables
• Multivariate analysis to assess interdependency and impact of multiple variables on the outcome
• Time Series analysis to understand the changes in a combination of variables over a sequential range of time
• Quintile Analysis to split the range of values of continuous variables into parts and study their relationship
• Principal Component Analysis to simplify analysis of large number of variables
• Pattern discovery to identify interesting observations on data – most common and unusual patterns
• Identify and profile parameter values for most optimal operating conditions (for maximizing output)
• Anomaly detection to identify unusual or unexpected behavior in data which can indicate a potential problem
• Hypothesis analysis to validate a phenomenon or an assumption to be true or false
• Failure pattern analysis to study data patterns associated with a failure scenario and profile them
• Classification and Clustering models to identify similarity and group / segregate data sets or scenarios
• Metaheuristics Algorithms for Optimization problems
• Predictive models to identify the most likely outcome and help in taking a decision
• Prescriptive models to recommend the next best action to be taken

Conceptual Architecture for IoT Data Lake and IoT Analytics

Building the IoT Data Lake and enabling advanced analytics follows a similar architecture approach as with the Business Data Lake architecture. The main difference is in the Real time data processing and real time data analytics and managing different layers of data at different granularity of time and other Business entities involved. A representative Conceptual architecture for IoT Data Lake and Analytics is depicted below:

Data sources for IoT generate sensor data continuously and this data flows through the streaming raw data processing layer. Data sources also include Master data sets that are associated to the IoT data like People, places, processes, devices/machinery, objects, events and others. Operations data will cover data about day-to-day operations like product schedules, supply chain movement, transactions, events, etc.

Raw data processing layer is the main real-time streaming data processing engine which needs to run continuously with high performance and high scalability and execute bulk of the data crunching and loading to the IoT Data Lake.

Analysis & Insights layer provides a variety of tools and applications to analyze the data, build analytical models, derive insights, search for information and collaborate with internal teams and external partner enterprises.

Managing Data Governance, Data Privacy and Secrecy

Governance and control are an extremely critical aspect of managing IoT Data for both Stored data as well as real-time incoming data. Some of the key aspects of governance include:

Data Privacy – Managing Personally Identifiable Information (PII) is guided by various Standards and Regulations across the world:

• Payment Card Industry Data Security Standard (PCI DSS)
• Health Insurance Portability and Accounting Act (HIPAA)
• Gramm Leach Bliley Act (GLBA)
• Telephone Consumer Protection Act (TCPA)
• California Consumer Privacy Act (CCPA) and other similar Acts for various US states
• Children's Online Privacy Protection Act (COPPA)
• European Union's General Data Protection Regulation (GDPR)
• European Union's upcoming Digital Services Act and Digital Markets Act
• European Union's upcoming EU’s Artificial Intelligence Act
• Various Data privacy Regulations across different Countries

Securing PII Data can be achieved through various ways like separating the PII Data from other data and restricting their access, de-identification of the data, masking of the data or complete data obfuscation such that the unmasking is not possible. Data privacy guidelines and implementing them will need to be applied at every stage from the point of collection of the data to processing stages, data storage, analysis and reporting and should also be auditable.

Data Secrecy – IoT Data in some cases may contain business critical information that need to be kept highly confidential (for E.g.: Sensor data in Oil Extraction may indicate potential quantum of Oil resources; Sensor data in Research and Product development; data analyzed in Forensics). Data Security measures like Cybersecurity, Cloud Security, Access Controls, Audit logs and other procedures need to be implemented to manage Data secrecy.

Data Access Control – Access to IoT data should be governed and controlled based on which individuals or roles or applications need to access which part of the IoT Data generated across the enterprise. The governance policies and procedures need to implement various levels of data access based on specific roles and need and manage the access control of the data accordingly.

Data Archival and Purge – As millions of IoT sensors produce a vast amount of data every second and throughout the year, the data volumes can grow to multiple terabytes to petabytes depending on the scale of IoT data generated. Particularly in cases where a mix of structured data from sensors and unstructured data from cameras, scanners is present, data volumes can grow drastically in a few days. The governance policy should define procedures for maintaining different layers of data like raw data, processed granular data, aggregated data, enriched data, analytical data sets, calculated statistics, metrics and scorecards, output data from analytics, labeling them based on associated entities and time periods and having a policy for periodically archiving them to secondary storage and purging older data. Archival and purge policies will also need to be defined for Reports/dashboards created from time to time.

Leading IoT Analytics Platforms

In the last four to five years a number of IoT platforms have emerged with most of them providing hardware and software tools to set up devices and collect the data. There are also a number of IoT Startups with products specialized in IoT analytics focused on a specific industry segment. A full-fledged IoT Platform should provide features and functionality covering all the points mentioned in Considerations for IoT Data processing and IoT Data analytics in previous sections of this document and offer a generic platform with complete infrastructure and tools for processing, storage, analyzing and delivering insights at high performance and large scale. We need to carry out a detailed analysis based on the IoT Analytics requirements of an enterprise, evaluate different platforms on the features offered, limitations, pricing model, infrastructure requirements and overall fit for purpose assessment of the platforms and decide the best choice.

There are over 600 IoT platforms as of 2021 providing capability in IoT Data processing, analysis and advanced analytics. Hyperscaler Cloud platforms and leading product companies are in the forefront of IoT Data Analytics platforms. Among the 600+ IoT platforms in the market and a lot of new companies emerging, a small set of the prominent platforms is given below (in alphabetical order):

Coforge’s Solutions and Enablers for IoT Data Analytics

At Coforge’s Data & Analytics practice we specialize in implementing transformational engagement for Data modernization of Enterprise Data Warehouses or Data Lakes and in delivering business value through data analytics and Data science. Our technology research and development initiatives are focused on building solutions for data engineering, data governance and advanced analytics which we offer as solutions and accelerators in our customer engagements. Led by our Data & Analytics Center of Excellence, and in collaboration with our technology partners offering leading IoT Data Analytics platforms, we provide a comprehensive offering for a successful implementation of IoT Data Analytics programs. We provide the methodology, IoT Platform expertise, tools and technology expertise on Big Data, Cloud, AI /Machine Learning, Advanced Visualization enterprise Data Governance, IoT strategy and technology consulting, end to end implementation of data science use cases for IoT Data Analytics across industries.

Conclusion

IoT data analytics offers transformative potential across various industries by providing real-time insights into operational efficiency, user behavior, and predictive maintenance. By integrating IoT devices and leveraging advanced analytics, companies can enhance decision-making, optimize resource allocation, and drive innovation. Emphasizing data security, scalability, and interoperability is crucial for maximizing the benefits of IoT data. Coforge’s expertise in IoT data analytics helps organizations harness these insights to improve performance, enhance customer experiences, and achieve sustainable business growth. Embrace IoT data analytics to stay competitive and drive future success.