Testbed sensordata report

This report describes the research that is done for research topic 1 in the Geonovum Testbed Sensordata. Part of the research was making a proof of concept, which is available on:

• GitHub: NenS sensordata-demo
• FROST-Server implementation of SensorThings API: FROST-Server
• Lizard data warehouse: Lizard staging

The implementation of sensors for measuring things in the field has grown over the years and will continue growing. It is a challenge to handle the vast amount of collected data and keep grasp on the internal relationships within the data.

Different uses of the data require different views on them. Registration of measurements is done based on the sensor that collects them. For other purposes, e.g. for dashboards or digital twins, you are often interested in the current situation for a collection of features of interest that are measured by sensors.

It is also interesting to see how such an API standard fits to existing applications. If an existing application would have to expose its data through a new standard, what difficulties can be foreseen?

OGC has multiple API standards that cover observations and measurements. Geonovum is interested in the applicability in the Dutch domain of two of these: OGC SensorThings API (STA) and OGC Connected Systems API (CS API).

In this research topic the focus is on what options there are to query combinations of observations. How to store data based on a datafeed from a sensor platform and how to expose them through an API, supporting different uses.

The municipality of Rotterdam would like to see their live sensordata exposed using standards such as SensorThings API and Connected Systems API and learn what this would mean in practice. For instance how would groundwater levels, which are derived by calculation from the raw sensordata, be exposed. How would sensors that measure multiple values (12-1 weather station) be represented. Can the sensors benefit from being exposed in an Observations Measurements and Sampling (OMS) solution?

They are also looking for a scalable and accessible solution to store their sensor data in and make it available for further use. For groundwater levels Nelen & Schuurmans Lizard data warehouse is already used as a system integrator. What role can the OGC standards play in data exchange between systems?

We first familiarized ourselves with the standards at hand by doing a desk study on available documentation. What concepts are used, how are datamodels designed and what is the setup of the API's?

Next, we set up a proof of concept application that makes use of the data made available by Rotterdam and process it through the API standards to our own data warehouse Lizard. By using the API we learn:

• How the mapping between sensor input and the API datamodel needs to be configured
• What query options there are in the API standard
• How API outputs can be shaped to the desired output for different use cases
• What mapping is needed to supply data from the API standard to another data platform

Unfortunately, we didn't succeed in installing the CS API, because there were technical issues with the available docker images of the 52North package (reported here). For the functional comparison with STA we used an existing implementation based on OGC CS API Docs, which refers to the GeoRobotix OGC testbed 18 CS API.

Although they have an overlap in underlying definitions, the two API standards have slightly different backgrounds and therefore different design choices. STA focusses on IoT devices where CS API is more generically applicable. Also, there is a difference is age. The first version of STA has been approved in 2016, while CS API is currently still in draft.

The relevant aspects for comparison for this research topic are:

• Datamodel: The datamodel design determines to a large extent how an API can be interacted with and how incoming data should be mapped. What entities should be configured to set up a usable system? Which entity relations are required?
• API setup and capabilities: How can data be retrieved? What options are available to shape the output to a desired format?

Items that are explicitly excluded from the comparison are:

• Tasking Core/Actuators: The research is about sensor observations and not about the capabilities to interact with sensors.
• API extensions: STA for instance has a PLUS extension. We will only consider this in the assessment of possible ways to improve the applicability of the standards.

2.6.1 Datamodel

Similarities relevant to this research topic:

• Measured results are stored as Observations with Datastreams as their main grouping mechanism.
• There are entities to describe the object that is observed with/from.
• An Observation concerns a Feature of Interest.

Differences relevant to this research topic:

• The entities Sensor and Thing from STA are generalized in CS API to the entity System.
• In STA the UltimateFeatureOfInterest is only linked to the Observation. CS API also has a more direct link from the UltimateFeatureOfInterest to a Datastream, through the SamplingFeature.

There are two possible extensions that support grouping of Observations to improve the accessibility to similar Observations. This is relevant for the use of these API's in the support of applications like dashboards or digital twins, where you often want to know the state of a collection of objects at a certain time. The original setups of the API standards lack specialized filter or aggregation options to retrieve a single value in time per object, given that in most cases Observations are not neatly lined up in time.

STA PLUS has an ObservationGroup entity which can contain "a bag of Observation and/or relations". This extension is scoped on citizen science and focusses on the division of data under licenses and mechanisms to share data. In practice it is for example used to relate samples of water quality done in the same water body. It is not modeled and optimized to support more generic applications.

The ObservationCollection in the Observations, Measurements and Sampling (OMS) model does have a generic scope and therefore a little less overhead in attribution.

The suitability of these grouping mechanisms for our purpose seems questionable. One would have to maintain a registration of collections per relevant timestep that might be used by e.g. a dashboard. This will take a huge administrative workload to keep up to date. It makes much more sense to put an effort in aligning Observations in time, so they can be retrieved using basic filter options.

On implementability of the API standards:

• The API standards are applicable in the Dutch domain, at least as stand-alone applications. Data can be exchanged between these standards and other applications.
• Suitability depends on the purpose of an application. If it relies on information about the sensors/systems, e.g. for validation of Observations, it makes sense to use them. If not, these standards come with quite some overhead in entity relationships and configuration.
• Due to overhead in the datamodels there is a risk of scalability issues for larger data systems. There is a limit to what can be solved by clever indexes, buckets and partitioning of Observations, their different possible types of results and entity relations.
• Full support of these standards by existing data warehouse applications like Lizard, e.g. Delft-FEWS and HydroNET, will be tricky due to a mismatch in mapping of entities between datamodels. It would probably require specific boundary conditions for how data is stored and metadata is configured to make either API standard applicable for both data ingestion and publication. STA has some additional disadvantages because of the use of OData.

On communication with a collection of sensors:

• STA supports query options to expand API responses with information of related entities and both have a select option to limt the response size. This can be used for dashboarding applications to retrieve results for multiple objects.
• The accuracy of temporal queries on Observations depends on the measurement regime of a collection of sensors and the way properties like phenomenonTime and resultTime are stored. If not lined up neatly on rounded times and intervals, there is a big risk of returning multiple results per Datastream, which may be hard to deal with by the retrieving application.
• Based on documentation, CS API has one advantage over STA when it comes to querying Observations for collections of Datastreams: The filter option latest on phenomenonTime and resultTime. Unfortunately, in the implementation that we tested with this did only work when requesting Observations for a single Datastream, which makes it irrelevant for dashboarding.
• A mechanism like ObservationCollections makes querying groups of Observations much easier. Disadvantage is that it requires a lot of additional administration, to group Observations like that and to keep Collections up to date in time. For operational services this will likely give a lot of overhead.

We've discussed the outcomes of our research with specialist Hylke van der Schaaf from research topic 3. He had a couple of interesting comments:

• In version 2 of STA FeaturesOfInterest can be related to Datastreams directly, overcoming the difficulties we had mapping the Observations based on that perspective in comparison with CS API.
• FROST-server logs queries that take longer than a certain treshold. This information can be used to improve database indexes and other data optimizations. Of course, the larger the storage size and the wider the use of an implementation it will become harder and harder to fine solutions for potential performance issues. He has good experiences with implementations that contain up to 850 million Observations.

For future research on the applicability of these and similar standards we have the following recommendations:

• Does the potential applicability reach further than stand-alone systems? To what extent could existing applications adopt these standards, e.g. when it comes to data exchange through underlying standards?
• What role could temporal aggregation of observations play? This is something that is missing in these standards but is widely used in the field of timeseries data.

An important aspect of working with sensor data is, of course, being able to get the data and load it into an application for visualisation and analysis. This research question is about demonstrating that this works, both in a web viewer and in a dashboard application (e.g. Grafana).

Demonstrate the following steps in both types of application:

1. The user searches for and finds different sensors that offer sensor data;
2. The users selects a subset of measurements based on several criteria. The following criteria must be demonstrated as a minimum:

• most recent measurement of one sensor;
• all measurements at a certain location within a certain time period;
• most recent measurement of a selection of sensors that are located within a bounding box.

3. The application loads the selected measurements;
4. The user can view and use the measurements within the application.

RIVM offers an existing SensorThings API (STA) endpoint (https://api-samenmeten.rivm.nl/v1.0) which can serve input for analysis and visualization demonstrations. Further, we are interested to investigate if this endpoint offers sufficient data and metadata detail to support connections to third party dashboards, or if (small) tweaks are required in order to enable easier integration of the offered data with other tools.

This research is conducted by Mindloops BV. Mindloops is a boutique software consultancy specializing in software engineering. For nearly 10 years, Mindloops has proudly delivered solutions to clients such as the Dutch Police, the Dutch Railways (NS), The Netherlands’ Cadastre, Land Registry and Mapping Agency (Kadaster) and Dutch National Road Data Portal (NDW). Together with our clients and partners, Mindloops is committed to making a positive impact on society. Mindloops has a proven track record in building and running high-quality, mission-critical, bespoke software designed to last and evolve for decades. These data-intensive systems often require extensive integration with other systems, as well as data visualization.

The viewers communicate with a SensorThings API and allow users to select sensors and view measurements. As a data source we used the RIVM SamenMeten SensorThings API. To prove that our viewer is able to handle SensorThings API in a generic way we’ve also added the Fraunhofer Airquality SensorThings API (as a bonus).

We’ve applied a risk-driven approach in which we tackled the most complex things early on. This maximises the desired outcome within the available time. Also we used an iterative and agile approach with short feedback cycles to validate our results, which aligned with the bi-weekly meetings with GeoNovum.

We've implemented the requested features in a dashboard application and web viewer. Both are accessible on a publicly accessible URL, which will be available for demonstration purposes until 31-10-2025.

• Dashboard application (Grafana): https://sensordata-dashboard.mindloops.io/
• Web viewer: https://sensordata-viewer.mindloops.io/

All source code needed to run both solutions in a local environment is delivered as open source under MIT license. See the README in each repository for details.

• Dashboard application (Grafana): https://github.com/mindloops/sensordata-dashboard
• Web viewer: https://github.com/mindloops/sensordata-web-ui

All research goals were successfully achieved for both the dashboard application and the bespoke web viewer. The Sensordata API allows for a flexible way to interconnect sensor data with different visualisation and analytics technologies. There are some caveats, e.g. issues in API implementations, and potentially vague or missing metadata in the generic API, that makes it necessary to customize or tune the viewer based on details outside the SensorThings API.

The OGC SensorThings API is a very convenient API for accessing sensor data. But to access sensor data, the data has to be added to a service first. Most sensors no not talk SensorThings, but some other standard, like LoRa-WAN, NB-IoT or 5G. What needs to happen to have a sensor be registered in a SensorThings service, and have the data from the sensor appear there too, is device registration, or onboarding. This research topic explores the issue of device registration.

This research is done by the Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB, specifically the department Information Management and Production Control (ILT). The department Information Management and Production Control develops open and secure architectures, software components and solutions for information, control and test systems in Internet of Things (IoT) application areas.

Device registration, and following that, device management, is an ongoing research topic. Not just in the context of the Internet of Things, but everywhere where sensors are deployed and managed. The complexity stems from the fact that there are very many different sensors and sensor types, used in very many different use cases, feeding their data into very many different sensor data management systems over many different communication protocols. The manufacturer of a Sensor system can’t know in advance the use cases the sensor will be used for, or the sensor data management systems it will need to feed data to.

When deploying a new sensor for a given use case, communicating over an existing communication infrastructure, sending data to an existing sensor data management system, information from four different sources needs to be combined:

• The sensor hardware, such as:
  • Sensor identification
  • Measured parameters
• Communication Infrastructure, such as:
  • Authentication and Authorisation
  • Addresses of brokers or gateways
• Sensor data management system, such as:
  • Authentication and Authorisation
  • API endpoints
• Use case, such as:
  • Feature the sensor observes
  • Location of the sensor
  • Responsible party

The initial configuration of sensor devices is manufacturer specific and needs to be done directly on the device For instance, many manufacturers of LoRaWAN sensor devices provide a smartphone app that uses Near-Field-Communication (NFC) to directly change the configuration of their devices. Next to the initial configuration of the device itself, the device also has to be registered on the communication network. On the case of LoRa, this is often the global network “The Things Network”, but many organisations have their own local LoRa instance using the “Chirpstack” software implementation. Once the initial configuration is completed, and the device is connected to a LoRa network, the configuration of many devices can also be changed using Over-The-Air updates.

The sensor also has to be registered in the sensor data management system of the organisation that deploys the sensor, so that the data from the sensor can be used for the purpose it was deployed for. There are many different systems for this, many custom-built for their use case, some based on standards such as O&M, OMS or the SensorThings API.

Because of the enormous diversity in each of the layers involved, the concept of connectors is used to transfer data between the communication infrastructure and the information management system. A Connector is a piece of software with Extract, Transform, Load (ETL) functionality, that takes data from a source system, transforms the data into the format of the target system, and then loads the data into the target system. A common piece of software used for this purpose is NodeRed, that features a browser-based, drag-and-drop interface for creating ETL flows.

One problem with this architecture is directly clear from figure 1: The data from a single sensor has to be managed and kept synchronised over multiple systems.

Taking a typical LoRa device, using NodeRed as ETL stack, the process looks something like this:

1. Device: Configure the device by setting the identifiers and secrets used to join the LoRa network and configure the intervals in which measurements are made.
2. LoRa Network: Register the device
   1. Create a device object in the LoRa network stack with the identifiers and secrets from the previous step.
   2. Set up a low-level decoder to decode the payload of the sensor into a generic JSON object. If multiple devices of the same type are used, this generally only needs to be done once.
3. SensorThings Service: Create entities for managing the sensor data
   1. Create a Thing with a Location
   2. For each Sensor on the device, create an ObservedProperty instance if a suitable one is not already present.
   3. For each Sensor on the device, create a Sensor instance if a suitable one is not already present.
   4. For each Sensor on the device, create a Datastream, linked to the Thing, a Sensor and an ObservedProperty.
4. NodeRed: Create the workflow that can decode the generic JSON object and insert each value into the correct Datastream for the device.

Device Registration is also only the first step in the long process of device management. Devices can break, be moved around, and be decommissioned. Battery lifetime needs to be tracked to ensure they are exchanged in time and sensors need to be monitored to ensure they still work as expected. It is quite likely that the admin in the diagram will start using a spreadsheet to keep track of his sensors, thus adding a fifth location that contains data for a sensor that needs to be kept in sync.

To make the complexities of sensor management more manageable, Fraunhofer IOSB started an internal research project to design the concept for a sensor management system, based on the OGC SensorThings API, and create an implementation of this system.

Since the SensorThings API version 1.1 is extendible by design, and already comes with several extensions, the architecture can be greatly simplified by using the SensorThings API service as the central data store for all sensor related data (figure 3). This means that all components communicate through a single service, reducing the number of interconnects between components and reducing the spread of primary information across components.

The standard tasking extension can be used to coordinate management actions between components, such as signalling to a connector that a sensor needs to be on-boarded, that a configuration needs to be changed, or that a sensor needs to be off-boarded. The connector concept can then be extended to not just be a one-way ETL process, but to take an active role in the sensor registration process on the LoRaWAN stack. It can receive information about new or updated sensors from the SensorThings service, and automatically take all required registration actions in the communication infrastructure.

Besides greatly simplifying the architecture, a second major advantage to using the SensorThing API service for all data storage is that it offers a consistent, powerful API for managing relational data. This makes all data relevant for managing sensors and their data available in a unified, consistent way, and management tools or other clients do not need to implement multiple APIs. While all publicly relevant sensor data and metadata can be stored in the core data model of the SensorThings API, internal management data can be stored in a custom data model extension. Since this does not alter the core data model of the SensorThings API, clients implementing only the Sensing part will not be affected by this data model extension.

Sensor configuration parameters are modelled using the SWE Common Data Model Encoding Standard, and translated by the connector into a form that the sensor understands. This means that regardless of sensor brand or type, the management GUI can offer a consistent interface for changing sensor settings.

OpenCitySense consists of:

• A data model extension for FROST-Server.
• A GUI for managing sensors.
• A generic framework for developing connectors.
• A connector for connecting LoRaWAN devices over Chirpstack.
• A connector for connecting LoRaWAN devices over TTN/TTI.

4.4.1 Data Model

To allow the representation of device management information, a data model extension has been designed for the data models of the SensorThings API and the tasking extension. The extended data mode is depicted in the following image.

Connectors and Devices are modelled as Things. To make it easier to distinguish between different types of Things, a "type" field has been added to the Thing entity type that indicates the type of the thing. Things of type "Connector" are linked to the Things of the devices they manage, through the ControlledDevices <-> ControllingConnector relation. This makes it easy to find all the devices controlled by a certain connector, and to find the connector controlling a certain device.

Each Thing can have a DeviceModel, describing the capabilities of the Device or Connector. A DeviceModel contains the schema for the Configurations of devices of this model, and can link to a Configuration that is the template or default configuration of devices of this model. DeviceModels can link to a Decoder that can be used to decode and encode data coming from and sent to devices of this model.

DeviceModels link to Sensors that describe the Sensors that a device of the model has. In turn, Sensors link to the ObservedProperties that a Sensor of this type observes. Using these two links, a Connector knows which Datastreams to create and which Sensor and ObservedProperty to link, when onboarding a Device.

DeviceModels link to the DeviceModels of the Connectors that they are compatible with. This allows a user interface to find the DeviceModels that work on a chosen Connector, and allows the Connector to specify additional configuration options it requires on a Device and a DeviceModel.

Configurations describe how a device can be, was or is configured. The schema for the config is stored in the DeviceModel of the device. The status field of a configuration indicates the current status of a sensor (Created, Active, Inactive, Removed) or if the Configuration is a Template. Configurations have a time field that indicates when this configuration became active. If a device has multiple configurations there must be only one configuration with status "Active". The other configurations are historical Configurations or templates.

To allow the secure storage of passwords or API keys, the DeviceSecret class was added to the data model. The secrets can be secured, both by only giving certain users read-access to these device secrets, and by encrypting the values of the device secrets. To allow Encryption, a connector has a public/private key pair. The private key of a connector is not stored in the SensorThings data model, but directly passed to the connector, usually using an environment variable. The public key of the connector is available in the SensorThings data model and can be used by clients to encrypt passwords before storing them in a DeviceSecret entity. This way only the connector can decrypt these secrets.

https://ogc-demo.k8s.ilt-dmz.iosb.fraunhofer.de/FROST-OpenCitySense/v1.1/Things?
  $filter=name eq 'TTN LoRa Connector'&
  $expand=DeviceSecrets($select=name,type,value),Configurations

https://ogc-demo.k8s.ilt-dmz.iosb.fraunhofer.de/FROST-OpenCitySense/v1.1/Things?
  $filter=name eq 'Elsys_A81758FFFE035140'&
  $expand=Datastreams(
    $select=name,description,unitOfMeasurement;
    $expand=
      ObservedProperty($select=name),
      Observations(
        $select=phenomenonTime,result;
        $orderby=phenomenonTime desc;$top=1))

https://ogc-demo.k8s.ilt-dmz.iosb.fraunhofer.de/FROST-OpenCitySense/v1.1/DeviceModels?
  $filter=name eq 'Elsys ERS 2'&
  $expand=
    Decoder,
    Sensors(
      $select=id,name;
      $expand=ObservedProperties($select=id,name))

{
  "temperature":21.3,
  "humidity":47,
  "light":1,
  "motion":0,
  "co2":488,
  "vdd":3615
}