Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion first-mile-guide/sections/clause2/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@

==== Requirements Class: Node

==== Requirements Class: Host
==== Requirements Class: Hub
20 changes: 10 additions & 10 deletions first-mile-guide/sections/clause6/index.adoc
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
== Overview (informative)

This clause provides a high-level view of the First-Mile Data Collection Guide. Communication is strictly unidirectional, with the Node transmitting protobuf payloads over MQTT to the Host. The two top-level payloads are the `Data` message, which carries Observation content, and the `Metadata` message, which carries the descriptive context needed to interpret those observations.
This clause provides a high-level view of the First-Mile Data Collection Guide. Communication is strictly unidirectional, with the Node transmitting protobuf payloads over MQTT to the Hub. The two top-level payloads are the `data` message, which carries observational content, and the `metadata` message, which carries the descriptive context needed to interpret those observations.

=== System architecture

The Node represents the remote station or logger through a Node Device and may describe attached sensors as Observer Device entries. The `Metadata` message carries the Node Device, Observer Device, Parameter Definition, and namespace information, while the `Data` message carries one or more Observation instances that refer to those Parameter Definition identifiers. Detailed message structure is described in Clause 7, standard name binding in Clause 8, and MQTT protocol binding in Clause 9.
The Node represents the remote station or logger through a Node Device and may describe attached sensors as Observer Device entries. The `metadata` message carries the Node Device, Observer Device, Parameter Definition, and namespace information, while the `data` message carries one or more Observation instances that refer to those Parameter Definition identifiers. Detailed message structure is described in Clause 7, standard name binding in Clause 8, and MQTT protocol binding in Clause 9.

[source,text]
----
Expand All @@ -14,21 +14,21 @@ The Node represents the remote station or logger through a Node Device and may d
| + Observers | | + Observers |
+-------------------+ +-------------------+
| |
| Metadata and Data messages | Metadata and Data messages
| metadata and data messages | metadata and data messages
| over MQTT | over MQTT
v v
+----------------------------------------------------------+
| Host |
| Hub |
| Receives, decodes, and interprets protobuf payloads |
+----------------------------------------------------------+

Multiple Nodes may publish
to a single Host.
to a single Hub.
----

=== Message flow

In typical operation, the `Metadata` message is sent when the Node is initialized and whenever the station configuration changes, and the `Data` message is sent as observations become available. This separates relatively stable descriptive information from frequently changing measurement values and supports both individual and batched Observation delivery. Where a measurement is unavailable, the value is represented explicitly using `emptyValue` rather than being inferred from zero or omission.
In typical operation, the `metadata` message is sent when the Node is initialized and whenever the station configuration changes, and the `data` message is sent as observations become available. This separates relatively stable descriptive information from frequently changing measurement values and supports both individual and batched Observation delivery. Where a measurement is unavailable, the value is represented explicitly using `emptyValue` rather than being inferred from zero or omission.

=== Conformance class summary

Expand All @@ -37,17 +37,17 @@ In typical operation, the `Metadata` message is sent when the Node is initialize
|Conformance class |Summary

|Core Data Model
|Covers the protobuf structure and semantics of the `Data` message, `Metadata` message, Observation content, devices, Parameter Definition content, and explicit missing values.
|Covers the protobuf structure and semantics of the `data` message, `metadata` message, observational content, devices, parameter definition content, and explicit missing values.

|Standard Name Binding
|Covers how Parameter Definition entries associate parameters with controlled standard-name vocabularies and namespaces.

|MQTT Protocol Binding
|Covers how protobuf payloads are conveyed over MQTT in the unidirectional exchange between Node and Host.
|Covers how protobuf payloads are conveyed over MQTT in the unidirectional exchange between Node and Hub.

|Node
|Covers the sender-side behavior for constructing and publishing `Metadata` message and `Data` message payloads.
|Covers the sender-side behavior for constructing and publishing `metadata` message and `data` message payloads.

|Host
|Hub
|Covers the receiver-side behavior for accepting, decoding, and interpreting payloads received from the Node.
|===
38 changes: 19 additions & 19 deletions first-mile-guide/sections/clause7/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,24 +2,24 @@

=== Overview

The data model defines the structure and organization of information transmitted from the Node to the Host using Protocol Buffers (Protobuf) as the Data Format. The data model is formally specified in the protobuf schema `firstmile.proto`.
The data model defines the structure and organization of information transmitted from the Node to the Hub using Protocol Buffers (Protobuf) as the Data Format. The data model is formally specified in the protobuf schema `firstmile.proto`.

The data model supports two primary use cases:

1. **Transmission of observation data** - The Node transmits measurement values collected from sensors
2. **Transmission of metadata** - The Node transmits information about the monitoring site, devices, and parameter definitions

These use cases are supported by two top-level message types: the `Data` message containing observations, and the `Metadata` message containing configuration and contextual information. Both messages are defined in a single protobuf schema and transmitted as independent payloads over MQTT. The `Metadata` message provides the necessary context for interpreting the `Data` message, including descriptions of devices and parameters.
These use cases are supported by two top-level message types: the `data` message containing observations, and the `metadata` message containing configuration and contextual information. Both messages are defined in a single protobuf schema and transmitted as independent payloads over MQTT. The `metadata` message provides the necessary context for interpreting the `data` message, including descriptions of devices and parameters.

==== Communication pattern

The communication follows a unidirectional pattern where data flows from the Node to the Host. The Node may transmit messages in the following patterns:
The communication follows a unidirectional pattern where data flows from the Node to the Hub. The Node may transmit messages in the following patterns:

* **Data-only transmission** - Frequent transmission of `Data` messages containing current or batched observations
* **Metadata transmission** - Transmission of `Metadata` messages when the configuration changes or upon initialization
* **Data-only transmission** - Frequent transmission of `data` messages containing current or batched observations
* **Metadata transmission** - Transmission of `metadata` messages when the configuration changes or upon initialization

The Node should transmit the `Metadata` message at least once after initialization and whenever the configuration of devices or parameter definitions changes. The `Data` message may be transmitted as frequently as needed to deliver observations.
The Host can only intepret data correctly if it has received the relevant `Metadata` message that defines the devices and parameters being observed. Therefore, it is recommended to transmit the `Metadata` message at least once before transmitting any `Data` messages.
The Node should transmit the `metadata` message at least once after initialization and whenever the configuration of devices or parameter definitions changes. The `data` message may be transmitted as frequently as needed to deliver observations.
The Hub can only intepret data correctly if it has received the relevant `metadata` message that defines the devices and parameters being observed. Therefore, it is recommended to transmit the `metadata` message at least once before transmitting any `data` messages.

==== Message hierarchy

Expand All @@ -28,7 +28,7 @@ The data model organizes information hierarchically:
* **Messages** - Top-level containers (`Data` and `Metadata`) that are transmitted as independent payloads
* **Devices** - Descriptions of physical equipment including the Node Device (node) and Observer Devices (sensors)
* **Observations** - Measurement data points consisting of a parameter identifier, timestamp, and one or more values
* **Parameter definitions** - Metadata describing what is being measured, including units, aggregation methods, and standard name mappings. Parameter definitions also describe how observations are structured in `Data` messages.
* **Parameter definitions** - Metadata describing what is being measured, including units, aggregation methods, and standard name mappings. Parameter definitions also describe how observations are structured in `data` messages.
* **Values** - The actual measurement data, supporting multiple numeric types, strings, booleans, and explicit representation of missing data

Each `Observation` references a `ParameterDefinition` by identifier. Each `ParameterDefinition` contains one or more `Parameter` specifications that describe the measured variable, its unit, temporal aggregation method, and which device performs the measurement.
Expand All @@ -41,16 +41,16 @@ The data model is designed according to the following principles:
* **Extensibility** - Support for multiple value types and flexible parameter definitions
* **Self-description** - Metadata provides complete context for interpreting observations
* **Explicit missing values** - Clear distinction between unavailable data and zero/false measurements
* **Batch transmission** - Multiple observations may be included in a single `Data` message
* **Batch transmission** - Multiple observations may be included in a single `data` message
* **Standard name interoperability** - Mapping to external naming conventions through namespace definitions

=== Messages

==== Metadata message

The `Metadata` message provides configuration and contextual information about the Node, including descriptions of devices and the parameters they measure. This message shall be transmitted at least once after Node initialization and whenever the configuration changes.
The `metadata` message provides configuration and contextual information about the Node, including descriptions of devices and the parameters they measure. This message shall be transmitted at least once after Node initialization and whenever the configuration changes.

The `Metadata` message contains the following fields:
The `metadata` message contains the following fields:

[cols="2,2,1,5"]
|===
Expand Down Expand Up @@ -82,9 +82,9 @@ The `namespaces` field provides the mapping between namespace keys used in `Para

==== Data message

The `Data` message is the primary container for transmitting observation data from the Node to the Host. This message may be transmitted as frequently as needed to deliver current or batched observations.
The `data` message is the primary container for transmitting observation data from the Node to the Hub. This message may be transmitted as frequently as needed to deliver current or batched observations.

The `Data` message contains the following fields:
The `data` message contains the following fields:

[cols="2,2,1,5"]
|===
Expand All @@ -97,11 +97,11 @@ The `Data` message contains the following fields:

|===

A `Data` message may contain multiple `Observation` messages, enabling efficient batch transmission of measurement data. Each observation is independent and contains its own timestamp and parameter reference.
A `data` message may contain multiple `Observation` messages, enabling efficient batch transmission of measurement data. Each observation is independent and contains its own timestamp and parameter reference.

[NOTE]
====
The Node may transmit `Data` messages containing observations that reference parameter definitions not yet received by the Host. In this case, the Host shall buffer or queue such observations until the corresponding `Metadata` message is received.
The Node may transmit `data` messages containing observations that reference parameter definitions not yet received by the Hub. In this case, the Hub shall buffer or queue such observations until the corresponding `metadata` message is received.
====

=== Devices
Expand Down Expand Up @@ -274,7 +274,7 @@ The `Observation` message contains the following fields:
|parameterDefinitionId
|uint32
|required
|Identifier referencing a `ParameterDefinition` in the `Metadata` message
|Identifier referencing a `ParameterDefinition` in the `metadata` message

|time
|google.protobuf.Timestamp
Expand All @@ -288,7 +288,7 @@ The `Observation` message contains the following fields:

|===

The `parameterDefinitionId` field shall reference a `ParameterDefinition.id` from the `Metadata` message. The number and type of values in the `values` field shall correspond to the number of parameters defined in the referenced `ParameterDefinition`.
The `parameterDefinitionId` field shall reference a `ParameterDefinition.id` from the `metadata` message. The number and type of values in the `values` field shall correspond to the number of parameters defined in the referenced `ParameterDefinition`.

The `time` field uses the standard `google.protobuf.Timestamp` format, representing time as seconds and nanoseconds since the Unix epoch (1970-01-01T00:00:00Z).

Expand Down Expand Up @@ -405,7 +405,7 @@ The `ParameterDefinition` message contains the following fields:

|===

The `id` field shall be unique within the scope of a single `Metadata` message. This identifier is used by the `Observation.parameterDefinitionId` field to associate observations with their parameter specifications.
The `id` field shall be unique within the scope of a single `metadata` message. This identifier is used by the `Observation.parameterDefinitionId` field to associate observations with their parameter specifications.

The `parameters` field contains one or more `Parameter` messages describing individual measurable variables. When multiple parameters are grouped in a single definition, each observation referencing this definition shall provide values in the same order as the parameters are defined.

Expand Down Expand Up @@ -558,5 +558,5 @@ The `DeviceRef` message contains exactly one of the following fields:

The `oneof` constraint ensures that exactly one of these fields is set, unambiguously identifying which device performs the measurement.

When the `node` field is present, the parameter is measured by the Node Device itself (such as internal voltage or temperature of a datalogger). When the `observerId` field is set, it shall match the `id` of an `ObserverDevice` defined in the `Metadata` message.
When the `node` field is present, the parameter is measured by the Node Device itself (such as internal voltage or temperature of a datalogger). When the `observerId` field is set, it shall match the `id` of an `ObserverDevice` defined in the `metadata` message.

10 changes: 5 additions & 5 deletions first-mile-guide/sections/clause8/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

=== Overview

Standard name binding enables `Parameter` definitions within a `Metadata` message to be associated with controlled vocabulary entries from external naming conventions. This allows observation data to be unambiguously interpreted by Hubs and downstream systems without dependence on the `Parameter.longName` field, which is a human-readable label and not guaranteed to be unique or machine-interpretable.
Standard name binding enables `Parameter` definitions within a `metadata` message to be associated with controlled vocabulary entries from external naming conventions. This allows observation data to be unambiguously interpreted by Hubs and downstream systems without dependence on the `Parameter.longName` field, which is a human-readable label and not guaranteed to be unique or machine-interpretable.

The binding mechanism uses two related fields in the data model:

Expand Down Expand Up @@ -31,7 +31,7 @@ The `Metadata.namespaces` field declares the naming convention vocabularies used

Each namespace key shall be a non-empty string containing only alphanumeric characters and underscores. The namespace URI shall be a resolvable, publicly accessible URI that identifies the vocabulary in a dereferenceable way.

A `Metadata` message may declare zero or more namespace entries. When no standard names are used, the `namespaces` map may be empty or omitted.
A `metadata` message may declare zero or more namespace entries. When no standard names are used, the `namespaces` map may be empty or omitted.

=== Standard name assignment

Expand All @@ -51,7 +51,7 @@ The `Parameter.standardNames` field maps namespace keys to the standard name for

|===

Each key in `Parameter.standardNames` shall match a key declared in `Metadata.namespaces`. A Node shall not include a standard name entry for a namespace that is not declared in the corresponding `Metadata` message.
Each key in `Parameter.standardNames` shall match a key declared in `Metadata.namespaces`. A Node shall not include a standard name entry for a namespace that is not declared in the corresponding `metadata` message.

A parameter may carry zero, one, or multiple standard name entries. When multiple entries are present, each entry associates the parameter with the equivalent standard name in a different vocabulary.

Expand Down Expand Up @@ -130,7 +130,7 @@ Where a standard name exists at Level 1 (CF) or Level 2 (WMO registry), that sta

=== Informative example

The following example shows a `Metadata` message extract demonstrating standard name binding across multiple vocabulary levels for a meteorological and hydrological station.
The following example shows a `metadata` message extract demonstrating standard name binding across multiple vocabulary levels for a meteorological and hydrological station.

[source,json]
----
Expand Down Expand Up @@ -166,7 +166,7 @@ The following example shows a `Metadata` message extract demonstrating standard
"longName": "Supply Voltage",
"unit": "V",
"standardNames": { "wmo": "supply_voltage" },
"device": { "host": {} },
"device": { "node": {} },
"cellMethod": "POINT",
"cellPeriodSeconds": 0
}
Expand Down
Loading