Skip to content
133 changes: 80 additions & 53 deletions doc/advanced-topics.md
Original file line number Diff line number Diff line change
@@ -1,37 +1,36 @@
## Advanced Topics

* [Secured access to the Context Broker](#secured-access-to-the-context-broker)
* [GeoJSON support NGSI-LD only](#geojson-support-ngsi-ld-only)
* [Metadata support](#metadata-support)
* [NGSI LD data and metadata considerations](#ngsi-ld-data-and-metadata-considerations)
* [NGSI-LD Linked Data support](#ngsi-ld-linked-data-support)
* [Autoprovision configuration (autoprovision)](#autoprovision-configuration-autoprovision)
* [Explicitly defined attributes (explicitAttrs)](#explicitly-defined-attributes-explicitattrs)
* [Configuring operation to persist the data in Context Broker (appendMode)](#configuring-operation-to-persist-the-data-in-context-broker-appendmode)
* [Data mapping plugins](#data-mapping-plugins)
* [Development](#development)
* [Provided plugins](#provided-plugins)
* [Timestamp Compression plugin (compressTimestamp)](#timestamp-compression-plugin-compresstimestamp)
* [Attribute Alias plugin (attributeAlias)](#attribute-alias-plugin-attributealias)
* [Event plugin (addEvents)](#event-plugin-addevents)
* [Timestamp Processing Plugin (timestampProcess)](#timestamp-processing-plugin-timestampprocess)
* [Expression Translation plugin (expressionTransformation)](#expression-translation-plugin-expressiontransformation)
* [Multientity plugin (multiEntity)](#multientity-plugin-multientity)
* [Bidirectionality plugin (bidirectional)](#bidirectionality-plugin-bidirectional)
* [Autoprovision configuration (autoprovision)](#autoprovision-configuration-autoprovision)
* [Explicitly defined attributes (explicitAttrs)](#explicitly-defined-attributes-explicitattrs)
* [Configuring operation to persist the data in Context Broker (appendMode)](#configuring-operation-to-persist-the-data-in-context-broker-appendmode)
* [Old IoTAgent data migration](#old-iotagent-data-migration)

- [Secured access to the Context Broker](#secured-access-to-the-context-broker)
- [GeoJSON support NGSI-LD only](#geojson-support-ngsi-ld-only)
- [Metadata support](#metadata-support)
- [NGSI LD data and metadata considerations](#ngsi-ld-data-and-metadata-considerations)
- [NGSI-LD Linked Data support](#ngsi-ld-linked-data-support)
- [Autoprovision configuration (autoprovision)](#autoprovision-configuration-autoprovision)
- [Explicitly defined attributes (explicitAttrs)](#explicitly-defined-attributes-explicitattrs)
- [Configuring operation to persist the data in Context Broker (appendMode)](#configuring-operation-to-persist-the-data-in-context-broker-appendmode)
- [Data mapping plugins](#data-mapping-plugins)
- [Development](#development)
- [Provided plugins](#provided-plugins)
- [Timestamp Compression plugin (compressTimestamp)](#timestamp-compression-plugin-compresstimestamp)
- [Attribute Alias plugin (attributeAlias)](#attribute-alias-plugin-attributealias)
- [Event plugin (addEvents)](#event-plugin-addevents)
- [Timestamp Processing Plugin (timestampProcess)](#timestamp-processing-plugin-timestampprocess)
- [Expression Translation plugin (expressionTransformation)](#expression-translation-plugin-expressiontransformation)
- [Multientity plugin (multiEntity)](#multientity-plugin-multientity)
- [Bidirectionality plugin (bidirectional)](#bidirectionality-plugin-bidirectional)
- [Autoprovision configuration (autoprovision)](#autoprovision-configuration-autoprovision)
- [Explicitly defined attributes (explicitAttrs)](#explicitly-defined-attributes-explicitattrs)
- [Configuring operation to persist the data in Context Broker (appendMode)](#configuring-operation-to-persist-the-data-in-context-broker-appendmode)
- [Old IoTAgent data migration](#old-iotagent-data-migration)

### Secured access to the Context Broker

For access to instances of the Context Broker secured with a
[PEP Proxy](https://github.com/telefonicaid/fiware-orion-pep), an authentication mechanism based in Keystone Trust
tokens is provided. A trust token is a way of Keystone to allow an user delegates a role to another user for a subservice.
It is a long-term token that can be issued by any user to give another user permissions
to impersonate him with a given role in a given project (subservice). Such impersonation itself is in turn based on a short-term
access token.
tokens is provided. A trust token is a way of Keystone to allow an user delegates a role to another user for a
subservice. It is a long-term token that can be issued by any user to give another user permissions to impersonate him
with a given role in a given project (subservice). Such impersonation itself is in turn based on a short-term access
token.

For the authentication mechanisms to work, the `authentication` attribute in the configuration has to be fully
configured, and the `authentication.enabled` subattribute should have the value `true`.
Expand Down Expand Up @@ -73,15 +72,18 @@ Agent.

Complete info on Keystone trust tokens could be found at:

- [Trusts concept](https://docs.openstack.org/keystone/stein/user/trusts)
- [Trusts API](https://docs.openstack.org/keystone/stein/api_curl_examples.html#post-v3-os-trust-trusts)
- [Trusts concept](https://docs.openstack.org/keystone/stein/user/trusts)
- [Trusts API](https://docs.openstack.org/keystone/stein/api_curl_examples.html#post-v3-os-trust-trusts)

### NGSI-LD `GeoProperty` support
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the ToC at the beginning of the file I see:

[GeoJSON support NGSI-LD only](#geojson-support-ngsi-ld-only)

So it seems section title and corresponding entry in ToC are not aligned...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed ef3dd11


### GeoJSON support NGSI-LD only
For NGSI-LD only, the defined `type` of any GeoJSON attribute can be any set using any of the standard NGSI-v2 GeoJSON
types - (e.g. `geo:json`, `geo:point`). NGSI-LD formats such as `GeoProperty`, `Point` and `LineString` are also
accepted `type` values. If the latitude and longitude are received as separate measures, the JEXL or legacy
[expression language](expressionLanguage.md) can be used to concatenate them into GeoJSON objects an array of tuples or
a string as shown

The defined `type` of any GeoJSON attribute can be any set to any of the standard NGSI-v2 GeoJSON types - (e.g.
`geo:json`, `geo:point`). NGSI-LD formats such as `GeoProperty`, `Point` and `LineString` are also accepted `type`
values. If the latitude and longitude are received as separate measures, the
[expression language](expressionLanguage.md) can be used to concatenate them.
#### Legacy - encode as String

```json
{
Expand All @@ -99,8 +101,30 @@ values. If the latitude and longitude are received as separate measures, the
}
```

For `attributes` and `static_attributes` which need to be formatted as GeoJSON values, three separate input formats are
accepted. Provided the `type` is provisioned correctly, the `value` may be defined using any of the following formats:
#### JEXL - encode as GeoJSON

```json
{
"entity_type": "GPS",
"resource": "/iot/d",
"protocol": "PDI-IoTA-JSON",
"expressionLanguage": "jexl",
..etc
"attributes": [
{
"name": "location",
"type": "geo:json",
"expression": "{coordinates: [longitude,latitude], type: 'Point'}"
}
]
}
```

JEXL can be used to create GeoJSON objects directly. The Legacy expression language does not support GeoJSON. However,
there is a workaround specifically for NGSI-LD Entities which always require `location` to be encoded as GeoJSON. For
`attributes` and `static_attributes` which need to be formatted as GeoJSON values, three separate input formats are
currently accepted. Provided the `type` is provisioned correctly, the `value` may be defined using any of the following
formats:

- a comma delimited string

Expand Down Expand Up @@ -269,26 +293,28 @@ updated as shown:

### Autoprovision configuration (autoprovision)

By default, when a measure arrives to the IoTAgent, if the `device_id` does not match with an existing one, then, the IoTA
creates a new device and a new entity according to the group config. Defining the field `autoprovision` to `false`
when provisioning the device group, the IoTA to reject the measure at the southbound, allowing only to persist the
data to devices that are already provisioned. It makes no sense to use this field in device provisioning since it is
intended to avoid provisioning devices (and for it to be effective, it would have to be provisional).
By default, when a measure arrives to the IoTAgent, if the `device_id` does not match with an existing one, then, the
IoTA creates a new device and a new entity according to the group config. Defining the field `autoprovision` to `false`
when provisioning the device group, the IoTA to reject the measure at the southbound, allowing only to persist the data
to devices that are already provisioned. It makes no sense to use this field in device provisioning since it is intended
to avoid provisioning devices (and for it to be effective, it would have to be provisional).

### Explicitly defined attributes (explicitAttrs)

If a given measure element (object_id) is not defined in the mappings of the device or group provision, the measure is stored
in the Context Broker by adding a new attribute to the entity with the same name of the undefined measure element. By adding the
field `explicitAttrs` with `true` value to device or group provision, the IoTAgent rejects the measure elements that are not defined
in the mappings of device or group provision, persisting only the one defined in the mappings of the provision. If `explicitAttrs`
is provided both at device and group level, the device level takes precedence.
If a given measure element (object_id) is not defined in the mappings of the device or group provision, the measure is
stored in the Context Broker by adding a new attribute to the entity with the same name of the undefined measure
element. By adding the field `explicitAttrs` with `true` value to device or group provision, the IoTAgent rejects the
measure elements that are not defined in the mappings of device or group provision, persisting only the one defined in
the mappings of the provision. If `explicitAttrs` is provided both at device and group level, the device level takes
precedence.

### Configuring operation to persist the data in Context Broker (appendMode)

This is a flag that can be enabled by activating the parameter `appendMode` in the configuration file or by using the `IOTA_APPEND_MODE`
environment variable (more info [here](https://github.com/telefonicaid/iotagent-node-lib/blob/master/doc/installationguide.md)).
If this flag is activated, the update requests to the Context Broker will be performed always with APPEND type, instead of the
default UPDATE. This have implications in the use of attributes with Context Providers, so this flag should be used with care.
This is a flag that can be enabled by activating the parameter `appendMode` in the configuration file or by using the
`IOTA_APPEND_MODE` environment variable (more info
[here](https://github.com/telefonicaid/iotagent-node-lib/blob/master/doc/installationguide.md)). If this flag is
activated, the update requests to the Context Broker will be performed always with APPEND type, instead of the default
UPDATE. This have implications in the use of attributes with Context Providers, so this flag should be used with care.

### Data mapping plugins

Expand Down Expand Up @@ -373,8 +399,8 @@ events in the IoT Agent with the configured type name will be marked as events.

##### Timestamp Processing Plugin (timestampProcess)

This plugin processes the entity attributes looking for a `TimeInstant` attribute. If one is found, for NGSIv2,
the plugin adds a `TimeInstant` attribute as metadata for every other attribute in the same request. With NGSI-LD, the
This plugin processes the entity attributes looking for a `TimeInstant` attribute. If one is found, for NGSIv2, the
plugin adds a `TimeInstant` attribute as metadata for every other attribute in the same request. With NGSI-LD, the
Standard `observedAt` property-of-a-property is used instead.

##### Expression Translation plugin (expressionTransformation)
Expand Down Expand Up @@ -438,7 +464,8 @@ When a device is provisioned with bidirectional attributes, the IoTAgent subscri
change notification for that attribute arrives to the IoTA, it applies the transformation defined in the device
provisioning payload to the notification, and calls the underlying notification handler with the transformed entity.

The following `attributes` section shows an example of the plugin configuration (using IOTA_AUTOCAST=false to avoid translation from geo:point to geo:json)
The following `attributes` section shows an example of the plugin configuration (using IOTA_AUTOCAST=false to avoid
translation from geo:point to geo:json)

```json
"attributes": [
Expand Down
Loading