Get best of the week

Interaction with NIDD via SCEF using the Postman utility. A brief tour of SCEF and its features

This article will allow those who are just starting their development or already applying NB-IoT technology to get an idea of ​​how to remotely interact with the NB-IoT device.

image

Short review


NB-IoT easily steps on the heels of 2G and has established itself as an energy-efficient standard for cellular communications, which in the foreseeable future will be able to squeeze out 2G, which has strengthened its position. The reason for this is the ability to flexibly approach the issue of energy consumption of one of the most consuming parts of the device - the radio transmitter. If you do not go deep into details, then along with NB-IoT, we have the opportunity to flexibly configure the device’s operating modes by setting the device’s communication schedule and the device’s interaction with servers on the Internet.

In parallel with this, the number of subscriber devices simultaneously connected to one cell is significantly increasing, as well as the costs of the mobile operator to maintain the operability of this communication itself.

It is assumed that the reader has an approximate understanding of the NB-IoT technology and there is minimal interaction experience.
The article is regularly updated and updated.


Difficulties within the NB-IoT network


Together with the ability to control the radio transmitter and high energy efficiency, the problem of sending data in the direction from the server to the module (device) has come. The reason is that it is possible to provide hundreds of thousands of devices with white IP addresses, but this entails a large amount of overhead and reduces the overall reliability of the network due to its complexity. The module receives the address for the NAT of the operator and it is difficult for the operator to translate it "out" due to the large number of devices. For example: 100 thousand devices is the same number of IP addresses and within the framework of IPv4 it simply does not seem possible to implement this. The transition to IPv6 can solve this problem, but you still have to pay for the “white” address of the device on the network, which will significantly affect the pocket of corporate clients.

That is, in the general case, it turns out that the device can only work in one-way mode, transmitting data to the server without the possibility of receiving data from the server, because the server is practically not aware when the device first gets in touch. Or, use on average no more than five minutes after the connection between the device and the server for transferring data from the server to the device is established.

NIDD (Non-IP Data Delivery) - why is this needed?


When working with the NB-IoT network, it is very difficult to turn to a device for transmitting a command or certain data. To solve this problem, a mechanism for optimizing the transfer of small amounts of data, Non-IP Data Delivery (NIDD), was invented. The mechanism reduces the overall size of the transmitted message by reducing the headers. This, in turn, positively affects the characteristics of the device: it reduces power consumption and increases autonomy (battery life). As a result, the rejection of IP support leads to cheaper devices, reduced development time, increased competitiveness in the market of IoT devices, etc.

SCEF (Service Capability Exposure Function) - a gift for users


SCEF is a functional network element that first appeared and 3GPP Release 13, deployed on the side of the mobile operator and providing a secure communication channel between the SCS / AS (Service Capability Server / Application Server) and the NB-IoT device. SCEF provides a communication channel when communicating with the device via NIDD and provides access to additional network capabilities / services of the NB-IoT network through a single application programming interface (from API T8 description). In 3GPP Release 13, only the SCEF mechanism for interfacing with cellular network interfaces was standardized. Thus, the network load is optimized, interaction with the device is simplified, the algorithm of the device itself is simplified.SCEF also fulfills high requirements for the security of data transmission and the fulfillment of requirements to confirm successful data transfer in both directions.


SCEF allows you to abstract from complex systems of interaction with the NB-IoT device, including when the latter is in eDRX or PSM mode and is not available for data transfer in the direction of the server-> device. When the device has received registration and “agreed” with the network operator’s network about the communication schedule, using a simple interface, you can transfer data to the device and receive data from it, manage the “subscription” of your device and AS to certain events, set and bind unique ones yourself Universal ID names and more. All this through the same interface - T8 API.

It is important to note that a server (AS) can be not one, but several, and you can flexibly configure the distribution of information between servers for certain events or groups of devices.

How it works


The most popular solution for organizing access of a device to a cellular network is the use of a cellular module, for example:


When registering in a cellular network, such a module transmits to the operator some information, including the IMSI of a subscriber SIM card which the operator can associate with the subscriber account or the subscriber himself if he has access to the operator’s personal account. Behind the SCEF screen is hiding knowledge of the next communication session with the device. Registration of a non-IP device in the operator’s network is possible only if there is at least one subscription from SCS / AS to this device. No subscription - no one will communicate with this device via NIDD, and the device will not be registered on the network. Thus, SCEF, knowing about the next communication session, is able to transfer data from / to the device, in accordance with the specified delivery parameters and the lifetime of the transmitted message, without the need to organize additional monitoring of the status of data transfer and delivery control.

Lightness


Encapsulating units of data bytes from the device into the TCP protocol and transferring it to the server is expensive in terms of hundreds of thousands of subscriber devices in the company's fleet. SCEF allows you to abandon IP on the device and transfer only non-IP data, without IP headers through the signal channel, which contributes to a multiple reduction in the cost of the transmitted byte and provides additional benefits from the use of the service. Moreover, SCEF does not make any changes to the transmitted data both to the device and from it, payload is transmitted transparently. Therefore, using NIDD it is possible to transfer not only unstructured data, but also data “wrapped” in an understandable standardized format, such as JSON, to simplify data processing on the AS side.

Beginning of work


The structure of the URI (Uniform Resource Identifier) ​​on the example of Postman


First of all, you need to get access to your personal account from the operator (M2M-manager service). For the commercial implementation of MTS, a single Personal Account interface is provided, where you can independently create APN, login / password access to the API and assign the names ScsAsID, extID for your devices.

Those. we will at least need:
  • ScsAsID - Your AS ID
  • APN - the one that is usually used to interact with the network will not work
  • Login / Password - data for access to your personal account and interaction with SCEF
  • URI –HTTP address and port on the network provided to you by SCEF
  • externalID - ID of our device


Let's move on to practice


The theory is over, let's move on to the most interesting part - the practice on the SIMCom Wireless Solutions production module - SIM7020E .

First you need to configure the module itself to work with NIDD. To do this, first put the module in CFUN = 0 mode and configure it:

AT+CFUN=0
+CPIN: NOT READY

OK
AT+ENVDM=1,tel_conn_mgr,default_pdn_flag,1,30
OK
AT*MCGDEFCONT="Non-IP","<APN>"
OK
AT+CFUN=1
OK

+CPIN: READY
AT+EGACT=1,4,"<APN>"
+EGACT:1

OK

+EGACT:1,1,1,4

Checking the registration of the module in the packet data network

AT+CGREG?
+CGREG: 0,1

OK

And finish setting up NIDD

AT+NIDD=0,"<APN>","<login>","<pass>"
+NIDD=0,0

OK

The response of the module will contain account_id here: + NIDD = 0, <account_id> - this will come in handy when activating NIDD on the module.

AT+NIDD=1,0
+NIDD=1,0

OK

In this case, account_id is 0.

Done. At this stage, work with the module is completed. Let's move on to setting up SCEF.

An important point: without a registered subscription in SCEF, the module will not receive registration on the network!

SCEF


API T8


There is a special document detailing the interaction with SCEF. This API defines a set of data models, resources, and related procedures for transmitting non-IP data.



Work with SCEF and service subscriptions - JSON (JavaScript Object Notation)


Data that includes SCEF configuration and is transmitted via HTTP / 1.1 to SCEF must be encoded in JSON format, and the body of the HTTP request itself in the Content-Type field must include the “application / json” header. At the same time, if the message was delivered to the recipient and the delivery confirmation was received - SCEF must delete the appropriate configuration, send the message via HTTP POST for AS with the status code “200 OK” and turn on the report on the event.

Postman


I will not delve into how to work with this utility, on the Internet you can find a lot of reviews. I can only say that it has some limitations in its free version, but we don’t need much, the functionality supplied for free is more than enough for our tasks.

After we downloaded it and opened it, the first tab will welcome us - Launchpad, where we will be offered to create our first request, we will not refuse.


Immediately proceed to the configuration of our new device.

Initially, we have configured the GET method, switch it to POST (a little later it will become clear why this is necessary). On the “Authorization” tab, enter the “credentials” we have on hand:


Now create our first request:


In the request body, we indicate:

{
    "externalId": "<ID >",
    "reliableDataService": false,
    "pdnEstablishmentOption": "WAIT_FOR_UE",
    "duration":"2020-12-31T23:59:59Z",
    "notificationDestination": "http://<IP >:<port>"
}

Important!
:
  • “duration” – duration SLA SCEF (SCSAS/ID) , SCEF ,
  • “maximumPacketSize” – /

Let's immediately note that the following options are possible in “pdnEstablishmentOption”:
  • WAIT_FOR_UE - buffer if the device is unavailable (not registered on the network or in the PSM or in another state)
  • INDICATE_ERROR - immediately respond with an error if the device is unavailable
  • SEND_TRIGGER - use Device Triggering service (alternative delivery channel via SMS). This article is not considered.

We use the same parameter to send data to the device. And when creating a subscription, we can immediately send data to the device, reducing the number of API requests.
All! You can click the Send button and carefully examine what we get in response from SCEF:

{
    "externalId": "<ID >",
    "duration": "2020-12-31T23:59:59Z",
    "notificationDestination": "http://<IP >:<port>",
    “reliableDataService": false,
    "maximumPacketSize": 500,
    "self": "/3gpp-nidd/v1/<ScsAsID>/configurations/<ID >",
    "status": "ACTIVE"
}

In the self line, we are primarily interested in the ID of the configuration we created, it is extremely undesirable to lose it, since most likely the operators will not support the request function of all created configurations. When there are several thousand devices created within the framework of one ScsAsID, too much load will be created on SCEF servers to broadcast all device configurations. We take it as a rule: one device = one subscription as part of the ScsAsID service.

Thus, we can already receive data from the module on the server, the IP address and port of which we indicated above.

Data transfer from module to AS


Let's try to transfer data from the device to AS, return to the SIM7020E module and if we did not touch it from the previous chapter and did not turn it off, we will send the command to it:

AT+NIDD=3,<account_id>,"6162636465"
OK

Please note that our message is encoded in HEX and contains a simple sequence: “abcde”.
Almost immediately on the server (AS), which is the host, we will see:

POST / HTTP/1.1
OCSGHTTPProcessor: 147ff7c6-a43d-4fc9-b303-0ca50f497747
X-callback-t8-type: 3
X-callback-url: http://<IP >:<port>
X-api-network-service-node: 0
Content-Type: application/json
Content-Length: 214
Host: <IP >:<port>
Connection: Keep-Alive
User-Agent: Apache-HttpAsyncClient/4.1.3 (Java/1.8.0_181)

{"externalId":"<ID >","niddConfiguration":"/3gpp-nidd/v1/<ScsAsID>/configurations/<ID >","data":"YWJjZGU=","reliableDataService":false}

The message itself arrived in Base64 encoding and looks like this:
YWJjZGU=

If we translate from Base64 encoding, we get our original message:
abcde

Base64 encoding is used due to the fact that when using ASCII encoding, some data is sometimes lost and using Base64 makes transmission more reliable.

It should also be noted that in this case the information transmitted from the module is not stored inside SCEF and is immediately transmitted to our server via HTTP.

Data transfer from AS to module


In order to send data in the direction from our AS to the module, let's go back to Postman and create a new request using the POST method and Base64 encoding. We will send 1234567890 (in Base64: MTIzNDU2Nzg5MA ==) to our module:



Please note that an addendum has appeared in the "address" field, in which we indicated for which configuration we want to send the message. If several devices are included in our configuration, you can use the identifier “externalGroupID” and then all of them will receive this data. Another important point is the lifetime of the sent message, indicated in seconds and has a fairly wide range.

By the way, if the device is not online at this moment, then our message will be buffered on SCEF, and the line “maximumLatency” will tell us how many seconds are left until the message is destroyed if the device does not get in touch before the timer we set . Below is what the requested current SCEF configuration will look like (using the GET mechanism, more on that below):

{
    "externalId": "<ID >",
    "duration": "2020-12-31T23:59:59Z",
    "notificationDestination": "http://<IP >:<port>",
    "reliableDataService": false,
    "status": "ACTIVE",
    "niddDownlinkDataTransfers": [
    {
        "externalId": "<ID >",
        "reliableDataService": false,
        "data": "MDEyMzQ1Njc4OTA=",
        "maximumLatency": 96,
        "priority": 1,
        "self": "/3gpp-nidd/v1/<ScsAsID>/configurations/<ID >/downlink-data-deliveries/1"
    }
}

If, after the timer expires, the device still has not contacted, SCEF will notify your server (AS) that the message was not delivered due to the timer expiration and the message will be deleted:

POST / HTTP/1.1
OCSGHTTPProcessor: 14c8cab8-ecce-4868-a59e-1f784224518b
X-callback-t8-type: 4
X-callback-url: http://<IP >:<port>
X-api-network-service-node: 0
Content-Type: application/json
Content-Length: 139
Host: <IP >:<port>
Connection: Keep-Alive
User-Agent: Apache-HttpAsyncClient/4.1.3 (Java/1.8.0_181)

{"niddDownlinkDataTransfer":"/3gpp-nidd/v1/<ScsAsID>/configurations/<ID >/downlink-data-deliveries/1","status":"FAILURE"}

If successful, SCEF will immediately respond:

{
    "externalId": "<ID >",
    "reliableDataService": false,
    "status": "SUCCESS"
}

You can add the ability to prioritize messages in case they are buffered. It is regulated by the “priority” parameter. When a new message is sent to the device and if the delivery buffer is exceeded by SCEF, the message will be replaced with a message with a lower priority, otherwise the message will not be accepted for delivery. It is also possible to delete a message from the buffer.

If the message cannot be delivered, and it is buffered, you will receive the following:

{
    "externalId": "<ID >",
    "reliableDataService": false,
    "self": "/3gpp-nidd/v1/<ScsAsID>/configurations/<ID >/downlink-data-deliveries/2",
   "status": "BUFFERING"
}

Important!
:
/<ID >/downlink-data-deliveries/1
– SCEF. , . .

Upon receipt of the message, the module will report the following URC:

+NIDD=4,0,11,0,"3031323334353637383930"

Which in translation from HEX to ASCII will be our message:

1234567890

Just leave it here. Notification of delivery of a "buffered" message:

{
"niddDownlinkDataTransfer":"3gpp-nidd/v1/<ScsAsID>/configurations/<ID >/downlink-data-deliveries/1",
"status":"SUCCESS"
}

Monitoring the status of the device status is possible through another SCEF service called “Monitoring Event (MONTE)”. Within MONTE, it is possible to receive notifications about events and time (for example, when the device becomes available), using a similar subscription system. But let's talk about this another time.

Getting configuration from SCEF


You probably noticed that you can get the current configuration from SCEF. Let's do that. We take the Postman we have already loved and create the following request using the GET method:


Those. we leave the body of the message itself empty, and in the address bar we just need to specify the ID of the configuration created for us in order to receive its current state in response:

{
    "externalId": "<ID >",
    "duration": "2020-12-31T23:59:59Z",
    "notificationDestination": " http://<IP >:<port>",
    "reliableDataService": false,
    "status": "ACTIVE",
    "niddDownlinkDataTransfers": []
}

Deleting a configuration on SCEF


Well and the last - we will delete the configuration created by us. To do this, use the same address bar as in the current configuration, but change it to DELETE:


In response, we will receive:

{
    "externalId": " <ID >",
    "duration": "2020-12-31T23:59:59Z",
    " notificationDestination": " http://<IP >:<port>",
    "reliableDataService": false,
    "status": "TERMINATED"
}

Where in the line “status” we will see that the configuration created by us is deleted.

Conclusion


More than one dissertation can be written on the topic of using SCEF, the topic is extensive and will soon become extremely relevant for many M2M devices in all areas, and especially for the Internet of things. The main thing I wanted to convey to you was how to start working with NIDD and SCEF out of the box so that you can figure it out on your own. But also I am always happy to help you: support@simcom.com (marked for RUS Team), in the letter you must indicate your contact details and a few words about your project.

In the following articles we will carefully analyze other aspects of working with cellular modules, and you write in the comments which topic will be of interest to you.

I want to say special thanks to Sergey Novikov, the senior expert of converged solutions and multimedia services.sanov) from MTS for invaluable assistance in preparing the article.

Sources used


NB-IoT: how does it work? Part 3: SCEF - a single window of access to the services of the operator
ETSI TS 129 122

More articles:


All Articles