The Afero Linux SDK supports Over-the-Air (OTA) image replacement of the entire (full) system on an Afero Linux-based device. This page describes the steps required for a third-party developer to manage the Linux device full OTA image update process.
This page contains the following sections:
The development of the Linux device software is a collaborative process between a third-party developer and Afero. Through the Afero Secure Linux Device SDK, Afero provides a Linux-based IoT platform and is responsible for the network connectivity management components. However, the development of the Linux device and its software is done by the third-party developer. This includes some of the work required for the OTA image update. As part of the platform, Afero provides a cloud-based OTA service and update tools that enable the third-party developer to upload an OTA image file to the Afero Cloud OTA service and download the image securely to the device.
From the Afero Cloud point of view, an OTA image is an opaque binary image; that is, the meaning of the image contents is unknown and ignored. It is up to you to define what that data means and how you package it.
Behind the scenes, Afero adds a header to your OTA image. The header contains a signature that allows hubby to determine if someone has tampered with the image. If it has been tampered with, hubby will ignore the image. The header also contains the OTA Version, which is assigned by the Afero Cloud so it can determine if the firmware on the device is up-to-date. If the firmware is found not to be up-to-date, the OTA service pushes the OTA image to the device.
The diagram below shows a high-level view of the journey of an OTA image from your system to the OTA service to the device:
Following the illustrated flow:
The image creation and deployment steps are detailed on this page; the steps taken by hubby are described in the Secure Linux SDK documentation, particularly the otamgr daemon section.
The solution described on this page assumes that the third-party developer has downloaded the Afero Secure Linux Device SDK, customized it for their development, then built a final software image. As part of the development work, they have also developed a system update strategy. Typically, this includes building an OTA update image and deciding how to manage the actual system update. Particularly on the embedded Linux system, the developer must decide whether to provide different partitions for system update or do a filesystem update. This decision dictates subsequent steps on how the device is updated. In the Yocto build environment, an entire filesystem is generated as the output of build system, which allows us to know the storage size of the image for a particular product. In a later section, we will discuss the update options and our suggestions; however, the final decision of how the OTA image is updated is left to the third-party developer.
An OTA update is a mechanism for remotely updating internet-connected hardware with new settings, software, and/or firmware. The OTA update mechanism is a core part of a system’s architecture, with the remote hardware device being responsible for identifying and applying updates to itself, and the Cloud service responsible for distributing updates to its connected hardware clients. It is important that system update by the OTA mechanism works properly to avoid "bricking" the device. It is important that system update by the OTA mechanism works properly. In this section, we will outline an approach to OTA update capabilities.
The embedded Linux system consists of the following components: bootloader(s), kernel, root filesystem, application, and user data. The OTA update design falls into one of these categories:
In both cases, the user data must be preserved and the image must be authenticated.
The option selected depends on memory space availability, as well as other factors. Sometimes, a single OTA image may be the practical choice. However, if there is enough memory space, it may be desirable to build a fallback mechanism in case of an update failure; that is, create two non-volatile storage partitions for storing the software images - one active, and one for update. In this case, the second partition is updated with the OTA image, and then verified before the system is switched to use it. In case of failure, the system continues to use image in the previous working partition.
The advantage of using two partitions is that it allows the system update to be done as a whole and validated before use. The disadvantage is that it requires doubling the non-volatile storage size.
We strongly recommend the two partitions approach to the full OTA image update. Ultimately, the OTA design is a business decision, and as the third-party developer, you have full design control of the OTA architecture and how the system update is performed. However, there are few things you need to know about Afero Cloud OTA service in order to integrate the process effortlessly.
Follow the instructions outlined below to perform a full OTA image update. (The steps include the use of the Python script tools Afero provides. These scripts are described in detail in the section, OTA Python Tools Reference).
This section focuses on how to create your OTA image, including integrating the OTA image creation with your build process.
Set up the build environment using the following instructions, as appropriate:
Note! The instructions below assume you are developing on an Atmel SAMA5D2 board. If you are on a BeagleBone Green Wireless board, you can use the instructions below, but substitute the appropriate directory paths. If you have questions, contact the Afero Customer Enablement Team (“ace@afero.io”).
$ cd ~/sama5/poky $ source oe-init-build-env build-microchip
$ bitbake -e | grep ^TMPDIR TMPDIR="/home/john/sama5/poky/build-microchip/tmp"
tmp
directory; if it doesn’t already exist, create it:
$ mkdir -p tmp $ cd tmpNote that we are using the
tmp
directory as a workspace for the OTA update tools.tmp
directory, then you only have to do this once:
$ git clone git@github.com:AferoCE/partner-hub-ota-tools.git
You should see the following files in the directory:
$ cd partner-hub-ota-tools $ ls -la drwxr-xr-x 8 john dialout 4096 Oct 28 13:58 .git -rw-r--r-- 1 john dialout 622 Oct 16 16:03 partner-ota-conf.json -rwxr-xr-x 1 john dialout 7731 Oct 28 13:29 partner-ota-hub-deploy.py -rwxr-xr-x 1 john dialout 14244 Oct 28 13:51 partner-ota-hub-uploader.py
Read details in About the Configuration File. Use your favorite editor to edit the configuration file partner-ota-conf.json
. Fill in the following fields and save it:
$ vim partner-ota-conf.json { "Comments": "Configuration file for partner Full OTA image upload tool", "description" : "<OTA Image description>", "name" : "<Your name for this product>", "version" : "<x.x>", "url" : "<dummy>", "username" : "<your_username@your_company_url>", "userpw" : "<account_password>", "auth-string" : "<auth string for your account>", "deviceTypeId": "<your_devicetypeId>", "partnerId" : "<your_partnerID>", "imageFiles" : { "A": "hub_update.bin" } }
Contact the Afero Enablement team (“ace@afero.io”) if you have any problem with this configuration file.
Read details in the reference sections below, About the OTA Record and Using the partner-ota-hub-uploader.py Script.
Run the partner-ota-hub-uploader.py
tool with the option --createOTARecord
to create an OTA record file using the above modified configuration file. Note: You must specify a build number with the -n option:
$ python partner-ota-hub-uploader.py -n <Your BuildNumber> --createOTARecord
Assuming success, the OTA record file is generated and saved; for example, as the following filename and in the following directory:
~home/sama5/poky/tmp/full_ota_record.json
BUILD_TYPE=prod
(as you normally would):
$ cd ../.. $ BUILD_TYPE=prod BUILD_TARGET=debug BUILD_PROFILE=potenco bitbake core-image-minimal
partner-hub-ota-tools
directory:
$ cd tmp/partner-hub-ota-tools $ cp ../deploy/images/sama5d2-xplained-emmc/core-image-minimal-sama5d2-xplained-emmc.tar.gz hub_update.bin
hub_update.bin
), using the same default configuration file and build number as used above:
$ python partner-ota-hub-uploader.py -n <your build number> --uploadOTAImage You are done!
Deploying an OTA image can be independent of the creating the image. In this section, it is assumed that you have already uploaded your OTA image to the Afero OTA service and now you want to deploy this uploaded image to an device. Currently, we only support deploying image one at a time using this tool.
$ python partner-ota-hub-deploy.py --listExample output for the above command is shown below. Note that
imageId
is what you need for the OTA deployment:
$ python partner-ota-hub-deploy.py --list ---- List of HUB FULL OTA images ---- partnerId : 4f7de484-cf23-478d-90a7-412104d5120b deviceTypeId: a6542896-8464-48e1-b12f-664a57e4e703 Total Number of Images: 2 Image Id Version Name Description ---------- --------------- --------------------- ------------------------------ 49856 1.0.1 Potenco-test OTA Image description 49867 1.0.2d Potenco-test OTA Image description
deviceId
from your mobile app. Your deviceId
should look something like this: 012359551bf18eba.partner-ota-hub-deploy.py
with the option -i <imageId> and -d <deviceId> to deploy your OTA images to the device with that ID:
$ python partner-ota-hub-deploy.py -i <imageId> and -d <deviceId>For example:
$ python partner-ota-hub-deploy.py -i 49867 -d 012359551bf18eba
These logs are based on the Atmel SAMA5D2 board. Other development boards should have very similar log entries.
hubby checked the OTA version, and decided that it is okay to start downloading the image: root@sama5d2-xplained-emmc:~# Oct 30 21:11:55 sama5d2-xplained-emmc hubby[2014]: BentoProduct::LocalOtaAllowed(5, 7112): current version 0 Oct 30 21:11:55 sama5d2-xplained-emmc hubby[2014]: DeviceUpdater::DownloadFile(012359551bf18eba, https://otacdn.dev.afero.io/ota-image429612701945115078912d8ee80-b131-4da6-9814-89f9bf22d49c): adding easy 0x2c4410 to multi 0x1765a8 hubby ota function 0::012359551bf18eba::0::59282871 hubby completes the download and stores the image in /tmp/ directory: Oct 30 21:12:55 sama5d2-xplained-emmc hubby[2014]: DeviceUpdater::CheckMultiInfos(): done: https://otacdn.dev.afero.io/ota-image429612701945115078912d8ee80-b131-4da6-9814-89f9bf22d49c => /tmp/ota.update.012359551bf18eba (59282871) 200:0: Oct 30 21:12:55 sama5d2-xplained-emmc hubby[2014]: BentoProduct::LocalOtaComplete(5, /tmp/ota.update.012359551bf18eba) Oct 30 21:12:55 sama5d2-xplained-emmc hubby[2014]: BentoProduct::LocalOtaComplete(/tmp/ota.update.012359551bf18eba): generating sha... Oct 30 21:13:00 sama5d2-xplained-emmc hubby[2014]: BentoProduct::LocalOtaComplete(/tmp/ota.update.012359551bf18eba): sha generated d5dedf43b14c1c5ef390bc1b15f6e5ac3653c2c9d4997c7cc4fba05c86b6ec2d hubby performs verification of the image using the header: Oct 30 21:13:00 sama5d2-xplained-emmc hubby[2014]: 3 verify image Validation is complete: status=0 (success), status=1 (failure): Oct 30 21:13:00 sama5d2-xplained-emmc hubby[2014]: 3 done with OTA update verification Oct 30 21:13:00 sama5d2-xplained-emmc hubby[2014]: BentoProduct::HandleValidateOTA(0) hubby sends out notification (attribute 51612 or AF_ATTR_HUBBY_OTA_UPGRADE_PATH), using attribute daemon attrd: client_set_own_attribute:attrId=51612,name=HUBBY_OTA_UPGRADE_PATH,owner=IPC.HUBBY,value[16]=2f746d702f6f74615f747970655f3500 otamgr received the notification, and starts to act on it: Oct 30 21:13:01 sama5d2-xplained-emmc otamgr[475]: I ota_notification:path=/tmp/ota_type_5 Oct 30 21:13:01 sama5d2-xplained-emmc attrd[363]: 3 handle_receive_message:len=17,seqNum=0000000d:received client message Oct 30 21:13:01 sama5d2-xplained-emmc otamgr[475]: 1 otamgr_image_src:path=/tmp/ota_type_5.img And the script sysupgrade is invoked (if it is available), otherwise, NO actual OTA update occurs: Oct 30 21:13:09 sama5d2-xplained-emmc otamgr[475]: 2 af_util_system::{ /sbin/sysupgrade /tmp/ota_type_5.img; } ; disown -a If sysupgrade is not available, you will see: sh: /sbin/sysupgrade: No such file or directory
Once the OTA image has been deployed to the device and hubby has informed the otamgr daemon (via an Afero attribute) the location of the validated firmware in the filesystem, the otamgr daemon can install the image on the device. Please refer to OTA Manager (otamgr) and OTA Manager Daemon for details.
The sections below describe the Afero OTA Python tools in detail.
The uploader and deploy scripts have the following limitations:
partner-ota-hub-deploy.py
script.Afero provides a Python script to help with the following two tasks:
The script is called “partner-ota-hub-uploader.py
” and is used to generate an OTA record for the full OTA image that you plan to upload, then is used to perform the upload as well. The script is used as a command line tool with input parameters and a JSON configuration file. This tool should be part of the build for the product release. Read more above in Create the OTA Image.
The script interacts with the backend of the Afero Cloud OTA service using the RESTful service API endpoints. The OTA service keeps records on the OTA images and each OTA record is used to evaluate whether a device is eligible for update by examining the version information, among other information. For convenience, some of the information required for the OTA record and access privileges are put in the JSON configuration file.
The Afero Cloud OTA service requires certain information about the OTA image to properly store it and for the OTA service to perform certain checks on it. Specifically, the following is required:
Required Field | Description |
---|---|
createdTimestamp | Time creation in ms. |
description | Easily-readable description of the OTA. Maximum length support is 500 characters. |
id | Unique record ID, used to identify the OTA record. |
name | User-provided short name of the image, up to the maximum length of 35 characters. |
partnerId | Partner ID, created as part of the user account. Go to the Afero Profile Editor Profile Editor, Account Info for this ID. |
tags | Tag name for the release, up to maximum length of 35 characters. |
type | Firmware image type, currently the tool only support HUB (type "5"). |
updatedTimestamp | Last updated time in ms. Same as createdTimestamp when first created. |
url | Storage URL after the OTA image is uploaded. It is an empty string initially. |
version | Friendly name you give this version of the OTA image being released. |
versionNumber | This is the version number that device reports. |
Collectively, this information is referred to as the OTA record. The record contains information from difference sources, some provided by the third-party developer and other from the different subsystems in the Afero ecosystem. Afero provides a script tool that facilitates the creation of the OTA record as well as uploads the image to the OTA service. Read more below.
As part of creating the OTA image, you must customize the configuration file. This file provides the required credentials to access the Afero Cloud OTA service, as well as partner identification information for uploading the OTA images. The file must be in JSON format. A default configuration file is provided in the same directory as the Python script and is named partner-ota-conf.json
.
An example configuration file is shown, with each field described below:
{ "Comments" : "Configuration file for partner full OTA image upload tool", "description" : "OTA image description", "version" : "1.0", "url" : "dummy", "username" : "john@company.io", "userpw" : "12334", "auth-string" : "NGY3ZGU0ODQtY2YyMy00NzhkLTktYTctEREyMTA0ZDUxMjBiOjcxYTllODU5NDYxYjQ2ZjMhNTFhOTE3OWM0Z1234OTRk", "name" : "Potenco-test", "deviceTypeId": "a6542896-8464-48f1-x12g-664a57e4e702", "partnerId" : "4f7de484-b438-578d-90a7-422104d5120i", "imageFiles" : { "A": "hub_update.bin" } }
Field | Description |
---|---|
description | User-friendly description of this OTA file, its purpose, etc. This is used to help the administrator and is not necessary for the OTA service. |
name | Short, user-friendly name that identifies the OTA image. |
version | Friendly name you give this version of the OTA image being released. If the -d option is used in the command, then the letter “d” is appended to the version string, designating a debug build. |
url | Placeholder that eventually must be replaced by the URL to the Cloud storage where the OTA image is uploaded. |
deviceTypeId | The device type ID generated by the Profile Editor for the Profile that identifies your device as a particular type. |
imageFiles | Provides the partition and the final full OTA image filename. Use A for the partition. It is important that the final OTA image filename is the same filename string specified in this field.
|
The following fields are security credentials that are used to access the user’s account. The username
, auth-string
, and partnerId
can be obtained from the Afero Profile Editor. Please refer to Cloud APIs for more details.
Field | Description |
---|---|
username and passwd | Username and password of the account that is used to manage the OTA. |
auth-string | Authorization header value (in string format). |
partnerId | ID that assigned to you when your account is created. This is the "OAuth Client Id" from the Profile Editor, Account Info screen. |
partner-ota-hub-uploader.py[-h] [-c <config_file>] [-d] [-s] -n <build number>
Option details are provided below:
Option | Description |
---|---|
-h | Displays help text. |
-d | Identifies this as a debug build. |
-s | Skip search for BitBake’s TMPDIR. This option can be used for testing, or by advance users. The script integrates the OTA record into the BitBake build; the output OTA record is stored in the $TMPDIR (of the Yocto build). This option allows the user to skip using the $TMPDIR , and store the OTA record file in the current running directory. |
-n | The build number from the build server for this build. For example, Jenkins build server starts each build with build number one (1), and increment it by one for each subsequent build. |
-c | Specifies the use of the default JSON configuration filename and location: |
This Python script is intended to be used in two different stages: 1) creating the OTA record, and 2) uploading the OTA image.
The same options should be used for both the createOTARecord
and uploadOTAImage
calls.
The OTA record created with createOTARecord
call is intended to be subsequently used in uploadOTAImage
; that is, the full-ota-record.json
is used by uploadOTAImage
.
Stage 1. Create the OTA Record
You must create an OTA record before building the actual image. An example of how to use the Python script is given below, where the build number is 10 and a user-provided JSON configuration file:
$ python partner-ota-hub-uploader.py -c dev-john-ota-conf.json -n 2 --createOTARecord
The output of this command is the OTA record, stored in a JSON file: full-ota-record.json
in Yocto’s BitBake ${TMPDIR} (i.e., /home/poky/build-microchip/tmp).
The Afero recipe automatically looks for the full-ota-record.json
file in ${TMPDIR} during the build. In this case, the full-ota-record.json
is automatically picked up by the BitBake recipe to install the file in the device /etc/
directory, or you can manually copy the file to the build image’s rootfs /etc/
directory using the -s
option.
An example of the OTA record in the full_ota_record.json
file is given below:
$ cat full_ota_record.json { "createdTimestamp": 1572280052532, "description": "OTA description", "id": "11561", "name": "Potenco-test", "partnerId": "4f7de484-cf23-478d-90a7-412104d5620b", "tags": [], "type": 5, "updatedTimestamp": 1572280052532, "url": "", "version": "1.0.2d", "versionNumber": "7112" }
Stage 2. Upload the OTA Image
After you have built the image, injected the full-ota-record.json
file in the appropriate place in the filesystem, and packaged the OTA image, the OTA image must be uploaded to the Afero OTA service using the command:
$ python partner-ota-hub-uploader.py -c dev-john-ota-conf.json -n 2 --uploadOTAImage Upload the OTA Image ..... Update OTA Record with the storage URL ..... Associate the Image with the deviceTypeId and ParnerId .... Done!
Afero provides the Python script, partner-ota-hub-deploy.py
, that allows a third-party developer to deploy an uploaded OTA image from the Afero Cloud OTA service. To deploy an OTA image to a device, you need to know the ID of the OTA image that you uploaded to the OTA service. The --list
option displays the images for a specified partnerId
and a deviceTypeId
(which is defined in the configuration file).
Note that you need the same configuration file that you used to create the OTA record and upload the OTA image to now deploy the image. Specifically, for deployment, the following configuration fields are required:
To list the uploaded OTA images for a particular partner and a particular device type, use the script’s --list
option, as shown in the example below:
$ python partner-ota-hub-deploy.py -c dev-john-ota-conf.json --list ---- List of DEVICE FULL OTA images ---- partnerId : 4f7de484-cf23-478d-90a7-412104d8120b deviceTypeId: a6542896-8464-48e1-b12f-664a57e6e703 Total Number of Images: 2 Image Id Version Name Description ---------- --------------- ---------------------- ------------------------------ 49856 1.0.1 Potenco-test OTA image description 49867 1.0.2d Potenco-test OTA image description
$ partner-ota-hub-deploy.py[-h] [-c <config_file>] -d <deviceId> -i <imageId>
Option details are provided below:
Option | Description |
---|---|
-h | Displays help text. |
-l | Displays a list of OTA images on the OTA service. This shows the |
-c | Specifies the use of the default JSON configuration filename and location: |
-d | Specifies the |
-i | Specifies the uploaded OTA |
Below is an example of deploying an OTA image, using deviceId
= 012359551jr17eba, OTA imageId = 49712:
$ python partner-ota-hub-deploy.py -d 012359551jr17eba -i 49712 --conf dev-john-ota-conf.json Request accepted for processing