System Architecture of the proxemic-aware IoT system
System Architecture
For multi-device support used centralized architecture, where all devices inform the server about their state in real-time. Subsequently, the server changes the states of all other devices accordingly. Moreover, the centralized system has a simpler design that excludes any consensus problem. All devices communicate with each other through the server that makes decisions.
Module Description
MyMascotApp is an Android application that has the following responsibilities:
- The application registers the phone in the system: the ID of a beacon that it is attached, mascot custom name, the personality of a mascot.
- The application measures the distance from the device-phone to all other beacons in the system.
- The application vibrates the phone with the vibration duration that the server sent to it.
MyTabletApp is an Android application that has the following functionalities:
- The application registers the tablet in the system, namely ID of a beacon that it is attached to it.
- The application changes the background color of a screen to a color that is retrieved from the server.
- Every second, the tablet application polls data from the server, where as a response it gets specific color code.
server is a server application that has the following features:
- Manages a database that consists of three tables (devices, distances, and personality).
- Implements controllers that handle the client requests such as post requests of device registration, distances, and get requests of required data from the database.
- Server checks whether the distance of all devices falls into the predefined distance range of the Proxemics theory.
- When the user reaches the lamp, the server requests Philips Hue API to change color based on the personality retrieved from the database.
- When the user approaches speakers, play audio files concurrently using the Audio File Play utility or afplay.
Software stack and protocols
The following programming languages, frameworks, and APIs were used to implement the system.
For client applications:
- Android framework with Java language
- - — — NSD API — Network Service Discovery
- AltBeacon Library
For the server-side:
- Spring Framework with Java language
- DNS-SD
- Philips Hue API
- Music Playback component
Database and server
Network requests are handled by controllers using the Spring framework on the server-side. It also allows easy data access to simply connecting and working with the database.
For database management, I use an open-source relational database system known as PostgreSQL.
Database schema and models are defined using Hibernate — an ORM (object relation mapping) framework. Hibernate is also responsible for managing database tables in PostgreSQL.
Service discovery
The way android clients discover a server is by using mDNS implementation on Android called NSD API.
NSD API (Network Service Discovery) is an Android implementation of Multicast DNS.
- In order to make requests the Android applications need to know the IP address and port of services. NSD API helps the client to discover the server. NSDHelper.java module allows the client-applications to find an HTTP server in the local network that supports services that clients are interested in. In our case, we specified _socialiot._tcp type of service.
DNS-SD is an implementation of mDNS protocol on macOS.
- In this project, it is used to register and publish the server information, such as the port number and the service name. Here is an example of how it can be used to set up the AutonamousThesisProject:
dns-sd -R mythesis _socialiot._tcp local 8080
- In the command specified above, mythesis is a service name; _socialiot._tcp is a service type; 8080 is a port number with domain type local. The parameter _socialiot._tcp must match, since the client applications using NSD API will start looking for this specific service type.
Registering devices in the system
Every user can register its mascot in the system by running MyMascotApp and following the instructions on the screen. The detailed description of how to configure the smartphone for the mascot application is given here:
The registration of such devices as the tablet, the lamp, and the speakers must be performed by the developers using MyTabletApp.
So, the tablet application is the device that registers all other devices with a fixed location. Also, the tablet application registers in the system all five personality traits with their associated actions such as color, music type, and the vibration level. The registration of all devices and the personality traits can be performed by following the instructions on the screen. This page shows the information of the data that will be sent to the server about the personality traits and associated actions.
Measuring distance between devices
AltBeacon Library is a library that provides APIs to interact with beacon tags. Beacon tags are broadcast-only Bluetooth Low Energy (BLE) radio transmitters which communicate by transmitting their proximity to the Android applications that supports BLE technology. With the help of AltBeacon Library, MyMascotApp detects beacon tags nearby and uses this information to measure the distance. The accuracy precision that MyMascotApp can measure the distance is around a half meter with a ten-second margin for new positions.
The communication between two mascots
The communication between devices and the workflow of the system is described in this figure. It shows the interaction between two phones including requests that clients and a server send each other.
After the client application finds the server in a local network, it looks for all nearby beacon tags. When the user types the custom name, chooses the personality, and the beacon for his mascot, the client application registers all this information in the system. The client application starts to measure the distance to all beacon tags that are located near other devices. The distance information is constantly sent to the server, where as soon as mascot approaches the other mascot within 45 cm, the server retrieves the personality information of approaching mascot with the associated vibration duration from a database. Afterward, the server passes the vibration information to the client, which in turn triggers the phone to vibrate with an invoked vibration level.
Working with lights
The interaction between mascot and lamp happens using Philips Hue API.
Philips Hue API helps the server to control the hue system. The workflow of the mascot-lamp interaction is similar to the mascot-mascot interaction (see picture above) and all other interaction types. The client measures the distance from the Android application to all beacons in the system and constantly sends this information to the server. The only difference is the use of Philips Hue API. When the server encounters that the distance between mascot and lamp fits the Proxemics theory i.e. x<45, it retrieves such parameters as hue, brightness, and saturation from the database and sends them to the Hue bridge through Philips Hue API. With the help of API, it can directly input commands such as changing light color which in turn will trigger the lamp to change its lighting color. All Philips Hue lamps need to be connected to the Philips Hue Bridge via an open standards protocol called ZigBee Light Link which in turn helps devices to communicate with each other via the internet. This picture shows the visual representation of the mascot interacting with the lamp.
The important note for the implementation part is that all devices such as Philips Hue Bridge, server, phones, and tablet have to be connected to the same network.
Technical Requirements and Hardware
The system comprises of the following devices: the phone with an attached beacon tag, the lamp with a beacon tag, the tablet with a beacon tag, and speakers with a beacon tag. Using the beacon tags that are located right next to devices, Android applications measure the distance from themselves to all other devices. Additionally, the following prerequisites have to be met:
- Philips Hue lamps to be connected to the local network.
- The server must be running on macOS to use afplay utility.
- In order to use Bluetooth BLE, the operating system of the android application must be higher than 6+.
- The android application with the location and the Bluetooth permissions granted.
