Keywords

1 Introduction

Portugal is a country of older adults and is the second most-aged country in the European Union. According to the Census 2021 report, a study conducted by INE (National Statistics Institute), the aging index in Portugal translates into 182 elderly for every 100 young people, contrasting with 128 elderly for every 100 young people in 2011 [1].

The last Senior Census [2], conducted in 2021, under the responsibility of GNR (National Republican Guard), a security force that acts precisely in the prevention and protection of people and property, reported 44,484 elderly living alone and isolated, or in a situation of vulnerability, due to their physical, psychological, or other condition that could jeopardize their safety. These older adults are concentrated mainly in the country’s interior, around 50%. Let’s consider the highly depopulated Alentejo region. This figure rises to about 64%, i.e., the most depopulated areas of mainland Portugal concentrate the most significant number of older adults in situations of risk in terms of health, security, and social abandonment.

Compounding the already worrying situation, we had the pandemic of COVID-19, which had a significant impact on the lives of older adults, especially those living isolated in remote areas without home support or permanent medical care. These people faced additional challenges due to mobility restrictions, lack of access to medical and social services, as well as the increased risk of other diseases that high restrictions in terms of mobility and medical assistance may have contributed to the worsening health conditions of the population, who reside in such conditions.

The article is organized as follows: in Sect. 2 we analyze recent works related to the theme; in Sect. 3 we present the framework and design of the SVBox system; in Sect. 4 we show the proof of concept, using the development of the prototype, with the integration of several microcontrollers and sensors, in a controlled environment and in Sect. 5 we present the conclusions and point the way for future work.

2 Related Work

Several authors have contributed to monitoring people and goods, with relevant works in this field, with complex applications. In this field, one of the first works was proposed in [3], in which the authors developed a system for tracking and monitoring the health of older adults living at home. The system was composed of an infrared communication device connected to several sensors whose data were collected using software installed on a personal computer at home. The collected data was transferred to a server over the Internet using a cable television (TV) connection. In this study, one person was monitored for approximately 6 months. The objective was to detect, throughout this time, atypical days when the older woman went out of her daily routine, and the results were presented with the suggestion of the system as a complement for remote rehabilitation monitoring as assistive technology.

More recently, in [4], the authors present a system that proposes itself as a SecureHome TV ecosystem, a technical solution based on the interaction of older adults with their TV set, one of the most used appliances in their daily lives, acting as a non-invasive sensor that allows the detection of possible risk situations through an elaborated alert algorithm. The authors describe the SecureHome TV ecosystem, emphasizing the alert algorithm, and report on its validation process. The algorithm detects the most dangerous situations, contributing to the monitoring of the elderly’s well-being at home, requiring, however, the contracting of a television and cable communications service, with the installation of the software in the television service box.

In previous works, namely [5] and [6], solutions were presented that allow monitoring of the health conditions of older adults using technologies supported by LoRa (Long Range) communication systems. LoRa (Long Range) communication networks can improve the living conditions of older adults living in remote and isolated areas. They are especially suitable for long-range and low-power consumption applications, making them a viable option for remote monitoring and communication in areas with little telecommunications infrastructure.

In [6], we proposed the development of an infrastructure based on LoRa networks supported by a LoRa gateway and an environmental conditions monitoring node only for signal strength testing. We showed the feasibility of this system for use in different contexts, particularly in monitoring older adults.

In [5], we extended the concept and its applicability to a practical case for outdoor monitoring of people, using a vest, with equipment with vital signs monitoring and GPS sensors, accelerometer, and temperature, among others, to prevent falls, disorientation, etc., sending SMS messages to family members with alert information, when problems occur, for example when it moves away from the residence beyond the configured radius or polygon. This system can be connected to TTN (The Things Network), allowing the data to be analyzed practically in real time. However, LoRa networks have several data transfer rates and sending frequency limitations. This leads to the fact that part of the system proposed above, namely sending SMS to family members or security and civil protection forces, is supported on 3G/4G networks.

3 System Design

Following the work that has been developed, in this section we present the framework and motivation that define the scope of this article, as well as the conceptual scheme and the monitoring areas on which this work focuses.

3.1 Framework and Motivation

The role of information systems and technologies, particularly monitoring systems, can play a key role in improving the living conditions of isolated older adults who, even after the end of the pandemic, will continue to suffer for a long time, not to say irreversibly, the consequences associated with the restrictions arising from COVID-19. These systems could benefit communities living in remote and isolated locations, especially the elderly population, notably through:

  • Remote health monitoring: Monitoring systems can allow older adults to take measurements of vital signs, such as blood pressure, heart rate, and blood oxygen levels, among others, from home. This information can be transmitted to health professionals who can keep track of the user’s health status and intervene in an emergency.

  • Virtual communication: Communication technologies such as video calls and instant messaging can help combat social isolation. Seniors can connect with their family members, friends, and healthcare professionals, making them feel safer and more supported, even at a distance.

  • Safety monitoring: Sensor networks linked with smart devices can be installed in homes to monitor the safety of older people. For example, motion sensors can detect falls or unusual activity and alert family members, emergency services, or law enforcement.

  • Access to information and services: Information technology can provide up-to-date health information, relevant news, and local services available to seniors, such as pharmacy opening hours, health clinics, and more, helping them make decisions and find needed resources without leaving home.

  • Telemedicine and remote consultations: One field that has had the most relevance during the COVID-19 pandemic has been virtual medical consultations, allowing seniors to get medical care and advice without needing physical travel. This primarily benefits those living in remote areas without easy access to medical services.

However, information and communication technologies naturally have excellent resistance in their adoption, both by the population in more depopulated and isolated areas, since these are older adults with high, not to say total digital illiteracy, having as their main point of contact with information and as a companion, open signal television or paid cable, fiber, ADSL or satellite television services, so this equipment naturally emerges as the device that motivated us in this work, and that led us to try to answer the main question: How can we support the monitoring of older adults and their homes, especially those living in remote and isolated areas, integrating security alerts and warnings, both locally and remotely?.

3.2 System Concept

The present work has developed a device called SVBOX for active surveillance, which integrates a video entry system with a network of internal sensors to the house and sensors for monitoring the vital signs of older adults. The system has an interconnection point with the home’s television to maintain a permanent state of alert with resources of communication and telecommunication means.

The operational objectives of the device are:

  • Integration of the housing sensor network for fire detection, flooding, and dangerous gases (CO2, CO, combustible gases), among others, with biometric sensors for individual use (Heartbeat, Temperature, fall detection), in a system of collection and processing of data in Real Time;

  • Integration of the Video Intercom with the TV so that it works as an intercom (larger vision area, better audio capacity);

  • Establish the concept of e-Neighbor, as an active neighborhood, permitting certain people to access and receive information from older adults whenever some anomalous situation occurs.

Figure 1 shows the operational scheme of the system, as well as the elements that constitute it. It should be noted that the proposed system is independent of any telecom operator, open and low-cost, and can be implemented with low-cost devices.

Fig. 1.
figure 1

Conceptual scheme

Full size image

The operation of the proposed system is quite simple, focused on day-to-day functionalities, where older adults, in their daily routines, will have in the SVBOX a companion, expanding the concept of active surveillance to assistive technologies, supported in IoT (Internet of Things), integrating the concept of (e-caregiver) [7].

3.3 Operationalization of the External Surveillance System

Whenever someone touches the Video Doorbell, the call is established with the TV, and the answering and conversation can be made by the user and the outside, using the microphone built into the SVBOX or the remote control.

The SVBOX should have a specific command that allows sending an automatic call to the emergency (panic button type) if something is not going well, and in this case, may have a payment plan for the use of GSM low cost (similar to the devices provided in Portugal by law enforcement and civil protection). Alternatively, the possibility of sending a message via the LoRa network (free of charge) to active neighbors (e-Neighbor), in a policy of solidarity and proximity support, when the family members do not live close by. In this case, it should be considered that LoRa communication is low-speed and is recommended for sending small data packets of up to 5 km (in ideal situations). This critical scenario was analyzed in previous works, namely in [5, 6], as mentioned above.

The option of recording communication with the outside world can be activated, always subject to the rules and restrictions imposed by the GDPR (General Data Protection Regulation) in force in the European Union [8].

3.4 Indoor Monitoring

When any sensor (from the sensor network) detects an anomalous value, the SVBOX will switch to the TV and show the message on the screen with an image of the danger level (reminder, Warning, Danger, etc.), accompanied by a beep, which will only be turned off when the resident presses a suspend button, to correct the possible problem (configurable). If the problem and the level of risk remain, it will be activated again after a certain period. To control the alerts for possible false positives, the system turns off the alarm after some time, also configurable, even without human intervention, or if the situation has been restored to normal values. However, if the condition persists, the alerts will be activated again.

The requirement of a connected TV is complemented by a small LCD and sound equipment always on and integrated with the SVBOX, redundant, which will remain active even if the TV is turned off, also being a solution for cases where the TV has no HDMI interface (homes with old equipment, where it is not possible to integrate with the SVBOX).

A UPS should allow the SVBOX system and the sensor network in the house to remain operational for some time, even if the power fails (no connection to the TV).

The sensors for vital data monitoring and fall detection will be for individual use, being sent to the SVBOX via Wi-Fi/Bluetooth, which will analyze them in real-time, and, if it detects any anomalous occurrence, it should send an alert signal to connected civil protection entities, neighbors (e-Neighbor’s) or family members, for help and emergency support. This communication may be via SMS or 112 calls, as already happens with the entities connected to civil protection in Portugal, which try to establish verbal communication with people needing assistance screening [9].

4 Proof of Concept

A prototype consisting of several components was developed for the proof of concept. The devices were interconnected in a local network and configured so that each one had a fixed IP address. Among these devices are a Raspberry PI3, an aggregator, and a main application server.

4.1 Video Intercommunication with ESP32-CAM

The ESP32-CAM [10] is an MCU (Micro Controller Unit) development board module with a small size with an ESP32-S chip and an OV2640 camera, a microSD card slot, and with several GPIOs to connect peripherals. The ESP32-CAM is widely used in various IoT applications, such as smart home equipment, wireless monitoring, and QR code identification (Fig. 2). Its specifications and the ESP32-CAM features can be found on the manufacturer’s website [10].

Fig. 2.
figure 2

ESP32-CAM in a protoboard

Full size image

The main limitation is that it does not have built-in audio. However, given the features and capabilities built into the MCU, it can be used in remote monitoring and video surveillance systems.

For programming the ESP32-CAM, we used the PL2303 TTL USB-Serial converter. This adapter uses Prolific’s PL2303HX chip to convert USB signals into TTL RS232 signals, which makes it worthwhile when we want the computer to communicate with microcontrollers in general.

This converter is a small board with a USB connector at one end to be connected directly to the computer and at the other end a pin bar for easy connection with the board whose communication will be established. Its voltages of 3.3 V and 5 V allow it to work in a broader number of microcontrollers regardless of their power supply.

4.2 AdaFruit M0

Another device we integrated into the prototype was the Feather M0 MCU from Adafruit [11], which has an ATSAMD21G18 ARM Cortex M0 processor with a 48 MHz clock and 3.3 V power supply, the same used in the new Arduino Zero. This chip has 256K FLASH and 32K RAM (Fig. 3). The chip has built-in USB, so it has USB-to-serial programming and built-in debugging without the need for an FTDI-type chip. Its specifications can be found on the manufacturer’s website [11].

Fig. 3.
figure 3

Adafruit Feather M0 Wi-Fi - ATSAMD21 + ATWINC1500 [11]

Full size image

Its application in the prototype focused on monitoring the sensors and sending the data in real-time to the server via Wi-Fi.

4.3 SVBOX System Prototype

In building the prototype, we started by working on the video intercom, the ESP32-CAM, mentioned earlier.

Since the ESP32-CAM does not contain a USB port, it was necessary to use this converter to upload the code via GPIOs. The connection between the microcontroller and the peripheral can be seen in the table below (Table 1).

Table 1. The connection between the microcontroller and the FTDI.
Full size table

After its installation and configuration, we resorted to the Arduino IDE to develop the web server source code for sending images in real time. Considering that at this stage of prototyping, it is unnecessary to configure the internal network, the IP addresses were set as static so that the various connected devices can be easily identified, from sensors and MCUs, among others. In the ESP32-CAM, through the IP address, the camera configuration page displays all the available settings of the OV2640 camera (Fig. 4).

It should be noted that the ESP32-CAM allows facial recognition, so this feature can be used to develop Machine Learning and Artificial Intelligence components that will be integrated into the system for the automatic identification of people.

Fig. 4.
figure 4

ESP32-CAM Web Server

Full size image

It was necessary to use the web application CyberChef [12] to convert code that the ESP32-CAM support software provides in hexadecimal to HTML. This conversion was required to make changes to the OV2640 camera so that it would transmit automatically without needing to be started by a command button.

We used another microcontroller, the Adafruit M0 MCU, to simulate the sensor network. We attached an MQ3 sensor to this microcontroller that allows the detection of certain types of flammable vapors (e.g., ethanol). Still, we could use another sensor, e.g., the MQ2, that enables the detection of toxic gases, such as carbon monoxide and dioxide, receiving different values. The MCU was configured and programmed to work as a web server permanently sending data, which will be parameterized by the Raspberry PI3 so that it can later detect the abnormal values and trigger the necessary alert and warning actions on the TV (Fig. 5).

It should be noted that the MQ3 sensor works efficiently at 5 V, becoming unstable with a voltage of 3.3 V, so it was necessary to guarantee this voltage, available on the MCU’s USB pin.

Fig. 5.
figure 5

MQ3 sensor Web Server

Full size image

During the implementation of the system with the ESP32-CAM, we found that the GPIOs that were supposed to be available for programming the call button (push button) were all occupied for operating the camera and the one intended for the memory slot, so we had to study another solution. To circumvent the problem, we turned to another MCU Adafruit M0, which was programmed in the same way as we had designed for the ESP32 (Fig. 6). Its function is to provide a push button sensor, working as a buzzer, lighting an LED for 10 s. This sensor will also work as a web server, permanently sending data of “button state”, state 0 or 1 (0 being for off and 1 for on).

The Raspberry reading function given by the button automatically makes the switch to its port (HDMI) and shows the camera transmitting what is on the street, with a warning sound signal that lasts for 20 s, that someone is ringing the bell.

Fig. 6.
figure 6

Video Enable and Bell Activation Web Server

Full size image

In designing this work, we resorted to using the VS1838B sensor, an infrared receiver, on which we used a command to turn off the alerts. This way, we can give instructions to the Raspberry, turning the switch on and off, among others that can be added as needed. The TV Switch, given the technical limitations, which prevented us from using a video signal multiplexer, such as the TS3DV421 Multiplexer [13], available only for industry, for the proof of concept, we used the relay connections, having studied the different types of signal in the HDMI cable, opting for a parallel analog solution of 12 relays (only the ones needed for sending signal switching) controlled by the Raspberry which automatically switches to the equipment to be displayed on the TV. The TV Switch has two inputs and one output, the inputs are the TV box, and the other is the Raspberry itself.

The output is connected to the TV set (Fig. 7).

Fig. 7.
figure 7

Analog switch with relays

Full size image

The core equipment of the prototype is the Raspberry PI. The model we used to develop the prototype was the Raspberry PI 3 Model B [14]. In programming the Raspberry PI, the Python language was used.

In the project prototype, the functional scheme has the ESP32-CAM always operational, constantly transmitting on the Raspberry through the graphical interface created using the QT5 libraries and the QT Designer.

We switch between the GUI (contained in a Raspberry) and the operator box through the remote command. The remote control was configured with four options so that it could also work manually. The system automatically switches to the Raspberry interface when we click the ON button. Consequently, when we click the OFF button, the system reverts the previously said process and goes back to the box.

A screen capture button has also been programmed in case the user wants to register some photograph from the outside (Fig. 8-B), for example, of someone who may be in the vicinity of the residence, within the capture range of the camera. When the sensors register an anomalous value, an “Alert” window will be displayed and overlay the graphical interface. Once the user knows the alert, they can close the window using the command button.

In Fig. 8-A and 8-B we can see the two connection interfaces in operation, the TV-DTT (Digital Terrestrial Television) Box and the SVBox in the reception of an external call respectively with 800 × 600 resolution. It should be noted that the image of the face in Fig. 8-B was intentionally distorted.

Fig. 8.
figure 8

TV screen image: (A) normal operation with DTT Box; (B) Interface with external call

Full size image

Once the system can be connected to the Internet, the Raspberry PI, the central heart of the system, can be used for remote access. A family member or the resident can access, via a remote computer or mobile device, the images and data of the sensors collected at any moment.

5 Conclusion and Future Work

The present work proved very important in developing an integrated system called SVBOX. It enables older adults inside their residences to control and monitor their safety and health conditions, using the television as a central monitor. Currently, with the expansion of IoT to particular areas such as home automation, and the monitoring of people and goods, the investigation of new equipment and modes of operation in different contexts becomes a constant challenge for the scientific community. However, the limitations imposed by external situations, as was the pandemic by COVID-19, showed us that we need to strengthen the means of communication and alternative methods, to meet the needs of keeping people safe.

In this work, we found some limitations and constraints for future work, namely the study of other MCUs existing in the market, which facilitate video communication and incorporate audio, which here was by using a conventional voice intercom activated by a push button. Using the TS3DV421 Multiplexer or similar could guarantee excellent quality in the video commutation, which we didn’t get with the analog solution, having the maximum resolution achieved with 800 × 600 VGA. Another limitation is related to the equipment available on the market. Since the project started in the middle of the pandemic, much of the equipment we needed for the project’s development was not available in time, preventing us from implementing the prototype with the features we wanted at the beginning.

In future work, in addition to audio, we intend to include a multiplexer with an I2C interface, such as TS3DV642EVM [15].

Finally, but equally important, the development of a mobile application that allows remote monitoring without using a browser, either by residents when they are away from home or by relatives or neighbors (e-neighbors) who have a relationship of trust and responsibility in monitoring the most vulnerable older adults.