�����������������

Citation: Paul, A.; Tajin, M.A.S.;

Das, A.; Mongan, W.M.;

Dandekar, K.R. Energy-Efficient

Respiratory Anomaly Detection in

Premature Newborn Infants.

Electronics 2022, 11, 682.

https://doi.org/10.3390/

electronics11050682

Academic Editor: Alexander

Barkalov

Received: 13 January 2022

Accepted: 17 February 2022

Published: 23 February 2022

Publisher’s Note: MDPI stays neutral

with regard to jurisdictional claims in

published maps and institutional affil-

iations.

Copyright: © 2022 by the authors.

Licensee MDPI, Basel, Switzerland.

This article is an open access article

distributed under the terms and

conditions of the Creative Commons

Attribution (CC BY) license (https://

creativecommons.org/licenses/by/

4.0/).

electronics

Article

Energy-Efficient Respiratory Anomaly Detection in PrematureNewborn Infants

Ankita Paul 1,*, Md. Abu Saleh Tajin 1, Anup Das 1, William M. Mongan 2 and Kapil R. Dandekar 1

1 Department of Electrical and Computer Engineering, Drexel University College of Engineering,Philadelphia, PA 19104, USA; mt3223@drexel.edu (M.A.S.T.); ad3639@drexel.edu (A.D.);krd26@drexel.edu (K.R.D.)

2 Department of Mathematics and Computer Science, Ursinus College, Collegeville, PA 19426, USA;wmongan@ursinus.edu

* Correspondence: ankita.paul@drexel.edu

Abstract: Precise monitoring of respiratory rate in premature newborn infants is essential to initiatingmedical interventions as required. Wired technologies can be invasive and obtrusive to the patients.We propose a deep-learning-enabled wearable monitoring system for premature newborn infants,where respiratory cessation is predicted using signals that are collected wirelessly from a non-invasivewearable Bellypatch put on the infant’s body. We propose a five-stage design pipeline involvingdata collection and labeling, feature scaling, deep learning model selection with hyperparametertuning, model training and validation, and model testing and deployment. The model used is a1-D convolutional neural network (1DCNN) architecture with one convolution layer, one poolinglayer, and three fully-connected layers, achieving 97.15% classification accuracy. To address theenergy limitations of wearable processing, several quantization techniques are explored, and theirperformance and energy consumption are analyzed for the respiratory classification task. Resultsdemonstrate a reduction of energy footprints and model storage overhead with a considerabledegradation of the classification accuracy, meaning that quantization and other model compressiontechniques are not the best solution for respiratory classification problem on wearable devices.To improve accuracy while reducing the energy consumption, we propose a novel spiking neuralnetwork (SNN)-based respiratory classification solution, which can be implemented on event-drivenneuromorphic hardware platforms. To this end, we propose an approach to convert the analogoperations of our baseline trained 1DCNN to their spiking equivalent. We perform a design-spaceexploration using the parameters of the converted SNN to generate inference solutions havingdifferent accuracy and energy footprints. We select a solution that achieves an accuracy of 93.33%with 18× lower energy compared to the baseline 1DCNN model. Additionally, the proposed SNNsolution achieves similar accuracy as the quantized model with a 4× lower energy.

Keywords: wearable; respiratory classification; deep learning; spiking neural network (SNN)

1. Introduction

A premature newborn infant is one who is born more than three weeks before theestimated due date. Common health problems of these infants include Apnea of Pre-maturity (AOP), which is a pause in breathing for 15 to 20 s or more [1], and NeonatalRespiratory Distress Syndrome (NRDS), which is shallow breathing and a sharp pulling inof the chest below and between the ribs with each breath [2]. Precise respiratory monitoringis often necessary to detect AOP and NRDS in premature newborn infants and initiatemedical interventions as required [3]. Wired monitoring techniques are invasive and canbe obtrusive to the patient. Therefore, non-invasive respiratory monitoring techniques arerecommended by pediatricians to increase the comfort of infants and facilitate continuoushome monitoring [4].

We have studied the use of wearable technologies in the respiratory monitoring ofinfants. To this end, we use the Bellypatch (see Figure 1), a wearable smart garment that

Electronics 2022, 11, 682. https://doi.org/10.3390/electronics11050682 https://www.mdpi.com/journal/electronics

Electronics 2022, 11, 682 2 of 22

utilizes a knitted fabric antenna and passively reflects wireless signals without requiring abattery or wired connection [5–7]. The Bellypatch fabric stretches and moves as the infantbreathes, contracts muscles, and moves about in space; the physical properties of the radiofrequency (RF) energy reflected by the antenna change with these movements. Theseperturbations in RF-reflected properties enable detection and estimation of the infant’srespiration rate [8]. There are other possibilities of collecting heart rate [9], movement ofthe extremities [10], and detection of diaper moisture [11] using RF for medical practice.

Conductive fabric

30

mm

a.

Ground layer

Top layer

b.

RFID chip

135 mm

Figure 1. (left) A smart-fabric Bellyband and (right) Bellypatch can be integrated into wearablegarments to enable wireless and passive biomedical monitoring in infants. (a) top and (b) bottomview of the Bellypatch.

The Bellypatch operates in the 900 MHz Industrial, Scientific, and Medical (ISM)frequency band using Radio Frequency Identification (RFID) interrogation. An RFID inter-rogator emits an interrogation signal multiple times per second (often 30–100 interrogationsper second). In typical RFID technology deployments, only a single interrogation responseis needed for inventory purposes; however, many interrogations are sent to overcomecollisions among the responding RFID tags and other signal interference or loss. We exploitthis redundant interrogation to sample the state of the antenna each time it is successfullyinterrogated. Specifically, we observe changes in the received signal strength indicator(RSSI), phase angle, and successful interrogation rate to estimate respiratory properties ofthe wearer’s state as well as other biosignals. The use of passive RFID enables a wirelessand unobtrusive wearable device that requires no batteries to operate; the interrogationsignal itself is sufficient to power the worn RFID tag for each interrogation. However, pathloss, multipath fading, and collision mitigation among the tags in the field require signaldenoising and interpolation of the received signal as well as intelligent algorithms forsignal processing and estimation. Because these biomedical estimates require real-time ornear-real-time processing and may take place on low-power embedded or portable systems,it is desirable to utilize techniques that place minimal constraints on power consumption,online training, and processing latency.

To this end, we propose a deep-learning-enabled respiratory classification approach.At the core of this approach is a five stage pipeline involving data collection and labeling,feature selection, deep learning model selection with hyperparameter optimization, modeltraining and validation, and model testing and deployment. We use a Laerdal SimBabyprogrammable infant mannequin (see Figure 1) to collect respiratory data using the sensorsattached on the Bellypatch. We use a 1-D convolutional neural network (1DCNN) forclassification of the features extracted from the SimBaby sensor data. The 1DCNN modelconsists of one convolution layer, one polling layer, and three fully connected layers,achieving 97.15% classification accuracy (see Section 6.1). This is higher than state-of-the-art accuracy obtained using machine learning techniques such as Support Vector Machine(SVM), Logistic Regression (LR), and Random Forest (RF), all of which are proposed for therespiratory classification of infants. Hyperparameters of this 1DCNN model are selectedusing a Grid Search method, which is discussed in Section 3.4, and the model parametersare trained using the Backpropagation Algorithm [12]. Our model is trained using therespiratory data on an external work station. We also test the model in the workstation.

Electronics 2022, 11, 682 3 of 22

Since we intend to deploy the trained network on a wearable device, we conduct energyefficiency explorations to find which approach provides the least power consumption andmaximum accuracy. We show that the energy consumption of our baseline 1DCNN modelis considerably higher, which makes it difficult to implement the same on the Bellypatchdue to its limited power availability. Our goal is to minimize the energy consumption, thusallowing more processing within a given power budget. Therefore, we propose severalquantization approaches involving limiting the bit-precision of the model parameters.We show that in order to achieve a significant reduction in energy, the model accuracycan be considerably lower. Therefore, model quantization may not be the best solution toimplement respiratory classification on the Bellypatch.

Finally, we propose a novel respiratory classification solution enabled by spikingneural network (SNN) [13], which can be implemented on event-driven neuromorphichardware such as TrueNorth [14], Loihi [15], and DYNAPs [16]. We perform design-spaceexploration using SNN parameters, obtaining SNN solutions with different accuracy andenergy. We select a solution that leads to 93.33% accuracy with 18× lower energy thanthe baseline 1DCNN model. This SNN-based solution has similar accuracy as the bestperforming quantized CNN model with 4× lower energy. This is particularly useful forwearable devices that are used for bio signals monitoring using less energy such as theHuman++ [17].

Overall, the SNN-based approach introduces two additional stages in our designpipeline: model conversion and SNN parameter tuning, making the overall approach aseven-stage pipeline. Using this seven stage design pipeline, we show that the accuracy issignificantly higher than all prior solutions, with considerably lower energy, making thissolution extremely relevant for the battery-less Bellypatch.

The remainder of this paper is organized as follows. Related works on respiratoryclassification are discussed in Section 2. The five-stage design pipeline is described inSection 3. Model quantization techniques are introduced in Section 4. The SNN approachto respiratory classification is formulated in Section 5. The proposed approach is evaluatedin Section 6 and the paper is concluded in Section 7.

2. Related Work

Recently, machine learning-based respiratory classification techniques have shownsignificant promise as enablers for continuous respiratory monitoring of newborn infants.To this end, a Support Vector Machine (SVM)-based classifier is proposed in [18], achieving82% classification accuracy. A Logistic Regression-based classifier is proposed in [19],achieving classification accuracy of 87.4%. An Ensemble Learning with Kalman filtering isproposed in [19] achieving 91.8% classification accuracy. All these techniques are proposedfor respiratory classification using pulseoximeter data collected from infants, makingthese approaches the relevant state-of-the-art for our work. In Section 6, we compare ourapproach to these state-of-the-art approaches and show that the proposed approach isconsiderably better in terms of both classification accuracy and energy. Thermal imaginghas also been proposed recently for the respiratory classification of infants [20]. The authorsreported a precision and recall score of 0.92. We achieved a score of 0.98. Respiratoryclassification using acoustic sensors is proposed in [21]. An accuracy of 95.7% is reported.We achieved an accuracy of 97.15%.

Beyond respiratory classification, deep-learning-enabled techniques have been usedextensively for health informatics [22]. For instance, sleep apnea classification is proposedusing deep convolutional neural networks (CNNs) and long short-term memory (LSTM)in [23], achieving an accuracy of 77.2%. A deep learning approach using InceptionV3CNN model is proposed in [24] to detect Alzheimer’s disease using brain images. Theauthors reported an area-under-curve (AUC) score of 0.98. CNN models are used in [25] todetect metastatic breast cancer in women. The authors reported an AUC score of 0.994. ACNN-based Arrhythmia classification is also proposed in [26],where the authors reportedsignificant improvement in classification accuracy over state-of-the-art.

Electronics 2022, 11, 682 4 of 22

Finally, many recent SNN-based techniques have shown comparable and in somecases higher accuracy than their deep learning counterparts with significantly lower energy.An unsupervised SNN-based heartrate estimation is proposed in [27]. The authors reporteda 1.2% mean average percent error (MAPE) with 35× reduction in energy. A spiking CNNarchitecture is proposed in [28] to classify heart-beats in human. The authors reported 90%reduction in energy with only 1% lower accuracy than a conventional CNN. SNN-basedepileptic seizure detection is proposed in [29], where the authors reported an accuracy of97.6% with a considerable reduction in energy.

To the best of our knowledge, this is the first work that uses SNN for respiratoryclassification in infants and shows that SNNs can achieve high accuracy (93.33% in ourevaluation) with a considerable reduction in energy (18× lower energy compared to abaseline 1DCNN model).

3. Design Pipeline

Figure 2 shows a high-level overview of the proposed respiratory classification ap-proach using deep learning techniques. The design pipeline comprises five stages—(1) datacollection, (2) feature selection, (3) deep learning model selection, (4) model training andvalidation, and (5) model testing and deployment. These stages are clearly indicated in thefigure. Once a trained model is obtained using this approach, the model is used for therespiratory classification of streaming pulseoximeter data collected from the sensors on theBellypatch. This is shown at the bottom-left corner of Figure 2.

(1) Data Collection (2) Feature Scaling (3) Model Selection (4) Model Training and Validation

(5) Model Testing and Deployment

Respiratory classification with trained model

Figure 2. Design pipeline for respiratory classification using deep learning.

In the following section, we describe the design pipeline stages.

3.1. Data Collection and Labeling3.1.1. Data Collection and Feature Selection

In this work, we use a Laerdal SimBaby programmable infant mannequin. Themannequin was programmed to breathe at a rate of 31 breaths per minute for variabletime intervals, then to stop breathing for 30 s, 45 s, and 60 s, alternating between thesestates for a period of one hour. An RFID investigator (Impinj Speedway R420) was used topoll the Bellypatch wearable RFID tag and the antenna with a 900 MHz band RFID signalcoming from the SimBaby. The RFID interrogator was also used to measure properties ofthe backscattered signal reflected from the RFID tag. The interrogator was positioned 1 footfrom the mannequin, oriented above, astride, and at the feet. Interrogations were performedwith a frequency of 90 Hz. RFID properties considered for model features include theReceived Signal Strength Indicator (RSSI), interrogation frequency, and timestamp.

Each of these properties is affected in-band by the frequency of the original signalemitted by the interrogator. Under United States Federal Communications Commission

Electronics 2022, 11, 682 5 of 22

(FCC) regulations, RFID interrogations must iterate (or channel hop) over 50 frequencychannels in the 900 MHz band. In addition to perturbing the raw measurement observationsat the interrogator, channel hopping poses challenges in computing higher order featuresfrom changes in the observed phase, because these features depend on observing changesin successive values of the phase under the assumption that they were observed from thesame interrogation frequency. As a result, the observed Doppler shift is used to identifyfine movements of the RFID tag, either in space or because of a strain force applied to thesurrounding knit antenna.

The received signal strength from an interrogation is influenced by several factors asdefined by the Radar Cross Section (RCS) formula in Equation (1) [30]. Specifically, the RCSrelates changes in received signal power (PRx) to the interrogation power (PTx), the readerand tag gains (Greader and Gtag, respectively), the return loss (R), and the interrogationwavelength λ [19].

PRx = PTx · G2reader · G2

tag · R ·(

λ

4· π · r

)4(1)

Some of these terms can be controlled by the interrogator configuration: for example,the interrogation transmitter power and antenna reader gain at the interrogator; however,the interrogation frequency changes due to channel hopping, and the receiving antennagain (Gtag) changes as the wearer stretches the antenna or moves about in space. Thus, theobserved RSSI alone confounds several artifacts about the state of the transmission alongwith the state of the wearer. As a result, a higher order feature ζ is computed from the RSSImeasure by accounting for the interrogation frequency.

We manipulate the RCS equation to arrange those terms related to wearer state on oneside and set them as equal to those terms related to the interrogator configuration, as shownin Equation (2). Thus, we observe that the changes in the gain of the tag Gtag (resulting frommovement or a strain force on the antenna), the distance r between the interrogator and thetag (resulting from movement), and return loss R (resulting from movement, strain force,fading, or multipath interference) are proportional to the interrogation wavelength lambdaand observed RSSI measure PRx, along with the interrogation power PTx and the readergain Greader, which are held constant at the interrogator and interrogating antenna [5].Specifically, ζ is defined as the ratio of the interrogation radius to the product of theantenna effective aperture and return loss, as shown in Equation (2), which represents theobserved terms after fixing the transmit power, interrogator antenna gain, and interrogationfrequency, given the observed RSSI of the reflected signal:

ζ = G−2tag · r4 · R−1 = PTx · G2

reader · P−1Rx ·

(λ

4π

)4, (2)

We remove a residual term δ = −10log10f 4

( f−(0.5∗106 MHz))4 ≈ −0.00941 to compensatefor a sawtooth artifact resulting from quantization of the observed RSSI as the interrogationfrequency changes among 50 discrete channels per FCC regulations in the United States.

In summary, we chose the following features for consideration during wireless respi-ratory state classification.

• Feature 1: Reflected signal strength as measured at the interrogator, ζ (PRx−deoscillated)• Feature 2: The difference between the current observed RSSI from the minimum RSSI

value observed in the recent time window (RSSI_ f rom_min)

We normalized RFID signal strength (PRx−deoscillated) data by frequency to utilize thesignal for respiratory analysis. The resulting time-series data were filtered and signal-processed to determine the mean power spectral density, derived from the amplitude ofthe oscillatory behavior observed in the signal during short time windows.

Figure 3 illustrates the two features (PRx−deoscillated and RSSI_ f rom_min) over a timewindow of 1.2 s.

Electronics 2022, 11, 682 6 of 22PRx_deoscillatedRSSI

0.01 9.704198 27.60416 10.02 9.704198 27.51442 20.03 9.704198 27.49245 30.04 9.704056 26.41036 40.05 8.704056 27.14085 50.06 8.704056 27.33658 60.07 8.704056 27.38903 70.08 8.704056 27.40308 80.09 8.704056 27.40685 9

0.1 8.704056 27.40785 100.11 8.704056 27.40812 110.12 9.704056 26.67615 120.13 9.704056 26.48001 130.14 9.703909 22.99332 140.15 9.703909 22.97924 150.16 9.703909 22.97546 160.17 9.703757 26.67202 170.18 9.703757 26.67175 180.19 47.70376 11.68742 19

0.2 45.70376 5.575803 200.21 9.703757 17.61292 210.22 9.703757 24.24437 220.23 9.703757 26.02126 230.24 9.703757 26.49738 240.25 9.703757 26.62495 250.26 9.703757 26.65914 260.27 9.703757 26.66829 270.28 9.7036 26.77647 280.29 10.7036 26.04507 29

0.3 10.7036 25.8491 300.31 9.7036 26.52864 310.32 10.7036 25.97867 320.33 10.7036 25.83131 330.34 10.7036 25.79182 340.35 10.7036 25.78124 350.36 9.703438 27.01953 360.37 9.703438 27.21492 370.38 9.703438 27.26728 380.39 9.703438 27.28131 39

0.4 9.703438 27.28507 400.41 9.703438 27.28607 410.42 9.703438 27.28634 420.43 9.703438 27.28642 430.44 8.70327 27.69022 440.45 8.70327 27.88638 450.46 8.70327 27.93894 460.47 8.70327 27.95302 470.48 8.70327 27.9568 480.49 8.70327 27.95781 49

0.5 8.70327 27.95808 500.51 8.70327 27.95815 510.52 8.703098 28.28277 520.53 46.7031 13.08881 530.54 8.703098 20.82898 540.55 8.703098 26.28554 550.56 8.703098 27.74762 560.57 45.7031 13.67746 570.58 9.703098 20.25466 580.59 8.703098 26.13165 59

0.6 8.703098 27.70638 600.61 8.703098 28.12833 610.62 48.7031 11.58332 620.63 8.703098 20.42559 630.64 46.7031 10.98348 640.65 47.70292 2.849085 650.66 9.70292 15.55172 660.67 9.70292 22.52845 670.68 9.70292 24.39786 680.69 9.70292 24.89876 69

0.7 9.70292 25.03298 700.71 9.70292 25.06894 710.72 9.70292 25.07858 720.73 9.70292 25.08116 730.74 9.70292 25.08185 740.75 43.70274 14.09419 750.76 43.70274 7.425061 760.77 8.702737 18.86422 770.78 43.70274 12.02459 780.79 44.70274 6.13846 79

0.8 8.702737 18.51948 800.81 8.702737 25.15836 810.82 8.702737 26.93724 820.83 8.702737 27.41389 830.84 44.70274 13.58341 840.85 8.702737 20.51434 850.86 43.70274 12.46673 860.87 9.702549 18.81264 870.88 9.702549 24.01407 880.89 10.70255 24.67573 89

0.9 9.702549 25.58508 900.91 48.70255 11.30587 910.92 10.70255 17.51203 920.93 10.70255 22.93352 930.94 10.70255 24.3862 940.95 10.70255 24.77545 950.96 10.70255 24.87974 960.97 8.702356 27.47204 970.98 8.702356 27.87183 980.99 9.702356 27.2469 99

1 9.702356 27.07945 1001.01 9.702356 27.03459 1011.02 9.702356 27.02256 1021.03 8.702356 27.75139 1031.04 9.702158 25.95935 1041.05 9.702158 25.81552 1051.06 9.702158 25.77699 1061.07 9.702158 25.76666 1071.08 9.702158 25.76389 1081.09 9.702158 25.76315 109

1.1 9.702158 25.76295 1101.11 9.702158 25.7629 1111.12 9.702158 25.76289 1121.13 10.70195 26.3522 1131.14 10.70195 26.15605 1141.15 10.70195 26.10349 1151.16 10.70195 26.08941 1161.17 10.70195 26.08563 1171.18 10.70195 26.08462 1181.19 10.70195 26.08435 119

1.2 10.70195 26.08428 12010.70175 25.891099.701746 26.623139.701746 26.819299.701746 26.871849.701746 26.885939.701746 26.88979.701746 26.890719.701746 26.890989.701746 26.891069.701746 26.891079.701533 26.899499.701533 26.899499.701533 26.899499.701533 26.899499.701533 26.899499.701533 26.899499.701533 26.899499.701533 26.899499.701533 26.8994910.70131 26.1827310.70131 25.9865810.70131 25.9340210.70131 25.9199310.70131 25.9161610.70131 25.9151510.70131 25.9148810.70131 25.914810.70131 25.9147910.70131 25.9147810.70109 25.9446310.70109 25.9446310.70109 25.9446310.70109 25.9446310.70109 25.9446310.70109 25.9446310.70109 25.9446310.70109 25.9446310.70109 25.9446310.70109 25.9446310.70109 25.944639.700862 26.776219.700862 26.972369.700862 27.024929.700862 27.039019.700862 27.042789.700862 27.043799.700862 27.044069.700862 27.044139.700862 27.044159.700628 23.941998.700628 24.674059.700628 24.138159.700628 23.994559.700628 23.956089.700628 23.945779.700628 23.943019.700628 23.942279.700628 23.942079.700628 23.942019.700628 23.9429.700628 23.94210.70039 22.9477910.70039 22.7516410.70039 22.6990810.70039 22.68510.70039 22.6812210.70039 22.6802110.70039 22.6799410.70039 22.6798710.70039 22.6798510.70014 24.7341510.70014 24.7341510.70014 24.7341510.70014 24.7341510.70014 24.7341510.70014 24.7341510.70014 24.7341510.70014 24.7341510.70014 24.734159.699896 25.731239.699896 25.927389.699896 25.979949.699896 25.994039.699896 25.99789.699896 25.998819.699896 25.999089.699896 25.999169.706049 26.855369.706049 26.855379.706049 26.855379.706049 26.855379.706049 26.855379.706049 26.855379.706049 26.855379.706049 26.855379.706049 26.855379.706044 26.897059.706044 26.897059.706044 26.897059.706044 26.897059.706044 26.897059.706044 26.897059.706044 26.897059.706044 26.897059.706044 26.897059.706044 26.897059.706033 27.212819.706033 27.212819.706033 27.212819.706033 27.212819.706033 27.212819.706033 27.212819.706033 27.212819.706033 27.212819.706033 27.2128146.70602 10.6025546.70602 3.3449099.706017 18.2132246.70602 8.13679545.70602 3.4162639.706017 18.232347.706017 26.4198.706017 27.880579.706017 27.540149.706017 27.448929.705996 27.909659.705996 27.90319.705996 27.901348.705996 28.632939.705996 28.09698.705996 28.68532

48.706 10.658889.705996 20.307099.705996 25.866

9.70597 25.981759.70597 26.380869.70597 26.4878

46.70597 10.361449.70597 19.2665

44.70597 9.8906039.70597 19.14034

11.70594 22.4932111.70594 23.5498111.70594 23.8329311.70594 23.9087911.70594 23.9291111.70594 23.9345611.70594 23.9360211.70594 23.9364111.70594 23.93651

10.7059 25.4252310.7059 25.6213910.7059 25.6739510.7059 25.6880310.7059 25.6918110.7059 25.6928210.7059 25.6930910.7059 25.69316

9.705858 26.654899.705858 26.851059.705858 26.903619.705858 26.91779.705858 26.921479.705858 26.922489.705858 26.922759.705858 26.922829.705858 26.922849.705858 26.922859.705858 26.92285

9.70581 26.47389.70581 26.47389.70581 26.47389.70581 26.47389.70581 26.4738

10.70581 25.741759.70581 26.27765

10.70576 22.0239710.70576 21.8662910.70576 21.8240510.70576 21.8127210.70576 21.8096910.70576 21.80888

10.7057 25.0085310.7057 25.0084710.7057 25.0084510.7057 25.0084510.7057 25.0084510.7057 25.0084510.7057 25.0084510.7057 25.0084510.7057 25.0084510.7057 25.0084510.7057 25.00845

9.705636 25.897679.705636 26.093829.705636 26.146389.705636 26.160469.705636 26.164239.705636 26.165249.705636 26.165529.705636 26.1655945.70564 10.053569.705636 19.104139.705567 23.343229.705567 24.728359.705567 25.099499.705567 25.198949.705567 25.225599.705567 25.232739.705567 25.234649.705567 25.235169.705567 25.235299.705567 25.235339.705567 25.235349.705567 25.235349.705493 26.410069.705493 26.410069.705493 26.410069.705493 26.410069.705493 26.410069.705493 26.410069.705493 26.410069.705493 26.410069.705493 26.410069.705493 26.4100610.70541 25.9523210.70541 25.7561644.70541 13.7376546.70541 5.5902810.70541 16.8372610.70541 23.3137910.70541 25.0491710.70541 25.5141710.70541 25.6387645.70541 12.9741410.70541 18.8157610.70541 23.8439310.70533 24.1613110.70533 24.5223110.70533 24.6190510.70533 24.6449610.70533 24.6519110.70533 24.6537710.70533 24.6542710.70533 24.654447.70533 10.6929210.70533 17.3968110.70524 23.9292910.70524 25.3528910.70524 25.7343410.70524 25.8365610.70524 25.8639410.70524 25.8712810.70524 25.8732510.70524 25.8737710.70524 25.8739210.70524 25.8739510.70524 25.8739610.70524 25.873979.705145 24.763829.705145 24.959979.705145 25.012539.705145 25.026619.705145 25.030399.705145 25.03149.705045 23.9844410.70504 23.2524610.70504 23.0563310.70504 23.0037710.70504 22.989698.705045 24.450028.705045 24.841318.705045 24.946169.704939 23.989729.704939 23.801099.704939 23.750559.704939 23.737019.704939 23.733389.704939 23.732419.704939 23.732159.704939 23.732089.704939 23.7320610.70483 25.19110.70483 24.9948511.70483 24.2102448.70483 9.453778.704829 18.8821811.70483 22.5723547.70483 9.74695210.70483 17.4966410.70483 22.9331511.70483 23.6578111.70483 23.8519810.70471 25.3798210.70471 25.5899110.70471 25.6462110.70471 25.661299.704592 26.379749.704592 26.576979.704592 26.629829.704592 26.643989.704592 26.647789.704592 26.648799.704592 26.649079.704592 26.6491410.70447 25.773710.70447 25.5775510.70447 25.5249910.70447 25.5109110.70447 25.5071411.70447 24.7740810.70447 25.309710.70447 25.4532210.70447 25.4916811.70433 25.2971610.70433 25.8358210.70433 25.9801511.70433 25.2867711.70433 25.1009811.70433 25.051211.70433 25.0378611.70433 25.0342911.70433 25.03333

10.7042 26.2166110.7042 26.4126910.7042 26.4652310.7042 26.4793110.7042 26.4830810.7042 26.484110.7042 26.4843710.7042 26.4844410.7042 26.48446

9.704056 26.140279.704056 26.336439.704056 26.388999.704056 26.403079.704056 26.406849.704056 26.407859.704056 26.4081210.70391 22.24210.70391 22.0458710.70391 21.9933210.70391 21.9792410.70391 21.9754610.70391 21.9744510.70391 21.9741810.70391 21.9741110.70376 25.6716510.70376 25.6716510.70376 25.6716510.70376 25.6716510.70376 25.671659.703757 26.403711.70376 25.1357510.70376 25.5280510.70376 25.6331711.70376 24.92929

11.7036 24.846410.7036 25.5279111.7036 24.9784811.7036 24.8312511.7036 24.7918111.7036 24.78124

10.70292 23.8151910.70292 24.0105910.70292 24.0629410.70292 24.0769710.70292 24.0807310.70292 24.0817410.70292 24.0820110.70292 24.0820810.70292 24.082110.70292 24.082110.70292 24.082119.702737 26.320418.702737 27.248619.702737 26.765278.702737 27.367819.702737 26.797218.702737 27.376378.702737 27.531568.702737 27.573149.702737 26.852239.702737 26.6590610.70255 25.2048111.70255 24.2627411.70255 24.0103111.70255 23.9426811.70255 23.9245511.70255 23.919711.70255 23.91849.702356 26.482399.702356 26.87469.702356 26.97979.702356 27.0078610.70236 26.283359.702356 26.8212710.70236 26.2333610.70236 26.0758210.70216 24.7783311.70216 24.0349711.70216 23.8357911.70216 23.78242

0

20

40

60

0 0.2 0.4 0.6 0.8 1 1.2Fe

atur

es

Time (seconds)

PRx_deoscillated RSSI

Figure 3. Variation of the two features PRx_deoscillated and RSSI_from_min for 1.2 s.

3.1.2. Data Labeling

In case of supervised learning of human-activity recognition from sensor data, it is nec-essary to appropriately label the output. The dataset contained approximately 1685 samplesper minute, obtained for 60 min, resulting in approximately one sample generated every0.03 s. This indicated that the RFID interrogation frequency was approximately 28 Hz perRFID tag. For each sample, we have two features—PRx−deoscillated and RSSI_ f rom_min.

The observations were broken into time windows of 1 s with no overlap. Hence, eachtime window contained approximately 28 samples, with two sets of observations from the2 features. We manually labeled the data collected from the two features as ‘1’ when theSimBaby is in breathing state and ‘0’ when it is in a non-breathing state. Over a period ofone hour, we collect and label the dataset to form train and test samples representing binaryrespiratory state (1: Breathing, 0: Non-Breathing state). Figure 4 shows the respiratory statecorresponding to the features of Figure 3 for 1.2 s.

0.01 10.02 10.03 10.04 10.05 10.06 10.07 10.08 10.09 1

0.1 10.11 10.12 10.13 10.14 10.15 10.16 10.17 10.18 10.19 1

0.2 10.21 10.22 10.23 10.24 10.25 10.26 10.27 10.28 10.29 1

0.3 10.31 10.32 10.33 10.34 10.35 10.36 10.37 10.38 10.39 1

0.4 10.41 10.42 10.43 10.44 10.45 10.46 10.47 10.48 10.49 1

0.5 10.51 10.52 10.53 10.54 10.55 10.56 10.57 10.58 10.59 1

0.6 10.61 10.62 10.63 10.64 10.65 00.66 10.67 10.68 10.69 1

0.7 10.71 10.72 10.73 10.74 10.75 10.76 10.77 10.78 10.79 1

0.8 10.81 10.82 10.83 10.84 10.85 10.86 10.87 10.88 10.89 1

0.9 10.91 10.92 10.93 10.94 10.95 10.96 10.97 10.98 10.99 1

1 11.01 11.02 11.03 11.04 11.05 11.06 11.07 11.08 11.09 1

1.1 11.11 11.12 11.13 11.14 11.15 11.16 11.17 11.18 11.19 1

1.2 1

0

1

0 0.2 0.4 0.6 0.8 1 1.2Resp

irato

ry

Stat

e

Time (seconds)

Figure 4. Respiratory state corresponding to features shown in Figure 3.

Since there are only two features and two output classes, the problem we aim to solveis a bivariate time series binary classification one.

3.2. Feature Scaling

From the time series features extracted from each time window, we apply featureengineering to make the input vectors suitable for the classifier. For multivariate data, itis necessary to transform features with different scales to have uniform distribution, toensure optimal performance of the classifiers. We first cleaned our feature set by filteringthe missing values (NaN). The data with the features and the labels were loaded from twocsv files and then we split the dataset 3:1 to form the training set and the testing set. Aftersplitting the dataset, we scale the features before we fit it into our classifier, which is aone-dimensional convolutional neural network (1DCNN).

The two features in our dataset were scaled to a standard range and the distribution ofthe values was rescaled, so the mean was 0 and the standard deviation was 1. The methodinvolved determining the distribution of each feature and subtracting the mean from eachfeature. Then we divide the values (after the mean has already been subtracted) of eachfeature by its standard deviation.

The standard score (Z) of a sample is given by Equation (3).

Z =x− µ

σ, (3)

Electronics 2022, 11, 682 7 of 22

where x is the sample value and µ and σ are the mean and standard deviation of all thesamples, respectively. Feature standardization transforms the raw values into the standardscale that helps the model to extract salient signal information from the observations.After rescaling the variables, we reshape the data according to dimension expected by theconvolution layer of the 1DCNN model.

3.3. Deep Learning Model Selection

A convolutional neural network (CNN) is a class of deep learning that uses a linearoperation called convolution in at least one of its layers. Equation (4) represents a convolu-tion operation, where x is the input and w represents the kernel, which stores parametersfor the model. The output s is called the feature map of the convolution layer.

s(t) =∫

x(a)w(t− a)da (4)

In a CNN, the first layer is a convolution layer that accepts a tensor as an input withdimensions based on the size of the data [31]. The second layer, or the first hidden layer, isformed by applying a kernel or filter that is a smaller matrix of weights over a receptivefield, which is a small subspace of the inputs. Kernels apply an inner product on thereceptive field, effectively compressing the size of the input space [12]. As the kernel stridesacross the input space, the first hidden layer is computed based on the weights of the filter.As a result, the first hidden layer is a feature map formed from the kernel applied on theinput space. While the dimension of the kernel may be much smaller in size comparedto the initial inputs of the convolution layer, the kernel must have the same depth of theinput space. The inputs and convolution layers are often followed by rounds of activation,normalization, and pooling layers [12]. The precise number and combination of these layersare specific to the problem at hand.

For the proposed respiratory classification problem, our CNN model consists of oneconvolution layer, which is activated by a rectified linear unit (ReLU). The ReLU activa-tion is a suitable choice for non-linear transformation without the problem of vanishinggradient. The filter size is set to 64 and the kernel size to 1. This layer is followed by aone dimensional Max Pooling layer with a pool size and stride length of 1, each. The nextlayer is a Flattening layer followed by a Dropout layer. The Dropout layer randomly setsinput neurons to 0 with a rate of 0.01 at each step during training time. This is done toprevent overfitting. The dropout layer is followed by two fully connected hidden layers.The first hidden layer consists of 200 neurons with ReLU activation, and the second hiddenlayer contains 100 neurons with a ReLU activation function and a Softmax function forthe output layer. Softmax is a choice for the output layer, for output to be interpreted asnormalized probabilities. Overall, the proposed CNN model uses one-dimensional convo-lutions and therefore, this model is referred to as 1DCNN. Figure 5 shows the proposed1DCNN architecture along with the dimension of each layer.

1 6464

75384

2

37692

input convolution max pooling fully connected

Figure 5. Our proposed 1DCNN architecture.

Electronics 2022, 11, 682 8 of 22

3.4. Hyperparameter Optimization

In a CNN, the parameters in each layer whose values control the learning procedureare called hyperparameters. Grid search is a hyperparameter tuning technique that canbuild a model for every new combination of hyperparameters that is specified in the searchspace and evaluates each model for that combination. A machine learning algorithm Aθ canestimate a model Mθ ∈ M that minimizes a loss function L with its model regularizationtermR given by

Aθ(Xtrain) = argminMθ∈M

L(Mθ , Xtrain) +R(Mθ , θ), (5)

where Xtrain is the training dataset, M is the set of all models, θ ∈ Θ is the chosen hyperpa-rameter configuration, and Θ = {Θ1, Θ2, · · · , Θp} is the p-dimensional hyperparameterspace of the algorithm. The optimal hyperparameter configuration θ∗ is calculated usingthe validation set as

θ∗ = argminθ∈Θ

L(Aθ(Xtrain, Xvalidation) = fD(θ), (6)

where Xvalidation is the validation set and fD is the misclassification rate. The Grid searchexhaustively searches through a grid of manually specified set of parameter values providedin a search space to find the accuracy obtained with each combination. We tuned themodel based on the number of epochs (ranging from 10 to 100) and the learning ratefor the Adam optimizer (0.001, 0.01, 0.1, 0.002, 0.02, 0.2, 0.003, 0.03, 0.3). We evaluatedthe accuracy as a performance metric for the different combinations. We also performeda Bayesian optimization that uses Bayes Theorem to tune the hyperparameters with afive-fold crossvalidation. We found the same parameters that are best fit to obtain bestscore. Figure 6 shows the selection of hyperparameters, where the accuracy is reported fordifferent combinations of epochs and learning rates [32].

02 04 06 08 01 0 0

0 . 0 00 . 0 50 . 1 00 . 1 50 . 2 00 . 2 50 . 3 0

0 . 7 0

0 . 7 5

0 . 8 0

0 . 8 5

0 . 9 0

0 . 9 5

L e a r n i n g R a t e

A cc u

r ac y

E p o c h s

Figure 6. Hyperparameter selection.

Table 1 summarizes the hyperparameters of the 1DCNN.

Electronics 2022, 11, 682 9 of 22

Table 1. Summary of hyperparameters for the proposed 1DCNN model.

Learning rate 0.001

Batch size 5

Optimizer Adam

Data shuffle per epoch

Maximum epochs 100

3.5. Model Training and Validation

We trained our 1DCNN model with 75,834 samples and used repeated k-fold cross-validation with 10 splits to validate our model performance. To improve the estimatedperformance, we repeated the cross-validation procedure multiple times and reported themean result across all folds from all runs. This reported mean accuracy is expected to be amore accurate estimate of the true unknown underlying mean performance of the modelon the dataset instead of a single run of k-fold cross-validation, ensuring less statisticalnoise. We also compute the standard error that provides an estimate of a given samplesize of the amount of error that is expected from the sample mean to the underlying andunknown population mean. The standard error is calculated as

σerror =σ√n

, (7)

where σ is the sample’s standard deviation and n is the number of repeats. We obtained avalidation classification accuracy of 87.78% with a standard error of 0.002 (see Section 6for detailed evaluation). We defined Early Stopping as a regularization technique at thevery beginning of declaring the model architecture. At end of every epoch, the trainingloop monitors whether the validation loss is no longer decreasing and once it is found nolonger decreasing, the training is terminated. We enabled the patience parameter equalto 5 to terminate the training after epochs of no validation loss decrease. This is anothermeasure to prevent the model from overfitting during training, alongside the addition of aDropout layer. Without Early Stopping, the training would terminate only after reachingthe maximum number of epochs.

3.6. Model Testing and Deployment

We deployed the trained model to test the performance on an unseen test set generatedfrom the SimBaby to classify the respiratory states. The model was tested on 25,279 samplesand achieved an accuracy of 97.15%, F1 Score of 0.98, AUC score of 0.98, sensitivity score of0.96, and specificity score of 0.99. These performance metrics are defined in Section 6.1.

4. Model Quantization

CNN models consume a considerable amount of energy due to their high computa-tional complexity. The high energy consumption is due to the large model size involvingseveral thousand parameters. It is therefore challenging to deploy the inference, i.e., atrained CNN model on battery-powered mobile devices, such as smartphones and wear-able gadgets due to their limited energy budget. To address this high energy overhead,energy-efficient solutions have been proposed to reduce the model size and computationalcomplexity. Some common approaches include the pruning of network weights [33] andlow bit precision networks [34]. We focus on the latter techniques. Specifically, we imple-ment both bit precision weights and activations to reduce model sizes and computationalpower requirements. To perform the training of a CNN with low-precision weights andactivations, we use the following quantization function to achieve a k-bit quantization [35].

Zq = Q(Zr) =1

2k − 1round

((2k − 1)Zr

), (8)

Electronics 2022, 11, 682 10 of 22

where Zr ∈ [0, 1] is the full precision value and Zq ∈ [0, 1] is the quantized value obtainedusing the k-bit quantization.

The quantization of weights is given by

wq = Q(

tanh(wr)

2 · max(|tanh(wr)|)+

12

), (9)

where wr is the original weight using full precision and wq is the quantized value usingk-bit quatization.

The quantization of activations is given by

xq = Q( f (x)), (10)

where f (x) = clip(xr, 0, 1) is the clip function bounding the activation function between0 and 1.

In this paper, we apply both bit precision techniques for both weights and activa-tions using quantization with the QKeras library, which is a quantization extension tothe Keras [36]. It enables a drop-in replacement of layers that are responsible for creatingparameters and activation layers like the Conv 1D, Dense layers. It facilitates arithmeticcalculations by creating a deeply quantized version of a Keras model. We tag the variables,weights, and biases created by the Keras implementation of the model and the outputof arithmetic layers by quantized functions. Quantized functions are specified as layerparameters and then passed as a cumulative quantization and activation function, QActiva-tion. The quantized bits quantizer used above performs mantissa quantization using thefollowing equation.

mantissa quantization = (11)

2b−k+1 · clip(round(xr · 2k−b−1),−2k−1, 2k−1 − 1

)where x is the input given to the model, k is the number of bits for quantization, and bspecifies how many bits of the bits are to the left of the decimal point.

We conduct our experiment to perform quantization of our Conv1D model using 2 bits,4 bits, 8 bits, 16 bits, 32 bits, and 64 bits. We observe the performance accuracy increasewith the increase in the quantization bits, with 2 bits achieving an 88.93% compared tousing all the 64 bits achieving 97.15% (see the detailed results in Section 6.2).

The QTools functionality is used to estimate the model energy consumption for the dif-ferent bit-wise quantization implementations. It estimates a layer-wise energy consumptionfor memory access and MAC operations in a quantized model derived from QKeras. Thisis helpful when comparing the power consumption of more than one model running on thesame device. The model size is calculated as the number of model parameters multipliedby the number of bits used in each scenario. We observe that when we increase the numberof bits, the model size increases as well as the accuracy, but so does the consumption ofenergy (pJ). This homogeneous replacement technique of Keras layers, with heterogeneousper-layer, per-parameter type precision, chosen from a wide range of quantizers, enabledquantization-aware training and energy-aware implementation to maximize the modelperformance given a situation of resource constraints, like detection of respiratory cessationon premature infants in critical care conditions, which is crucial for high-performanceinference on wearables.

5. SNN-Based Respiratory Classification

Spiking neural networks (SNNs), also known as the third generation of neural net-works, are an interconnection of integrate-and-fire neurons that emulate the workingprinciple of a mammalian brain [13]. SNNs enable powerful computations due to theirspatio-temporal information-encoding capabilities. In an SNN, spikes (i.e., current) in-jected from pre-synaptic neurons raise the membrane voltage of a post-synaptic neuron.

Electronics 2022, 11, 682 11 of 22

When the membrane voltage crosses a threshold, the post-synaptic neuron emits a spikethat propagates to other neurons. Figure 7 shows the integration of spike train from fourpre-synaptic neurons connected to a post-synaptic neuron via synapses.

binary events

spike voltage

post-synaptic neuron

pre-

syna

ptic

ne

uron

s

initer-spike interval (ISI)

Figure 7. Integration of spike trains at the post-synaptic neuron from four pre-synaptic neurons in aspiking neural network (SNN). Each spike is a voltage waveform of ms time duration.

SNNs can implement many machine learning approaches such as supervised, un-supervised, reinforcement, few-shot, and lifelong learning. Due to their event-drivenactivation, SNNs are particularly useful in energy-constrained platforms such as wearableand embedded systems. Recent works demonstrate a significant reduction in memoryfootprint and energy consumption in SNN-based heart-rate estimation [27], heartbeatclassification [28,37], speech recognition [38], and image processing [39].

To integrate SNN-based respiratory classification into our design pipeline, we intro-duce two additional stages—model conversion and SNN parameter tuning—before theSNN model is deployed to perform classification from live data collected from the SimBaby.Figure 8 shows the new design pipeline.

(1) Data Collection

and Labeling

(2)Feature Scaling

(3)Model

Selection

(4)Model

Training

(4)Model

Conversion

(5)Parameter

Tuning

(6)Model

Deployment

Figure 8. Seven-stage pipeline, including the two new stages to process and optimize the SNN model.

5.1. Model Conversion

In this work, the 1DCNN architecture is converted to SNN in order to executeit on a neuromorphic hardware such as Loihi [15]. The conversion steps are brieflydiscussed below.

1. ReLU Activation Functions: This is implemented as the approximate firing rate of aleaky integrate and fire (LIF) neuron.

2. Bias: A bias is represented as a constant input current to a neuron, the value of whichis proportional to the bias of the neuron in the corresponding analog model.

3. Weight Normalization: This is achieved by setting a factor λ to control the firing rateof spiking neurons.

4. Softmax: To implement softmax, an external Poisson spike generator is used togenerate spikes proportional to the weighted sum accumulated at each neuron.

5. Max and Average Pooling: To implement max pooling, the neuron which fires first isconsidered to be the winning neuron, and therefore, its responses are forwarded to thenext layer, suppressing the responses from other neurons in the pooling function. Toimplement average pooling, the average firing rate (obtained from total spike count)of the pooling neurons are forwarded to the next layer of the SNN.

6. 1-D Convolution: The 1-D convolution is implemented to extract patterns from inputsin a single spacial dimension. A 1xn filter, called a kernel, slides over the inputwhile computing the element-wise dot-product between the input and the kernel ateach step.

Electronics 2022, 11, 682 12 of 22

7. Residual Connections: Residual connections are implemented to convert the residualblock used in CNN models such as ResNet. Typically, the residual connection connectsthe input of the residual block directly to the output neurons of the block, with asynaptic weight of ‘1’. This allows for the input to be directly propagated to the outputof the residual block while skipping the operations performed within the block.

8. Flattening: The flatten operation converts the 2-D output of the final pooling operationinto a 1-D array. This allows for the output of the pooling operation to be fed asindividual features into the decision-making fully connected layers of the CNN model.

9. Concatenation: The concatenation operation, also known as a merging operation, isused as a channel-wise integration of the features extracted from two or more layersinto a single output.

We now briefly elaborate how an analog operation such as Rectified Linear Unit(ReLU) is implemented using SNN. The output Y of a ReLU activation function is given by

Y = max 0, ∑i

wi ∗ xi, (12)

where wi is the weight and xi is the activation on the ith synapse of the neuron. To map theReLU activation function, we consider a particular type of spiking neuron model known asan integrate and fire (IF) neuron model. The IF spiking neuron’s transfer function can berepresented as

vm(t + 1) = vm(t) + ∑i

wi ∗ xi(t), (13)

where vm(t) is the membrane potential of the IF neuron at time t, wi is the weight, andxi(t) is the activation on the ith synapse of the neuron at time t. The IF spiking neuronintegrates incoming spikes (Xi) and generates an output spike (Yspike) when the membranepotential (vm) exceeds the threshold voltage (vth) of the IF neuron. Therefore, by ensuringthat the output spiking rate Yspike is proportional to the ReLU activation Y, i.e., Yspike ∝ Y,we accurately convert the ReLU activation to the spike-based model. To further illustratethis, we consider the multi-layer perceptron (MLP) of Figure 9a and its SNN conversionusing rate-based encoding (Figure 9b) and inter-spike interval (ISI) encoding (Figure 9c).

Electronics 2022, 1, 0 12 of 22

7. Residual Connections: Residual connections are implemented to convert the residualblock used in CNN models such as ResNet. Typically, the residual connection connectsthe input of the residual block directly to the output neurons of the block, with asynaptic weight of ‘1’. This allows for the input to be directly propagated to the outputof the residual block while skipping the operations performed within the block.

8. Flattening: The flatten operation converts the 2-D output of the final pooling operationinto a 1-D array. This allows for the output of the pooling operation to be fed asindividual features into the decision-making fully connected layers of the CNN model.

9. Concatenation: The concatenation operation, also known as a merging operation, isused as a channel-wise integration of the features extracted from two or more layersinto a single output.

We now briefly elaborate how an analog operation such as Rectified Linear Unit(ReLU) is implemented using SNN. The output Y of a ReLU activation function is given by

Y = max 0, ∑i

wi ∗ xi, (12)

where wi is the weight and xi is the activation on the ith synapse of the neuron. To map theReLU activation function, we consider a particular type of spiking neuron model known asan integrate and fire (IF) neuron model. The IF spiking neuron’s transfer function can berepresented as

vm(t + 1) = vm(t) + ∑i

wi ∗ xi(t), (13)

where vm(t) is the membrane potential of the IF neuron at time t, wi is the weight, andxi(t) is the activation on the ith synapse of the neuron at time t. The IF spiking neuronintegrates incoming spikes (Xi) and generates an output spike (Yspike) when the membranepotential (vm) exceeds the threshold voltage (vth) of the IF neuron. Therefore, by ensuringthat the output spiking rate Yspike is proportional to the ReLU activation Y, i.e., Yspike ∝ Y,we accurately convert the ReLU activation to the spike-based model. To further illustratethis, we consider the multi-layer perceptron (MLP) of Figure 9a and its SNN conversionusing rate-based encoding (Figure 9b) and inter-spike interval (ISI) encoding (Figure 9c).

(a) MLP in analog domain (b) MLP in spiking domain (rate coding)

(c) MLP in spiking domain (ISI coding)

Figure 9. Example of converting an analog MLP to its spiking equivalent.

In Figure 9a, neurons 1, 2, and 3 are the input neurons and neurons 4 and 5 are theoutput neurons. To keep the model simple, let us consider the case where the activations ofthe input neurons 1, 2, and 3 are equal to 1. Using Equation (12), we know that the outputof neurons 4 and 5 are 0.6 and 0.3, respectively. Figure 9b,c shows the mapped SNN model,using rate-based and inter-spike interval encoding schemes, respectively. In the rate-basedmodel in Figure 9b, the rate of spikes generated is expected to be proportional to the outputof neurons 4 and 5 in the MLP. In the case of the ISI-based SNN model, the inter-spikeinterval of the spikes generated by neurons 4 and 5 is expected to be proportional to theoutput generated in the MLP, as shown in Figure 9c.

Figure 9. Example of converting an analog MLP to its spiking equivalent.

In Figure 9a, neurons 1, 2, and 3 are the input neurons and neurons 4 and 5 are theoutput neurons. To keep the model simple, let us consider the case where the activations ofthe input neurons 1, 2, and 3 are equal to 1. Using Equation (12), we know that the outputof neurons 4 and 5 are 0.6 and 0.3, respectively. Figure 9b,c shows the mapped SNN model,using rate-based and inter-spike interval encoding schemes, respectively. In the rate-basedmodel in Figure 9b, the rate of spikes generated is expected to be proportional to the outputof neurons 4 and 5 in the MLP. In the case of the ISI-based SNN model, the inter-spikeinterval of the spikes generated by neurons 4 and 5 is expected to be proportional to theoutput generated in the MLP, as shown in Figure 9c.

Electronics 2022, 11, 682 13 of 22

5.2. SNN Mapping to Neuromorphic Hardware

The SNN model generated using the conversion approach is analyzed in CARLsim [40]to generate the following information.

• Spike Data: the exact spike times of all neurons in the SNN model.• Weight Data: the synaptic strength of all synapses in the SNN model.

The spike and weight data of a trained SNN form the SNN workload, which is usedin the NeuroXplorer framework [41] to estimate the energy consumption. Figure 10 showsthe NeuroXplorer framework.

ANN-to-SNN Converter(in-house)

CARLSim Clustering Mapping Cycle-Accurate(Loihi)1DCNN

EnergyAccuracy

Decomposition

Figure 10. The NeuroXplorer framework [41].

The framework inputs the 1DCNN model and estimates the accuracy and energyconsumption of the model on a neuromorphic hardware. Internally, NeuroXplorer firstconverts the 1DCNN to SNN using the steps outlined before. It then simulates the SNNusing CARLsim. The extracted workload is first decomposed using the decompositionapproach presented in [42]. This is to ensure that the workload can fit on to the resource-constraint hardware.

Typically, neuromorphic hardware is designed with tile-based architecture [43], whereeach tile can accommodate only a limited number of neurons and synapses. The tilesare interconnected using a shared interconnect such as Network-On-Chip (NoC) [44] orSegmented Bus [45]. Therefore, to map an SNN into a tile-based neuromorphic hardware,the model is first partitioned into clusters, where each cluster consists of a proportion ofthe neurons and synapses of the original machine learning model [46]. Each cluster canthen fit onto a tile of the hardware. Then, the clusters are mapped to the tiles to optimizeone or more hardware metrics such as energy [47,48], latency [49–53], circuit aging [54–59],and endurance [60–62]. We use the energy-aware mapping technique of [48].

Once the clusters of the converted 1DCNN model are placed with the resources ofthe neuromorphic hardware, we perform cycle-accurate simulations using NeuroXplorer,configured to simulate the Loihi neuromorphic system. Table 2 shows the hardwareparameters that are configured in NeuroXplorer.

Table 2. Major simulation parameters extracted from Loihi [15].

Neuron technology 16 nm CMOS (original design is at 14 nm FinFET)

Synapse technology HfO2-based OxRRAM [63]

Supply voltage 1.0 V

Energy per spike 23.6 pJ at 30 Hz spike frequency

Energy per routing 3 pJ

Switch bandwidth 3.44 G. Events/s

5.3. SNN Parameter Tuning

Unlike the baseline 1DCNN architecture, where model hyperparameters are exploredonly during model training, SNNs allow parameter tuning on the trained (and converted)model, such that the energy and accuracy space could be explored to generate a solutionthat satisfies the given energy and accuracy constraints of the target wearable platform.To explore such exploration capabilities, we analyze the dynamics of SNNs.

Electronics 2022, 11, 682 14 of 22

The membrane potential of a neuron at time t can be expressed as [13]

u(t) = u0 + a∫ t

0D(s) · w · σ(t− s)ds, (14)

where u0 is the initial membrane potential, a is a positive constant, D(s) is a linear filter, w isthe synaptic weight and σ represents a sequence of N input spikes, which can be expressedusing the Dirac delta function as

σ(t) =N

∑i=1

δ(t− ti) (15)

The membrane potential of a neuron increases upon the arrival of an input spike.Subsequently, the membrane potential starts to decay during the inter-spike interval (ISI).When the neuron is subjected to an input spike train, the membrane voltage keeps rising,building on the undissipated component. When the membrane potential crosses a threshold(Vth), the neuron emits a spike, which then propagates to other neurons of the SNN. Thespike rate of a neuron can be controlled using this threshold. If the threshold is set toohigh, fewer spikes will be generated, meaning that not only will the energy be lower butalso the accuracy, because spikes encode information in SNNs. Therefore, by adjusting thethreshold, the design space of accuracy and energy can be explored (see Section 6.3).

6. Results and Discussions

All simulations are performed on a workstation, which has AMD Threadripper 3960Xwith 24 cores, 128 MB cache, 128 GB RAM, and 2 RTX3090 GPUs. Keras [36] is used toimplement the baseline 1DCNN, which uses TensorFlow backend [64]. QKeras [65] is usedfor training and testing the quantized neural network. Finally, CARLsim [40] is used forSNN function simulations.

We present our respiratory classification results organized into (1) results for thebaseline 1DCNN model (Section 6.1), (2) results using quantization (Section 6.2), and(3) SNN-specific results (Section 6.3).

6.1. Baseline 1DCNN Performance

In this section, we evaluate the performance of the proposed 1DCNN specified usingthe following metrics.

• Top-1 Accuracy: This is the conventional accuracy and it measures the proportionof test examples for which the predicted label (i.e., respiratory state) matches theexpected label. To formulate top-1 accuracy, we introduce the following definitions.

– True Positives (TP): For binary classification problems, i.e., ones with a yes/nooutcome (such as the case of respiratory classification), this is the total numberof test examples for which the value of the actual class is yes and the value ofpredicted class is also yes.

– True Negatives (TN): This is the total number of test examples for which the valueof the actual class is no and the value of the predicted class is also no.

– False Positives (FP): This is the total number of test examples for which the valueof the actual class is no but the value of the predicted class is yes.

– False Negatives (FN): This is the total number of test examples for which thevalue of the actual class is yes but the value of the predicted class is no.

Top-1 Accuracy =TP + TN

TP + FP + FN + TN(16)

• F1 Score: To formulate the F1 score, we introduce the following definitions.

Electronics 2022, 11, 682 15 of 22

– Precision: This is the ratio of correctly predicted positive observations to the totalpredicted positive observations, i.e.,

Precision =TP

TP + FP(17)

– Recall: This is the ratio of correctly predicted positive observations to the allobservations in actual class, i.e.,

Recall =TP

TP + FN(18)

The F1 score conveys the balance between the precision and the recall. It is calculatedas the weighted average of precision and recall, i.e.,

F1 Score =2 ∗ (Recall ∗ Precision)(Recall + Precision)

(19)

• AUC: In machine learning, a receiver operating characteristic (ROC) curve is a graphi-cal plot that illustrates the diagnostic ability of a binary classifier as its discriminationthreshold is varied. The area under curve (AUC) measures the two-dimensional areaunderneath the ROC curve. AUC tells how much the model is capable of distinguish-ing between classes. The higher the AUC, the better the model is at predicting yesclasses as yes and no classes as no.

• Sensitivity: This is the true positive rate, i.e., how often the model correctly generatesa yes out of all the examples for which the value of actual class is yes. Sensitivity isformulated as

Sensitivity =TP

TP + FN(20)

• Specificity: This is the true negative rate, i.e., how often the model correctly generatesa no out of all the examples for which the value of actual class is no. Specificity isformulated as

Specificity =TN

TN + FP(21)

Table 3 compares the classification performance using the proposed 1DCNN againstthree state-of-the-art approaches: (1) Support Vector Machine (SVM) classifier of [18],(2) Logistic Regression (LR) classifier of [19], and (3) Random Forest classifier of [19]. Wemake the following four key observations.

Table 3. Comparison with state-of-the-art approaches.

Classification Technique Top-1 Accuracy F1 Score AUC Sensitivity Specificity

SVM 92.34% 0.91 0.92 0.93 0.92LR 91.60% 0.91 0.90 0.90 0.92RF 93.40% 0.92 0.90 0.92 0.93

1DCNN (proposed) 97.15% 0.98 0.98 0.96 0.99

First, the proposed 1DCNN has the highest top-1 accuracy of all the evaluated tech-niques (higher top-1 accuracy is better). The top-1 accuracy of 1DCNN is better than SVMby 5.2%, LR by 6.0%, and RF by 4.0%.

Second, the proposed 1DCNN has the highest F1 score of all the evaluated techniques(higher F1 score is better). The F1 score of 1DCNN is higher than SVM by 7.7%, LR by 7.7%,and RF by 6.5%.

Third, the proposed 1DCNN has the highest AUC of all the evaluated techniques(higher AUC score is better). The AUC score of 1DCNN is higher than SVM by 6.2%, LR by8.5%, and RF by 8.5%.

Electronics 2022, 11, 682 16 of 22

Fourth, the proposed 1DCNN has the highest sensitivity of all the evaluated techniques(higher sensitivity score is better). The sensitivity score of 1DCNN is higher than SVM by3.2%, LR by 6.7%, and RF by 4.3%.

Finally, the proposed 1DCNN has the highest specificity of all the evaluated techniques(higher specificity score is better). The specificity score of 1DCNN is higher than SVM by7.6%, LR by 7.6%, and RF by 6.4%.

The reason for high performance using the proposed 1DCNN model is two-fold.First, we perform intelligent feature selection from the data collected using sensors onthe SimBaby programmable infant mannequin. Second, we perform hyperparameteroptimization with neural architecture search to generate a model that gives the highestclassification accuracy using the selected hyperparameters.

To give further insight to the improvement, Figure 11 shows the confusion matrixobtained for the training and test sets. We observe that the proposed 1DCNN model hasvery low false positives and false negatives, which are critical for respiratory classificationin premature newborn infants.

13616(99.58%)

57(0.42%)

2065(3.33%)

60095(96.67%)

4465(99.06%)

42(0.94%)

711(3.4%)

20061(96.5%)

Targ

et C

lass

Targ

et C

lass

Predicted Class Predicted Class

0

1

0

1

0 1 10

Confusion Matrix (training set) Confusion Matrix (test set)

Figure 11. Confusion matrix for the 1DCNN model.

6.2. Quantization Results

Table 4 and Figure 12 reports the top-1 accuracy (%), energy (in pJ), and modelsize (in bits) with 2-bit, 4-bit, 8-bit, 16-bit, and 32-bit precision for the model parameters.For comparison, we have included results using the baseline 1DCNN, which uses full 64-bitprecision for the model parameters. We make the following three key observations.

Table 4. Model quantization results.

Quantization Top-1 Accuracy Energy (pJ) Model Size (bits)

2-bit/parameter 88.93% 7089 92,2584-bit/parameter 88.98% 15,994 184,5168-bit/parameter 93.00% 29,871 369,03216-bit/parameter 96.55% 57,640 738,06432-bit/parameter 97.03% 113,386 1,476,128

Baseline 1DCNN 97.15% 134,613 2,952,256

First, the top-1 accuracy reduces with a reduction in the bit precision (higher accuracyis better for respiratory classification in premature newborn infants). With 2-bit, 4-bit, 8-bit,16-bit, and 32-bit precision, the top-1 accuracy is lower than the baseline 64-bit precision by8.5%, 8.4%, 4.3%, 0.6%, and 0.1%, respectively. The top-1 accuracy with 32-bit precision iscomparable to 64-bit precision.

Electronics 2022, 11, 682 17 of 22

0 4 8 1 2 1 6 2 0 2 4 2 8 3 2 3 6 4 0 4 4 4 8 5 2 5 6 6 0 6 48 89 09 29 49 69 8

0 4 8 1 2 1 6 2 0 2 4 2 8 3 2 3 6 4 0 4 4 4 8 5 2 5 6 6 0 6 40

2 0 0 0 04 0 0 0 06 0 0 0 08 0 0 0 0

1 0 0 0 0 01 2 0 0 0 01 4 0 0 0 0

Accu

racy

%

Energ

y

Q u a n t i z e d B i t s

p J

Figure 12. Relationship between quantization bits, accuracy, and energy consumption.

Second, energy reduces with a reduction of the bit precision (lower energy is better forrespiratory classification in wearables due to limited battery, as we mentioned in Section 1).With 2-bit, 4-bit, 8-bit, 16-bit, and 32-bit precision, energy is lower than the baseline 64-bitprecision by 94.7%, 88.1%, 77.8%, 57.2%, and 15.8%, respectively. These results showthe significant reduction in energy achieved using quantization. Lower energy leads tolonger battery life in wearables. Finally, model size also reduces with a reduction in the bitprecision (lower model size is better for wearables due to their limited storage availability).With 2-bit, 4-bit, 8-bit, 16-bit, and 32-bit precision, energy is lower than the baseline 64-bitprecision by 96.8%, 93.7%, 87.5%, 75.0%, and 50.0%, respectively.

We conclude that to reduce energy and model size for respiratory classification inwearables, quantization techniques can lead to a significant reduction in accuracy.

6.3. SNN-Related Results

Table 5 reports the top-1 accuracy and energy results using the proposed SNN-basedapproach compared to the baseline 1DCNN, 2-bit, and 8-bit quantized model. We makethe following three key observations.

Table 5. SNN Accuracy and Energy Results.

Model Top-1 Accuracy Energy (pJ)

Baseline 1DCNN 97.15% 134,6132-bit quantized 1DCNN 88.93% 70898-bit quantized 1DCNN 93.00% 29,871

SNN 93.33% 7282

First, the top-1 accuracy of the proposed SNN is only 4% lower than the baseline1DCNN model with 18x lower energy. Second, compared to the 2-bit quantized model, thetop-1 accuracy is 5% higher, while the energy is only 2% higher. Finally, compared to the8-bit quantized model, the top-1 accuracy is comparable while the energy is 4× lower. Weconclude that SNN-based respiratory classification achieves the best tradeoff in terms oftop-1 accuracy and energy. To achieve similar accuracy, SNN can lead to a 4× reductionin energy, which is a critical consideration for respiratory classification on wearables. Thefollowing results are reported to give further insight into these improvements.

Electronics 2022, 11, 682 18 of 22

6.3.1. SNN Accuracy Compared to 1DCNN

Figure 13 shows the Bland–Altman plot comparing the accuracy of SNN solutionagainst the baseline 1DCNN model. Bland–Altman plots are extensively used to evaluatethe agreement among two models, each of which produced some error in their predictions.As can be seen from the plot, the average accuracy difference between the 1DCNN and theconverted SNN is 7.3%, while the minimum and maximum accuracy difference are 2.1%and 12.5%, respectively.

Figure 13. Bland−Altman plot comparing the accuracy of different SNN solutions against the baseline1DCNN model.

6.3.2. Design Space Exploration with SNN Parameters

We perform design-space explorations to identify SNN model parameters that givethe best tradeoff in terms of energy and accuracy.

In spiking neural networks, a spike is not fired by a neuron unless the specifiedactivation threshold voltage is attained. This implies that the larger the firing thresholdvoltage, the more selectively a neuron is fired while communicating between each layer. Todemonstrate this, Figure 14 shows the variation in accuracy as a function of the activationthreshold (Vth) (see Section 5.3).

Electronics 2022, 11, 682 19 of 22

0 2 4 6 8 1 00 . 8 6

0 . 8 7

0 . 8 8

0 . 8 9

0 . 9 0

0 . 9 1

0 . 9 2

0 . 9 3

0 . 9 4

Accu

racy (

%)

A c t i v a t i o n T h r e s h o l d ( m V )

1 0 0 s a m p l e s 2 0 0 s a m p l e s 5 0 0 s a m p l e s 1 0 0 0 s a m p l e s 2 0 0 0 s a m p l e s

Figure 14. Accuracy impact with varying the activation.

In the figure, we vary the activation threshold across different number of test samples.We observe that for a smaller number of test samples, as we increase the threshold voltagefor firing spikes, the performance accuracy varies highly. However, test samples witha larger size, the performance accuracy does not vary as much when we increase thethreshold voltage.

For larger test sample sizes (500, 1000, 2000, etc.), the performance accuracy variesbetween 92% and 94%. When we use small sample sizes (e.g., 100 and 200), the predictionaccuracy varies between 86% and 92%. We observe that the firing threshold is most idealwhen set within 1–3 mV for smaller test samples and within 1–5 mV for larger test samples.

7. Conclusions

We propose a deep-learning-enabled wearable monitoring system for prematurenewborn infants, where respiratory cessation is predicted using signals that are collectedwirelessly from a non-invasive wearable Bellypatch put on the infant’s body. To this end,we developed an end-to-end design pipeline involving five stages data collection, featurescaling, model selection, training, and deployment. The deep learning model is a 1Dconvolutional neural network (1DCNN), the parameters of which are tuned using a gridsearch methodology. Our design achieved 97.15% accuracy compared to state-of-the-artstatistical approaches. To address the limited energy in wearable settings, we evaluatemodel compression techniques such as quantization. We show that such techniques can leadto a significant reduction in respiratory classification accuracy in order to minimize energy.To address this important problem, we propose a novel spiking neural network (SNN)-based respiratory classification technique, which can be implemented efficiently on anevent-driven neuromorphic hardware. SNN-based respiratory classification involves twoadditional pipeline stages: model conversion and parameter tuning. Using respiration datacollected from a Laerdal SimBaby programmable infant mannequin, we demonstrate 93.33%respiratory classification accuracy with 18x lower energy compared to the conventional1DCNN model. We conclude that SNNs have the potential to implement respiratory

Electronics 2022, 11, 682 20 of 22

classification and other machine learning tasks on energy-constrained environments suchas wearable systems.

Author Contributions: Conceptualization, A.P., M.A.S.T. and W.M.M.; investigation and software, A.P.and M.A.S.T.; writing—original draft preparation, A.P.; writing—review and editing, A.D., K.R.D.,W.M.M. and M.A.S.T. All authors have read and agreed to the published version of the manuscript.

Funding: Our research results are based upon work supported by the National Science FoundationDivision of Computer and Network Systems under award number CNS-1816387. Any opinions,findings, and conclusions or recommendations expressed in this material are those of the author(s)and do not necessarily reflect the views of the National Science Foundation. Research reportedin this publication was supported by the National Institutes of Health under award number R01EB029364-01. The content is solely the responsibility of the authors and does not necessarily representthe official views of the National Institutes of Health.

Institutional Review Board Statement: Not applicable.

Informed Consent Statement: Not applicable.

Conflicts of Interest: The authors declare no conflict of interest.

References1. Eichenwald, E.C.; Watterberg, K.L.; Aucott, S.; Benitz, W.E.; Cummings, J.J.; Goldsmith, J.; Poindexter, B.B.; Puopolo, K.;

Stewart, D.L.; Wang, K.S. Apnea of Prematurity. Pediatrics 2016, 137, e20153757 . [CrossRef] [PubMed]2. Clements, J.A.; Avery, M.E. Lung surfactant and neonatal respiratory distress syndrome. Am. J. Respir. Crit. Care Med. 1998, 157,

S59–S66. [CrossRef] [PubMed]3. Rocha, G.; Soares, P.; Gonçalves, A.; Silva, A.I.; Almeida, D.; Figueiredo, S.; Pissarra, S.; Costa, S.; Soares, H.; Flôr-de Lima, F.; et al.

Respiratory care for the ventilated neonate. Can. Respir. J. 2018, 2018, 7472964. [CrossRef] [PubMed]4. Antognoli, L.; Marchionni, P.; Nobile, S.; Carnielli, V.P.; Scalise, L. Assessment of cardio-respiratory rates by non-invasive

measurement methods in hospitalized preterm neonates. In Proceedings of the 2018 IEEE International Symposium on MedicalMeasurements and Applications (MeMeA), Rome, Italy, 11–13 June 2018.

5. Patron, D.; Mongan, W.; Kurzweg, T.P.; Fontecchio, A.; Dion, G.; Anday, E.K.; Dandekar, K.R. On the Use of Knitted Antennasand Inductively Coupled RFID Tags for Wearable Applications. IEEE Trans. Biomed. Circuits Syst. 2016, 10, 1047–1057. [CrossRef]

6. Tajin, M.A.S.; Amanatides, C.E.; Dion, G.; Dandekar, K.R. Passive UHF RFID-based Knitted Wearable Compression Sensor. IEEEInternet Things J. 2021, 8, 13763–13773. [CrossRef]

7. Mongan, W.; Anday, E.; Dion, G.; Fontecchio, A.; Joyce, K.; Kurzweg, T.; Liu, Y.; Montgomery, O.; Rasheed, I.; Sahin, C.; et al.A Multi-Disciplinary Framework for Continuous Biomedical Monitoring Using Low-Power Passive RFID-Based Wireless Wear-able Sensors. In Proceedings of the 2016 IEEE International Conference on Smart Computing (SMARTCOMP), St. Louis, MO, USA,18–20 May 2016.

8. Ross, R.; Mongan, W.M.; O-Neill, P.; Rasheed, I.; Dion, G.; Dandekar, K.R. An Adaptively Parameterized Algorithm EstimatingRespiratory Rate from a Passive Wearable RFID Smart Garment. In Proceedings of the 2021 IEEE 45th Annual Computers,Software, and Applications Conference (COMPSAC), Madrid, Spain, 12–16 July 2021.

9. Vora, S.A.; Mongan, W.M.; Anday, E.K.; Dandekar, K.R.; Dion, G.; Fontecchio, A.K.; Kurzweg, T.P. On implementing anunconventional infant vital signs monitor with passive RFID tags. In Proceedings of the 2017 IEEE International Conference onRFID (RFID), Phoenix, AZ, USA, 9–11 May 2017.

10. Gentry, A.; Mongan, W.; Lee, B.; Montgomery, O.; Dandekar, K.R. Activity Segmentation Using Wearable Sensors for DVT/PERisk Detection. In Proceedings of the 2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC),Milwaukee, WI, USA, 15–19 July 2019.

11. Tajin, M.A.S.; Mongan, W.M.; Dandekar, K.R. Passive RFID-based Diaper Moisture Sensor. Sensors 2020, 21, 1665–1674. [CrossRef]12. LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444. [CrossRef]13. Maass, W. Networks of spiking neurons: The third generation of neural network models. Neural Netw. 1997, 10, 1659–1671.

[CrossRef]14. Debole, M.V.; Taba, B.; Amir, A.; Akopyan, F.; Andreopoulos, A.; Risk, W.P.; Kusnitz, J.; Otero, C.O.; Nayak, T.K.; Appuswamy, R.; et al.

TrueNorth: Accelerating from zero to 64 million neurons in 10 years. Computer 2019, 52, 20–29. [CrossRef]15. Davies, M.; Srinivasa, N.; Lin, T.H.; Chinya, G.; Cao, Y.; Choday, S.H.; Dimou, G.; Joshi, P.; Imam, N.; Jain, S.; et al. Loihi: A

neuromorphic manycore processor with on-chip learning. IEEE Micro 2018, 38, 82–99. [CrossRef]16. Moradi, S.; Qiao, N.; Stefanini, F.; Indiveri, G. A scalable multicore architecture with heterogeneous memory structures for

dynamic neuromorphic asynchronous processors (DYNAPs). IEEE Trans. Biomed. Circuits Syst. 2017, 12, 106–122. [CrossRef][PubMed]

Electronics 2022, 11, 682 21 of 22

17. Gyselinckx, B.; Vullers, R.; Hoof, C.V.; Ryckaert, J.; Yazicioglu, R.F.; Fiorini, P.; Leonov, V. Human++: Emerging Technology forBody Area Networks. In Proceedings of the 2006 IFIP International Conference on Very Large Scale Integration, Nice, France,16–18 October 2006. [CrossRef]

18. Mongan, W.; Dandekar, K.; Dion, G.; Kurzweg, T.; Fontecchio, A. Statistical analytics of wearable passive RFID-based biomedicaltextile monitors for real-time state classification. In Proceedings of the 2015 IEEE Signal Processing in Medicine and BiologySymposium (SPMB), Philadelphia, PA, USA, 12 December 2015.

19. Acharya, S.; Mongan, W.M.; Rasheed, I.; Liu, Y.; Anday, E.; Dion, G.; Fontecchio, A.; Kurzweg, T.; Dandekar, K.R. Ensemblelearning approach via kalman filtering for a passive wearable respiratory monitor. IEEE J. Biomed. Health Inform. 2018, 23,1022–1031. [CrossRef] [PubMed]

20. Navaneeth, S.; Sarath, S.; Nair, B.A.; Harikrishnan, K.; Prajal, P. A deep-learning approach to find respiratory syndromes ininfants using thermal imaging. In Proceedings of the 2020 International Conference on Communication and Signal Processing(ICCSP), Chennai, India, 28–30 July 2020.

21. Basu, V.; Rana, S. Respiratory diseases recognition through respiratory sound with the help of deep neural network. InProceedings of the 2020 4th International Conference on Computational Intelligence and Networks (CINE), Kolkata, India,27–29 February 2020.

22. Ravì, D.; Wong, C.; Deligianni, F.; Berthelot, M.; Andreu-Perez, J.; Lo, B.; Yang, G.Z. Deep learning for health informatics. IEEE J.Biomed. Health Inform. 2016, 21, 4–21. [CrossRef] [PubMed]

23. Van Steenkiste, T.; Groenendaal, W.; Deschrijver, D.; Dhaene, T. Automated sleep apnea detection in raw respiratory signalsusing long short-term memory neural networks. IEEE J. Biomed. Health Inform. 2018, 23, 2354–2364. [CrossRef]

24. Henaff, M.; Bruna, J.; LeCun, Y. Deep convolutional networks on graph-structured data. arXiv 2015, arXiv:1506.05163.25. Bejnordi, B.E.; Veta, M.; Van Diest, P.J.; Van Ginneken, B.; Karssemeijer, N.; Litjens, G.; Van Der Laak, J.A.; Hermsen, M.;

Manson, Q.F.; Balkenhol, M.; et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases inwomen with breast cancer. JAMA 2017, 318, 2199–2210. [CrossRef]

26. Sannino, G.; De Pietro, G. A deep learning approach for ECG-based heartbeat classification for arrhythmia detection. FutureGener. Comput. Syst. 2018, 86, 446–455. [CrossRef]

27. Das, A.; Pradhapan, P.; Groenendaal, W.; Adiraju, P.; Rajan, R.; Catthoor, F.; Schaafsma, S.; Krichmar, J.; Dutt, N.; Van Hoof, C.Unsupervised heart-rate estimation in wearables with Liquid states and a probabilistic readout. Neural Netw. 2018, 99, 134–147.[CrossRef]

28. Balaji, A.; Corradi, F.; Das, A.; Pande, S.; Schaafsma, S.; Catthoor, F. Power-accuracy trade-offs for heartbeat classification onneural networks hardware. J. Low Power Electron. 2018, 14, 508–519. [CrossRef]

29. Masquelier, T. Epileptic seizure detection using a neuromorphic-compatible deep spiking neural network. In Proceedings of theInternational Work-Conference on Bioinformatics and Biomedical Engineering, Granada, Spain, 8 May 2020.

30. Su, Z.; Cheung, S.C.; Chu, K.T. Investigation of radio link budget for UHF RFID systems. In Proceedings of the 2010 IEEEInternational Conference on RFID-Technology and Applications, Guangzhou, China, 17–19 June 2010.

31. Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT Press: London, UK, 2016.32. Bergstra, J.; Bengio, Y. Random search for hyper-parameter optimization. J. Mach. Learn. Res. 2012, 13, 28–305.33. Liu, X.; Li, W.; Huo, J.; Yao, L.; Gao, Y. Layerwise sparse coding for pruned deep neural networks with extreme compression ratio.

In Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA, 7–12 February 2020.34. Choukroun, Y.; Kravchik, E.; Yang, F.; Kisilev, P. Low-bit Quantization of Neural Networks for Efficient Inference. In Proceedings

of the ICCV Workshops, Seoul, Korea, 27 October–2 November 2019; pp. 3009–3018.35. Coelho, C.N.; Kuusela, A.; Li, S.; Zhuang, H.; Ngadiuba, J.; Aarrestad, T.K.; Loncar, V.; Pierini, M.; Pol, A.A.; Summers, S.

Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors. Nat.Mach. Intell. 2021, 3, 675–686. [CrossRef]

36. Gulli, A.; Pal, S. Deep Learning with Keras, 1st ed.; Packt Publishing: Birmingham, UK, 26 April 2017.37. Das, A.; Catthoor, F.; Schaafsma, S. Heartbeat classification in wearables using multi-layer perceptron and time-frequency

joint distribution of ECG. In Proceedings of the 2018 IEEE/ACM International Conference on Connected Health: Applications,Systems and Engineering Technologies, New York, NY, USA, 26–28 September 2018.

38. Dong, M.; Huang, X.; Xu, B. Unsupervised speech recognition through spike-timing-dependent plasticity in a convolutionalspiking neural network. PLoS ONE 2018, 13, e0204596. [CrossRef] [PubMed]

39. Sengupta, A.; Ye, Y.; Wang, R.; Liu, C.; Roy, K. Going deeper in spiking neural networks: VGG and residual architectures. Front.Neurosci. 2019, 13, 95. [CrossRef] [PubMed]

40. Chou, T.; Kashyap, H.; Xing, J.; Listopad, S.; Rounds, E.; Beyeler, M.; Dutt, N.; Krichmar, J. CARLsim 4: An open source libraryfor large scale, biologically detailed spiking neural network simulation using heterogeneous clusters. In Proceedings of the 2018International Joint Conference on Neural Networks (IJCNN), Rio, Brazil, 8–13 July 2018.

41. Balaji, A.; Song, S.; Titirsha, T.; Das, A.; Krichmar, J.; Dutt, N.; Shackleford, J.; Kandasamy, N.; Catthoor, F. NeuroXplorer 1.0:An Extensible Framework for Architectural Exploration with Spiking Neural Networks. In Proceedings of the InternationalConference on Neuromorphic Systems, Knoxville, TN, USA, 27–29 July 2021.

42. Balaji, A.; Song, S.; Das, A.; Krichmar, J.; Dutt, N.; Shackleford, J.; Kandasamy, N.; Catthoor, F. Enabling Resource-Aware Mappingof Spiking Neural Networks via Spatial Decomposition. IEEE Embed. Syst. Lett. 2020, 13, 142–145. [CrossRef]

Electronics 2022, 11, 682 22 of 22

43. Catthoor, F.; Mitra, S.; Das, A.; Schaafsma, S. Very large-scale neuromorphic systems for biological signal processing. In CMOSCircuits for Biological Sensing and Processing; Springer International Publishing: Berlin/Heidelberg, Germany, 2018.

44. Liu, X.; Wen, W.; Qian, X.; Li, H.; Chen, Y. Neu-NoC: A high-efficient interconnection network for accelerated neuromorphicsystems. In Proceedings of the 2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC), Jeju, Korea,22–25 January 2018.

45. Balaji, A.; Wu, Y.; Das, A.; Catthoor, F.; Schaafsma, S. Exploration of segmented bus as scalable global interconnect for neuromor-phic computing. In Proceedings of the 2019 on Great Lakes Symposium on VLSI, Tysons Corner, VA, USA, 9–11 May 2019.

46. Das, A.; Wu, Y.; Huynh, K.; Dell’Anna, F.; Catthoor, F.; Schaafsma, S. Mapping of local and global synapses on spikingneuromorphic hardware. In Proceedings of the 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE),Dresden, Germany, 19–23 March 2018.

47. Titirsha, T.; Song, S.; Balaji, A.; Das, A. On the Role of System Software in Energy Management of Neuromorphic Computing.In Proceedings of the 18th ACM International Conference on Computing Frontiers, Virtual Event, Italy, 11–13 May 2021.

48. Balaji, A.; Das, A.; Wu, Y.; Huynh, K.; Dell’anna, F.G.; Indiveri, G.; Krichmar, J.L.; Dutt, N.D.; Schaafsma, S.; Catthoor, F. Mappingspiking neural networks to neuromorphic hardware. IEEE Trans. Very Large Scale Integr. (VLSI) Syst. 2020, 28, 76–86. [CrossRef]

49. Song, S.; Balaji, A.; Das, A.; Kandasamy, N.; Shackleford, J. Compiling spiking neural networks to neuromorphic hardware.In Proceedings of the 21st ACM SIGPLAN/SIGBED Conference on Languages, Compilers, and Tools for Embedded Systems,London, UK, 16 June 2020.

50. Das, A.; Kumar, A. Dataflow-Based Mapping of Spiking Neural Networks on Neuromorphic Hardware. In Proceedings of the2018 on Great Lakes Symposium on VLSI, Chicago, IL, USA, 23–25 May 2018.

51. Balaji, A.; Das, A. A Framework for the Analysis of Throughput-Constraints of SNNs on Neuromorphic Hardware. In Proceedingsof the 2019 IEEE Computer Society Annual Symposium on VLSI (ISVLSI), Miami, FL, USA, 15–17 July 2019.

52. Balaji, A.; Marty, T.; Das, A.; Catthoor, F. Run-time mapping of spiking neural networks to neuromorphic hardware. J. SignalProcess. Syst. 2020, 92, 1293–1302. [CrossRef]

53. Balaji, A.; Das, A. Compiling Spiking Neural Networks to Mitigate Neuromorphic Hardware Constraints. In Proceedings of theIGSC Workshops, Pullman, WA, USA, 19–22 October 2020.

54. Titirsha, T.; Das, A. Thermal-Aware Compilation of Spiking Neural Networks to Neuromorphic Hardware. In Proceedings of theLCPC, New York , NY, USA, 14–16 October 2020.

55. Song, S.; Das, A.; Kandasamy, N. Improving dependability of neuromorphic computing with non-volatile memory. In Proceedingsof the EDCC, Munich, Germany, 7–10 September 2020.

56. Song, S.; Hanamshet, J.; Balaji, A.; Das, A.; Krichmar, J.; Dutt, N.; Kandasamy, N.; Catthoor, F. Dynamic reliability management inneuromorphic computing. ACM J. Emerg. Technol. Comput. Syst. (JETC) 2021, 17, 1–27. [CrossRef]

57. Song, S.; Das, A. A case for lifetime reliability-aware neuromorphic computing. In Proceedings of the MWSCAS,Springfield, MA, USA, 9–12 August 2020.

58. Kundu, S.; Basu, K.; Sadi, M.; Titirsha, T.; Song, S.; Das, A.; Guin, U. Special Session: Reliability Analysis for ML/AI Hardware.In Proceedings of the VTS, San Diego, CA, USA, 25–28 April 2021.

59. Song, S.; Das, A. Design Methodologies for Reliable and Energy-efficient PCM Systems. In Proceedings of the IGSC Workshops,Pullman, WA, USA, 19–22 October 2020.

60. Titirsha, T.; Song, S.; Das, A.; Krichmar, J.; Dutt, N.; Kandasamy, N.; Catthoor, F. Endurance-Aware Mapping of Spiking NeuralNetworks to Neuromorphic Hardware. IEEE Trans. Parallel Distrib. Syst. 2021, 33, 288–301. [CrossRef]

61. Titirsha, T.; Das, A. Reliability-Performance Trade-offs in Neuromorphic Computing. In Proceedings of the IGSC Workshops,Pullman, WA, USA, 19–22 October 2020.

62. Song, S.; Titirsha, T.; Das, A. Improving Inference Lifetime of Neuromorphic Systems via Intelligent Synapse Mapping.In Proceedings of the 2021 IEEE 32nd International Conference on Application-specific Systems, Architectures and Processors(ASAP), Gothenburg, Sweden, 7–8 July 2021.

63. Mallik, A.; Garbin, D.; Fantini, A.; Rodopoulos, D.; Degraeve, R.; Stuijt, J.; Das, A.; Schaafsma, S.; Debacker, P.; Donadio, G.; et al.Design-technology co-optimization for OxRRAM-based synaptic processing unit. In Proceedings of the 2017 Symposium onVLSI Technology, Kyoto, Japan, 5–8 June 2017.

64. Abadi, M.; Barham, P.; Chen, J.; Chen, Z.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Irving, G.; Isard, M.; et al. Tensorflow:A system for large-scale machine learning. In Proceedings of the 12th USENIX symposium on operating systems design andimplementation (OSDI 16), Savannah, GA, USA, 2–4 November 2016.

65. Coelho, C.N., Jr.; Kuusela, A.; Zhuang, H.; Aarrestad, T.; Loncar, V.; Ngadiuba, J.; Pierini, M.; Summers, S. Ultra low-latency,low-area inference accelerators using heterogeneous deep quantization with QKeras and hls4ml. arXiv 2020, arXiv:2006.10159.