IOP PUBLISHING NANOTECHNOLOGY

Nanotechnology 24 (2013) 000000 (11pp) UNCORRECTED PROOF

(Ed: Andy Thomas)

Ascii/Word/NAN/

nano461325/SPE

Printed 26/4/2013

Spelling US

Issue noTotal pagesFirst pageLast pageFile nameDate reqArtnum

Cover date

A theoretical and experimental study ofneuromorphic atomic switch networks forreservoir computing

Henry O Sillin1, Renato Aguilera2, Hsien-Hang Shieh2,Audrius V Avizienis1, Masakazu Aono3, Adam Z Stieg2,3 andJames K Gimzewski1,2,3

1 Department of Chemistry and Biochemistry, University of California—Los Angeles,607 Charles E Young Drive East, Los Angeles, CA 90095, USA2 California NanoSystems Institute, 570 Westwood Plaza, Los Angeles, CA 90095, USA3 WPI Center for Materials Nanoarchitectonics (MANA), National Institute for Materials Science(NIMS), 1-1 Namiki, Tsukuba, Ibaraki 305-0044, Japan

E-mail: stieg@cnsi.ucla.edu and gimzewski@cnsi.ucla.edu

Received 24 January 2013, in final form 3 April 2013PublishedOnline at stacks.iop.org/Nano/24/000000

AbstractAtomic switch networks (ASNs) have been shown to generate network level dynamics thatresemble those observed in biological neural networks. To facilitate understanding and controlof these behaviors, we developed a numerical model based on the synapse-like properties ofindividual atomic switches and the random nature of the network wiring. We validated themodel against various experimental results highlighting the possibility to functionalize thenetwork plasticity and the differences between an atomic switch in isolation and its behaviorsin a network. The effects of changing connectivity density on the nonlinear dynamics wereexamined as characterized by higher harmonic generation (HHG) in response to AC inputs. Todemonstrate their utility for computation, we subjected the simulated network to trainingwithin the framework of reservoir computing (RC) and showed initial evidence of the ASNacting as a reservoir which may be optimized for specific tasks by adjusting the input gain.The work presented represents steps in a unified approach to experimentation and theory ofcomplex systems to make ASNs a uniquely scalable platform for neuromorphic computing.

S Online supplementary data available from stacks.iop.org/Nano/24/000000/mmedia

(Some figures may appear in colour only in the online journal)

Q.1

Q.2

1. Introduction

Synapses play an essential role in cognitive function.Q.3

Brain activity is characterized by spatio-temporal varyingelectrical signals travelling through a vast interconnectedrecurrent network of neurons, where the synapses mediatesignaling. Until recently, research has tended to focuson an approach that promulgates that advances beyondcomplementary metal–oxide semiconductors (beyond-CMOS)computation [1] may be achieved by fabricating electronicelements that recreate the fundamental behaviors of neuronsand synapses [2, 3]. Accordingly, new generations of synthetic

synapses have been demonstrated or modeled which displayshort- and long-term potentiation/depression (STP, LTP andLTD respectively), spike timing dependent plasticity, andother neuroscientific phenomena [4, 5]. These devices havebeen incorporated into hybrid-CMOS/molecular-CMOS [5,6] circuitry with the aim of recreating synaptic spikingpatterns typically realized with crossbar array geometriesfor optimized memory storage [7–15]. The architecture ofthe arrays is designed to address each functional elementindividually and sequentially in a programmed fashion,essentially precluding the elements from interacting withina network. However, actual complex systems exploit the

10957-4484/13/000000+11$33.00 c© 2013 IOP Publishing Ltd Printed in the UK & the USA

Nanotechnology 24 (2013) 000000 H O Sillin et al

non-trivial effects of interconnectivity [16, 17] that allowindividual units to function in synchrony over multiple spatialand temporal scales resulting in self-organized patterns ofactivity [18–22]. Consequently, emergent phenomena aredistributed throughout the entire system and cannot beassociated with any particular node or local group [23–25].In the brain, modification of these intricate networks isbelieved to form the basis of memory, motor pathways, andcognition [26–29].

Through a combination of top-down and bottom-upfabrication techniques, highly interconnected wire networkscontaining synaptic functional units have been fabricatedand studied [30, 31]. These atomic switch networks(ASNs), composed of more than 109 individual inorganicsynapses cm−2 [32–34], i.e. atomic switches [35], representQ.4

a unique class of physical devices capable of exhibitingsynapse-like properties in neurally inspired architectures.Interfacing functionalized networks with multielectrodearrays offers the ability to harness intrinsic systemdynamics through input and read out of real-time electricalsignals at various spatio-temporal scales toward practicalimplementation of neuromorphic computation [30, 31]. ASNsretain the adaptive plasticity and memory of their componentatomic switches while exhibiting emergent properties suchas criticality and spontaneous switching between discretemetastable resistance states.

Most attempts to mimic the brain’s function use simulatedneural networks. Recurrent neural networks consist of nodes,each with an adjustable connective weight, and allowsignals to propagate forwards and backwards through thenetwork [36]. Such structure allows information to beintegrated at different time points, enabling online training.The main drawback of recurrent networks remains thedifficulty of adjusting individual connective weights, whichresults in an inability to efficiently differentiate inputs oradapt to increasing noise levels in the environment [37]. Thecomputational degrees of freedom are also too vast to permitconvergence in a reasonable time.

Recently, reservoir computing (RC) was developed toovercome these issues, while retaining biologically relevantfeatures such as feedforward and recurrent structures [38–43].RC is achieved through a two-step process. First, timevarying input data are introduced to a fixed weight recurrentnetwork or ‘reservoir’, in which the nonlinear action of nodesproduces higher dimensional representations of the inputdata. This transformation permits the second step, wherethe new representations are read out through a feedforwardnetwork or ‘linear readout’. Information that was not initiallylinearly separable can then be processed through simple linearregression techniques in the feedforward layer. Consequently,RC can perform complicated classifications in real time, andenables generalization of learned tasks. Currently, RC is themost effective technique available for certain tasks and hasbeen realized in a variety of physical implementations as proofof concept [44–46].

ASNs were proposed as well suited to RC, becausethey contain a physically recurrent network of nonlinearelements which are amenable to serve as a reservoir while

also exhibiting feedforward properties useful for the outputlayer [31]. ASNs also possess a readily scalable architecture,multiple spatio-temporal outputs, and synaptic nonlinearelements displaying critical dynamics. Consequently, ASNsare potential embodiments for enabling neuromorphiccomputational theories and represent physical neuromorphicdevices that have a direct connection with neuroscience.

Understanding and harnessing the rich dynamics found incomplex networks comprised of coupled nonlinear elementsis challenging. In particular, practical engineering goalsare based on stimulus–response relationships governed byinternal system reorganization with minimal fine tuning atthe microscopic level. Here we present a numerical modelof ASNs which illustrates key aspects of the spatial andtemporal dynamics of the system, and investigates their utilityin the context of RC. Our simulation was built from thephysical ASN devices: the design and implementation wasbased on the well-documented physics of single synapticswitches, and the connectivity was modeled on the knownnetwork architectures determined from SEM images. Thegoal of the simulation was to deepen our understandingof network function, optimize network design and explorethe applicability of device architectures for neuromorphiccomputational tasks.

First, we validate the accuracy of the simulation byreproducing data from the devices such as controlledinterconnect plasticity and emergent behavior, despite thesimulation’s much smaller network size. Second, we show thatfluctuations in a simulated isolated single link are distinctlydifferent from those of an identical link within the network.This highlights the role of the network where synapticelements behave differently as a result of interconnectivity.The simulation is also found to show emergent properties thatare impossible to measure in a single device. Third, we showthat higher harmonic generation reported experimentally [30]and theoretically predicted [47, 48] can be modeled andutilized for reservoir computing. We finally discuss thepotential impact of ASNs as a unique physical embodimentthat is capable of integrating neuromorphic architecture,dynamics, and computation.

2. Methods

2.1. Network devices

The ASNs were grown using self-assembly of a rhizomic–dendritic network of highly interconnected silver (Ag)nanowires which were sulfurized to provide distributednanojunctions comprised of inorganic synthetic synapses. TheASNs were interfaced to input–output electrodes fabricatedusing conventional micro-lithographic processing to createa functional device [30, 31] (figure 1(a)). Electrodeswere fabricated by electron beam evaporation followingphotolithography on the surface of a p-type Si wafer (borondoped, 0–100 � cm−1) insulated by a 500 nm thick thermaloxide layer. Deposition of 4–16 Cr/Pt (15/150 nm) electrodeswith diameters of 10–50 µm at pitches between 50 and500 µm was followed by spin-coating and patterning of an

2

Nanotechnology 24 (2013) 000000 H O Sillin et al

Figure 1. (a) Atomic switch network devices are fabricated on a SiO2 substrate with 16 Pt electrodes and an insulating SU-8 layer. Thedevices are approximately 4 cm2. (b) The resultant Ag wires vary in size (<100 nm to >1 mm) and create self-assembled networks withcomplex interconnections (109 cm−2). The electrodes shown have 10 µm diameter and 50 µm pitch, and range up to 50 µm diameter with500 µm pitch. (c), (d) The density of interconnections can be changed by altering the size/pitch of the Cu posts shown: (c) 1 µm/5 µm; (d)1 µm/1 µm. Scale bars = 10 µm.

insulating layer of SU-8 deep UV resist, which served toexpose only point contact regions of the electrodes. Microndiameter cylindrical copper seeds were then deposited at arealdensities between 1 × 106 and 2.5 × 107 seed sites cm−2

for the electroless deposition (also referred to as galvanicdisplacement or cementation) of Ag from Cu on theSU-8 layer. Electroless deposition was performed underdiffusion-limited conditions using 50 mM aqueous solutionsof AgNO3 [49]. This spontaneous electrochemical reactionproduced the complex networks of metallic silver nanowiresshown in figure 1(b). Previous studies have shown that thegeometry and spacing of pre-patterned Cu posts providescontrol over the global qualities of the network, generatingstructures ranging from extended nanowires to dendrites andfractals [50, 51]. The pitch of the Cu posts was found todetermine the relative density of the network (figures 1(c), (d))while seed size was used to control the presence of long-rangeconnections. Typically, smaller Cu seeds (<3.5 µm) producemany long wires and larger seeds (>3.5µm) produce spatiallyconfined dendritic or fractal structures.

The self-assembled silver networks intrinsically containcrossbar-like junctions resulting from the three-dimensionalnature of the solution deposition process. Upon exposureto sulfur gas [52] (10−1 Torr at 130 ◦C for 3 min), theAg nanowire junctions are functionalized to form thinAg|Ag2S|Ag metal–insulator–metal interfaces which, in thepresence of post-processing activation with external biaspotential, are transformed into ‘atomic switches’. Electricalcharacterization of the devices was conducted throughcurrent–voltage (I–V) spectroscopy using a bipotentiostat

(Pine Instruments model AFCBP1) in conjunction witheither a data acquisition module (National Instruments USB6259) or a multiplexed (National Instruments PXI 1073)source-measurement unit (National Instruments PXI 4130).The maximum bandwidth of the measurement systems was1 MHz and 10 kHz enabling 2 Ms and 20 ks s−1 with 16-bitresolution. Subsequent data analyses were carried out usingMATLAB 2010b (MathWorks) and Origin 8.1 (OriginLabCorporation).

2.2. Network model and simulation

Simulation efforts employed previously have reported Q.5

physical properties of atomic switches composed of aAg|Ag2S|Ag interface, shown schematically in figure 2, thatexhibit both volatile and non-volatile memory propertiesas well as multi-state switching [32, 33, 35]. Atomicswitches are known to operate through two mechanisms:(i) formation/dissolution of conductive filaments, and (ii) aphase transition between monoclinic acanthite (α) and bodycentered cubic argentite (β) Ag2S. Application of a biasvoltage across the junction induces the formation of nanoscaleconducting channels across the Ag2S interface through abias-catalyzed phase transition, converting the surroundingα-Ag2S matrix to a conductive and β-Ag2S phase whichexhibits high ionic mobility as illustrated by TEM-electrondiffraction studies [53]. In the absence of continued appliedbias, the conductive channels eventually return to theirstoichiometric, thermodynamically favored equilibrium state,which reverts the atomic switch to its initial high resistance.

3

Nanotechnology 24 (2013) 000000 H O Sillin et al

Figure 2. Atomic switches are comprised of a Ag|Ag2S|Agjunction. Applied electrical bias causes Ag cation migration to thecathode where it is reduced, forming a stable metallic filament,resulting in resistance change. This migration is modeled by thefilament length w(t), Ag cation mobility µv, and additionalstochastic terms.

This transition gives rise to a weakly memristive behaviorprior to the formation of Ag filaments across the interface.

Continued application of bias voltage results in aconcurrent increase in current through the device, which thenfurther drives migration of silver cations toward the cathode.At the cathode, mobile silver cations are subsequently reducedto Ag0, forming a highly conductive Ag nanofilamentary wire.The completion of this filament results in a strong transitionto an ON state with a dramatic increase in conductivity [53].Removal of the applied bias results in filament dissolution asthe device again returns its thermodynamic equilibrium state.The completion and dissolution of this filament characterizesstrongly memristive behavior. Continuous application of abias voltage serves to increase filament thickness as additionalsilver cations are reduced, causing thickening of the metallicfilament. This dynamic process has been shown to alter thedissolution time constant, and can be externally controlledby changing the input bias pattern (e.g. pulse frequency).Such changes in volatility can be interpreted as long-term orshort-term potentiation (LTP and STP) [32, 33].

Using a similar construction, a recent report simulated anearest neighbor grid of resistors and memristors [48]. Here,the memristive equations were augmented with new termsthat reflect the known properties intrinsic to Ag|Ag2S|Agatomic switches as well as network effects. The state variableof the memristive elements was chosen to be the length ofthe Ag filament, represented as w(t) ∈ [0,w0], where thejunctions’ gap sizes, w0, were randomized with a mean of5 nm according to known values [32, 33, 35]. The voltageacross each atomic switch junction was given by

V(t) =

[Ron

w(t)

w0+ Roff

(1−

w(t)

w0

)]I(t) (1)

where Ron and Roff represent the resistance values for theON/OFF states of, and I(t) the current across, the atomicswitch junction. The rate of change in filament length wasmodeled according to

dw(t)

dt=

[µv

Ron

w0I(t)

]�− τ(w(t)− w0)+ η(t) (2)

where µv signifies the ionic mobility [54, 55]. The windowfunction, given by

� =

[w(t)(w0 − w(t))

w20

](3)

was included in (2) to incorporate the behavior of elementswith state variables at the extreme limits due to ionic drift [48,55]. The term [µv

Ronw0

I(t)]� provided the dependence of thefilament growth rate on the electronic flux, while τ(w(t) −w0) operated as a dissolution term that served to returnthe filament length to its original value w0 due to thethermodynamic stability of the high resistance state. Althoughthe dissolution rate constant (τ ) has not been extensivelyinvestigated in the network setting, a numerical survey overthree orders of magnitude (1–1000 s−1) determined thevalue that best reproduced the prior experimental resultsfor the size of the simulated network. Lastly, a stochasticterm η(t) accounted for fluctuations in the density ofavailable silver ions and the stochastic nature of the filamentformation/dissolution process in physical ASNs. This term,defined by

η(t) = α(t)1w(t) (4)

governed the growth rate of filament sizes, where 1w(t)represented the change in filament length at time t reflectingthe amount of electric flux through the switch junction. Here,the random variable α(t) introduced a noise factor to theterm η(t) that was distributed across the network following arandom distribution centered at zero with a standard deviationσα . The probability distributions for both w0 and α wereexamined using simulated network sizes ranging from 3×3 to10× 10 arrays with varying degrees of connectivity density.

Finally, the connectivity in the simulation was modeledafter the known interconnectivity of ASNs [31]. Physicalnetworks are grown from copper seeds spatially distributedon the substrate, which serve as nucleation sites forthe electroless deposition of silver metal under diffusion-limited conditions [51]. This directed self-assembly processgenerates networks with a large distribution of wire lengths,characterized by SEM in the range of 100 nm to over 1 mm.The networks contain elements of a nearest neighbor network,since the wires are likely to connect to those grown fromnearby copper seeds (figures 1(c)–(d)). They also exhibitcharacteristics of a random network as long wires extendacross the entire network, connecting distant nodes, and alsoconnecting many nodes at once (figure 1(b)).

Connectivity in the simulation was created by startingwith a square lattice of nodes, mimicking the copper posts inthe hardware design. Links connecting nearest neighbors aswell as distant nodes were then assigned randomly, with thetotal number of connections ranging from N = 50 to 400. Thesimulation results reported here represent the typical networkresponse observed with connections reassigned for each run.Finally, the values of the physical parameters such as ionicmobility, RON/ROFF ratio, and average gap size were chosenaccording to experimental literature values as schematicallyillustrated in figure 2 [32, 33, 35], leaving only the network

4

Nanotechnology 24 (2013) 000000 H O Sillin et al

Figure 3. Simulation of device activation using a 10× 10 network with N = 126, average Ron/Roff = 10−2 and τ = 10 s−1 under a trianglewave input bias of ±2 V at 10 Hz, demonstrating (a) an initial soft switching (σα = 10%) repeated indefinitely, until (b) a transition inbehavior from soft (blue, σα = 10%) to hard (red, σα = 0%) switching. (c) Hard switching persists indefinitely with σα = 0%. Thisbehavior was ubiquitous across all configurations with discrepancies in the bias amplitude/frequency. Experimental device activation curvesare shown as insets for comparison.

Table 1. Parameters used in the simulation were tested over ranges that are physically relevant to the atomic switch network system: totalgap width (w0); ionic mobility (µv) of Ag+ in Ag2S; ratio of resistances (Ron/Roff) at w = 0 and w = w0; filament dissolution rate constant(τ ); modulation (α) level of noise in the w(t) term with each time step; and total number of connections (N).

w0 (nm) µv (m2 s−1 V−1) Ron/Roff τ (s−1) α N

Ave: 5 0.5× 10−12 Ave: 10−1–10−3 1–103 Ave: 0 50–400σw0 : 0–40% σon/off: 0–40% σα : 0–30%of 1w(t)

size and wiring density as free parameters. table 1 summarizesthe range of the values of the parameters explored to producethe results in this report.

Network simulations were executed in MATLAB througha graphical user interface (GUI) that provides controlover these parameters in addition to the locations andnumbers of sources/drains, structure of input waveforms, andmagnitude of input signals. The GUI also provides directmonitoring of simulated I–V statistics as well as the current,power dissipation and conductivity of each link within thesimulated network (figure S1 available at stacks.iop.org/Nano/24/000000/mmedia).

3. Results and discussion

Various studies were carried out under conditions similarto the physical implementations in order to comparethe numerical model’s results with prior ASN deviceexperiments [31]. Results from these simulations wereexamined in the context of network complexity on the basis ofthe underlying device physics and their associated emergentproperties. The simulation was then used to explore theparameter space of ASNs, establishing a systematic approachto optimize network performance in the context of giventraining schemes and computational tasks.

3.1. Device activation

Robust, hysteretic switching in ASN devices requires deviceactivation by a symmetric triangle wave ramp appliedacross the network. As net flux through the networkincreases, connections become increasingly polarized andconductive, resulting in different behavioral regimes (insets

of figures 3(a)–(c)). A lack of completed metal filamentscharacterizes the initial state in figure 3(a) (inset) as the ‘softswitching’ regime. Continued sweeping causes the formationof a continuously conductive path across the network, withintermediate connections operating in a higher conductancestate. This transition is observed as a dramatic changein conductance, shown in the inset of figure 3(b), wherenetwork response changes from ‘soft’ to ‘hard switching’as the fraction of strongly memristive elements increasespast the percolation threshold [31]. Continuous sweepingof the applied bias is known to encourage thickeningand stabilization of as-formed nanofilaments [32, 33, 35],producing behavior that is robust to fluctuations in silver iondeposition/dissolution (figure 3(c) inset). As a result, thereis an increased likelihood for an element in the network tooperate in the strongly memristive regime.

To reproduce this activation process, the simulationincluded the effects of filament stability on network responseby tuning the distribution of noise factor α(t) while holdinginput bias amplitude and frequency constant. A high noiselevel (σα > 7%) was found to inhibit stable transitions, orw(t) from crossing the interfacial barrier in the windowfunction, which enforced the soft switching state (figure 3(a)).Lowering σα enables w(t) to increase past the barrierwithout interruption, inducing the transition from soft tohard switching at the network level (figures 3(b)–(c)).Consequently, the noise level serves as a control parameterfor the number of strongly memristive elements and thesoft/hard switching behavior, supporting the conjecture on themechanism behind the different switching behaviors.

Selecting the appropriate strength of the stochastic termenabled the ASN simulation to agree qualitatively withthe experimental memristive behavior. While simulationand experiment show a quantitative difference in the hard

5

Nanotechnology 24 (2013) 000000 H O Sillin et al

switching regime (figure 3(c) and inset), specifically inthe rate of change between the high and low resistancestates, stronger agreement with respect to the Ron/Roffratio (figure S2 available at stacks.iop.org/Nano/24/000000/mmedia) and the rate of resistance state change (figure 6(b′′))was observed in a more densely connected simulated network.To elucidate the underlying dynamics of the activationprocess, internal conductance maps of a sparse networkreveal the conductive pathways responsible for maximumcurrent flow when operating in the soft switching, transitional,and hard switching states (figure S3 available at stacks.iop.org/Nano/24/000000/mmedia). A single, dominant pathwayemerged at the transitional state and was destroyed in thesubsequent input bias sweeps that drove the network intothe hard switching state. Further examination of functionalconnectivity over the entire activation process (figure S4available at stacks.iop.org/Nano/24/000000/mmedia) enabledidentification of equivalent regions of network conductancefor the transitional and hard switching states. In particular,the network followed different trajectories to achieve valuesof maximum conductance, whereby network conductance wasincreasingly distributed in nature for the hard switching case.

In the results described below, both w0 and α(t) weresampled from Gaussian distributions. Parameters of thedistribution of w0 were selected to reproduce the experimentalI–V curves by matching the bias voltage used in thesimulation to our experiment.

3.2. Recurrent structure

Physically implemented ASNs are observed to exhibitnon-equilibrium dynamics under applied DC bias [31].These network fluctuations are attributed to two primarymechanisms: (i) external bias causes silver ion migrationtoward the cathode where the ions are reduced toform metallic filaments, in opposition to the stochastic,thermodynamically driven return to equilibrium, and (ii)fluctuations in local resistance within the highly recurrentnetwork can trigger cascading resistance changes elsewherein the system. The behavior can be likened to neuronalavalanches observed in multielectrode array studies ofneuronal cultures [18, 31]. In a single isolated link(figure 4(a)), the stochastic term η(t) results in thegeneration of white noise in the current output. In contrast,when embedded within a recurrent network, single linksdisplay 1/f γ power law scaling. This clear difference incharacteristics shows the role of connectivity in a network.Each link in the network receives voltage inputs from manylocations in the system and integrates, which in turn modifiesthe link’s instantaneous resistance. This behavior facilitatesthe emergence of spatially correlated structures in localnetwork activity. The recurrent structure also enables theintegration of signals originating at different points in time,giving rise to non-trivial correlations in the temporal domainas indicated by 1/f γ power law scaling with 0 < γ < 2 inthe power spectral density (figure 4). The current passingacross the ASN in both simulation and physical measurementsalso displays 1/f γ in the power spectrum (figure 4(b)) [56].

Figure 4. Comparisons of the power spectrum of a simulated10× 10 network with N = 332, average Ron/Roff = 10−2,τ = 10 s−1, σα = 2.5% (blue). (a) Comparison with a simulation ofa single isolated atomic switch with identical parameters. Theisolated atomic switch (red) shows a power spectrum dominated bywhite noise in contrast with a single element in the network (blue)showing 1/f γ power law scaling. (b) Comparison with a networkdevice (black) that shows 1/f γ power law scaling with γ = 1.87(simulation) and 1.78 (device).

Although challenging to characterize in physical devices, thesimulation data provide insight to both spatial correlation andphase synchrony for experiment.

3.3. Feedforward subassemblies

Network plasticity was investigated as a mechanism for

Q.6

the formation of feedforward pathways within ASNs,although they have an inherently recurrent architecture.Previous studies on physical ASNs have demonstrated theirfunctionality as a two-bit memory storage device withspatially controlled, independent switching channels usingpulsed electrical stimulation [31]. This experimental result,which used macroscopic electrodes in contact with a largearea of the network to apply bias voltage stimulation, was alsosuccessfully simulated as illustrated in figure 5. To complywith the experimental setup, a 10 × 10 network simulationwas partitioned such that in each corner, a 4 × 4 block ofnodes served the same purpose as a physical device electrode.A channel was defined by selection of one block as the source

6

Nanotechnology 24 (2013) 000000 H O Sillin et al

Figure 5. Spatially overlapping channels A and B can be modifiedindependently by write/rewrite pulses, emulating the 2-bit switchingfunctionality of actual device behavior (inset). This simulated10× 10 network (N = 219, average Ron/Roff = 10−3, σα = 2.5%)was partitioned with four separate 4× 4 blocks to serve aselectrodes. Spatially defined ON/OFF switching was induced byapplying write/rewrite voltage pulses (15 V, 10 ms duration) acrossthe channels specified in the figure. Measurements of conductanceacross all six possible channels were conducted with 1 V readpulses of negligible period.

and another as the drain for application of an input biasvoltage, with four blocks allowing for six possible channels.As shown in figure 5, suprathreshold training pulses appliedacross two channels altered their respective conductancesindependently, even though the pathways were physicallyoverlapping. By monitoring simulated connectivity maps ofthe other four conductance channels during this process,dynamical reconfigurations of the network connectivity wereobserved (figure S5 available at stacks.iop.org/Nano/24/000000/mmedia). Thus, non-volatile memory write/rewritesteps occur concurrently with non-trivial changes elsewherein the network. Different write/rewrite pulse combinationscan store information while simultaneously allowing thenetwork to evolve through new configurations. Investigationsof structures and stability of feedforward subassemblies maybe conducted by carrying out random or targeted deletionof links belonging to a given channel in relation to thestrength and duration of the external pulses that induced itsformation. Scaling the network size increases the numberof distinguishable network states, allowing for increasedmemory storage and diversity of nonlinear interactions.

3.4. Nonlinear network dynamics

Resistive switches have recently attracted attention forhigher harmonic generation (HHG), presented in bothsingle switches and networks [47, 48]. Experimental atomicswitch networks show HHG to be a function of appliedinput bias amplitude [31]. Here, numerical simulation wasemployed to explore HHG by stimulating a network with asinusoidal input (10 Hz) while varying the input amplitudeand network connection density (connectivity). For eachsimulated network of a given connectivity, the HHG analysiswas performed on data collected over 10 cycles of theinput signal. The network was then reset to the same initial

state for the next level of input amplitude. A sharp rise inthe ratio of higher harmonic amplitudes to the fundamentalat a threshold voltage was found in both experiment andsimulation (figure 6(a)).

The network I–V response curves at increasing levels ofinput bias amplitude (figures 6(b′)–(b′′′)) illustrate the onsetof nonlinearity as characterized by HHG in figures 6(a)–(c).As the voltage increased past the threshold magnitude, theswitching behavior moved progressively toward the hardswitching regime. The threshold voltage decreased withincreased density of connections as shown in figure 6(c). Anincrease in connectivity provides more recurrent substructuresin the network and can be related to the nonlinearity inthe integration of electrical responses within the system.The decrease in the magnitude of the threshold voltage canbe attributed to an enhanced nonlinearity in the networkdynamics where, even at lower bias voltage levels, the systemtends to reside outside of a linear regime. These changesin the behavior of HHG suggest a way toward quantitativecharacterization of functional connectivity within ASNs basedon their dynamics. As illustrated in figure 1 and describedelsewhere [51], the fabrication conditions for ASNs allowsubstantial control over the size, morphology and density ofinterconnects. Simulations may therefore be used to optimizephysical networks for specific applications.

3.5. Reservoir computing

Higher harmonic generation is potentially of great usefulnessin a modern computational paradigm utilizing recurrentcomplex networks, called reservoir computing (RC). Theamplitude and frequency characteristics of the producedhigher harmonics may be used to quantitatively evaluate theefficiency of a reservoir in different dynamical regimes byaccounting for the accessible degrees of freedom in higherdimensional representation space. Using voltage time tracesas outputs, it is shown here that the ASN can effectively serveas a nonlinear reservoir capable of performing the waveformgeneration task (figure 7) considered as a prerequisite toperform reservoir computing [37].

Maximizing the number of output signals is advantageousin the context of RC training. Each network node wastherefore chosen to serve as an output electrode. By subjectingthe network to a sinusoidal input at one corner electrode in theform of an input bias voltage, multiple waveforms includingtriangle, square and frequency doubling sinusoidal waveformswere constructed through superposition of voltage outputs ateach electrode in the simulation. The generated waveformrq (q = 1, 2, 3) was then a weighted sum of the voltageoutputs from the electrodes with the weights W i

q calculatedby linear regression,

rq =

m∑i=1

W iqVi, q = 1, 2, 3 (5)

where Vi are the output electrode voltages. Reservoirperformance was assessed by the quality of waveformgeneration and compared across networks with different

7

Nanotechnology 24 (2013) 000000 H O Sillin et al

Figure 6. Higher harmonic generation can be influenced by network connectivity and input amplitude. Harmonic overtones of severalsimulated 10× 10 networks with average Ron/Roff = 10−2, σα = 2.5%, and τ = 10 s−1. (a) The first three harmonic overtones of a networkwith N = 332 showed a threshold voltage for higher harmonic generation. Experimental device curves are shown as an inset forcomparison. (b) Harmonic generation as a function of input bias amplitude for a network of intermediate connectivity (N = 229).(b′, b′′, b′′′) The network I–V characteristics tend toward hard switching behavior and increased higher harmonic generation as a function ofinput bias amplitude. (c) The sum of the first three harmonic overtones of several simulated networks with N = (126; 229, 332) indicated ashift toward lower threshold voltages with increasing connectivity ascribed to an increasingly complex network.

parameters by calculating the mean square error (MSE),which quantifies the differences between the target and thegenerated waveforms,

MSE =

∑Pn=1

(ytarget(tn)−

∑mi=1 W iVi(tn)

)2P

(6)

where ytarget is the target waveform. The W i represent theweight coefficients to be trained with maximum numberof outputs (m = 16) at discrete time indices (tn) over atotal length (P) from n = 1 to 9000. To see whether thediverse dynamical regimes of ASNs may affect the efficiencyof RC training, the input gain was varied to access thedifferent characteristics generated of the higher harmonics.The weights were calculated independently using the outputresponses from the network at each input voltage amplitude.As the magnitude of the applied input voltage was increased,an abrupt onset of HHG (section 3.4) was observed toinfluence the associated error for the waveform generationtasks. Specifically, the ratio of the second to higher harmonics(calculated with unweighted average of network outputs)dropped rapidly with increasing voltage (≈8 V) as seen infigure 7(c). This bias dependent reduction in second harmonicamplitude was found to correlate directly with the criticalvoltage where MSE increased dramatically in the task ofgenerating a sine wave exhibiting frequency doubling of theinput signal. As the second harmonic became less pronouncedcompared to the higher harmonics it became more difficult

to isolate in a linear combination of the output signalsthrough linear regression. In contrast, an increased ratio of Q.7

higher harmonics to the second harmonics resulted in betterperformance for the generation of waveforms containinghigher harmonic components such as square and triangularwaveforms. While HHG is not a universal parameter designedto indicate increased computational performance, it doesfaithfully explain computational performance on specific RCtasks.

The results clearly demonstrate that an ASN can be usedas a pattern generating kernel in RC where it can be optimizedby adjusting input gain and network connectivity. TrainingASNs to carry out more complex tasks requiring multiple,simultaneous inputs/outputs as well as real-time feedback iscurrently under investigation [40].

4. Conclusions and outlook

Numerical modeling of atomic switch networks is essentialfor understanding the experimentally observed emergentphenomena, and the microscopic degrees of freedom forsynaptic elements. A valid numerical simulation offers acontrollable, convenient platform from which to study specificaspects of the device functionality, and permits identificationof control parameters for network level behaviors as well assystem optimization. By extracting the relevant dynamicalcomponents of the network, our model can be expanded to

8

Nanotechnology 24 (2013) 000000 H O Sillin et al

Figure 7. (a) Schematic of the network simulation used in the waveform generation RC task, with specific electrodes chosen asinputs/outputs (16 output electrodes). RC was implemented using a 10× 10 network (N = 126, σα = 2.5%) with a 5 V, 10 Hz sinusoidalinput signal and tasked to produce 10 Hz triangle/square and 20 Hz sinusoidal waveforms. (b) Mean-squared error (MSE) for each task withrespect to driving amplitude showed minimal error in triangle/square waveform generation task at 10 V, corresponding to the onset of higherharmonic generation (see red curve of figure 6(b)). Performance in the 20 Hz sinusoidal waveform generation task decreased when (c) therelative amplitude of the average second harmonic intensities of the readouts became increasingly diminutive. These results correspond to astrong dependence on the second harmonic for 20 Hz sine generation and the need for HHG in triangle/square generation as expected byFourier analysis.

understand the functionalities of ASNs with respect to a largertheoretical framework.

The simulation, incorporating network stochasticity andfilament dissolution into its state equation, shows ASNs asdevices capable of forming feedforward subassemblies thatutilize network plasticity. Simulation results also reveal thenetworks’ nonlinear integration of local electrical responses.Specifically, individual atomic switch elements show changesin their power spectral density when embedded in a network,while HHG emerges in the ASN with external bias andconnectivity as control parameters.

Our simulation results support the feasibility of utilizingnonlinear dynamics without needing to control or ‘train’the connections in the reservoir, and have indicated howto best optimize physical device parameters to maximizeRC efficiency for a given task. Future efforts will focus onimplementing benchmark tasks in RC in both simulationand hardware to quantitatively assess the kernel quality andgeneralization rank [40] in relation to the changing parametersas compared to other software and hardware reservoirs.Note that the plasticity intrinsic to the ASN makes it adynamical reservoir, which shows improved function for someRC tasks [57]. The dynamics of these devices during thetraining period may be further characterized by calculating

the Lyapunov exponent and used to elicit the connections toincreased computational power at the edge of chaos [58].

The results presented here also demonstrate thevalue of using synaptic elements within a biologicallyinspired connective architecture. Substantial efforts have beenundertaken to characterize and comprehend the dynamicalhierarchy of a functioning complex system such as thebrain [29, 59–61]. The existence of a readily scalable,physical device exhibiting many of the same dynamicsas biological neural assemblies underscores its potentialas a tool to study complexity. The greatest potential ofASNs lies not only in the versatility of hardware designbut also in their scalability. A strategy that scales up thehierarchical dimension, combined with insights obtained byactual learning and task performance through RC couldgenerate valuable new computational devices.

Acknowledgments

The authors gratefully acknowledge Cristina Martin-Olmos,Walter Freeman, Robert Kozma and Narayan Srinivasa fortheir helpful assistance. Physical ASN chips were fabricatedin the Integrated Systems Nanofabrication Cleanroom(ISNC) at the California Nanosystems Institute (CNSI)and simulations utilized resources at the Nano & Pico

9

Nanotechnology 24 (2013) 000000 H O Sillin et al

Characterization Lab (NPC) of CNSI. This work waspartially supported by the Japanese Ministry of Education,Culture, Sports, Science, and Technology (MEXT) WorldQ.8

Premier International (WPI) Research Center for MaterialsNanoarchitectonics (MANA), HRL Laboratories, and theDefense Advanced Research Projects Agency (DARPA)—Physical Intelligence Program (BAA-09-63), US Departmentof Defense.

ReferencesQ.9

[1] 2011 International Technology Roadmap for Semiconductors:Emerging Research Devices (San Jose: SemiconductorIndustry Association)

[2] Mead C 1990 Neuromorphic electronic systems Proc. IEEE78 1629–36

[3] Mahowald M and Douglas R 1991 A silicon neuron Nature354 515–8

[4] Indiveri G et al 2011 Neuromorphic silicon circuits Front.Neurosci. 5 1–23

[5] Jeong D S, Kim I, Ziegler M and Kohlstedt H 2013 Towardsartificial neurons and synapses: materials point of view RSCAdv. 3 3169–83

[6] Likharev K K and Strukov D B 2005 CMOL: devices, circuits,and architectures Introducing Molecular Electronicsvol 680 pp 447–77

[7] Borghetti J, Li Z, Straznicky J, Li X, Ohlberg D A A, Wu W,Stewart D R and Williams R S 2008 A hybridnanomemristor/transistor logic circuit capable of selfprogramming Proc. Natl Acad. Sci. 106 1699–703

[8] Xia Q et al 2009 Memristor—CMOS hybrid integratedcircuits for reconfigurable logic Nano Lett. 9 3640–5

[9] Zhao W S, Agnus G, Derycke V, Filoramo A,Bourgoin J-P and Gamrat C 2010 Nanotube devices basedcrossbar architecture: toward neuromorphic computingNanotechnology 21 175202

[10] Chang T, Jo S, Kim K, Sheridan P, Gaba S and Lu W 2011Synaptic behaviors and modeling of a metal oxidememristive device Appl. Phys. A 102 857–63

[11] Kugeler C, Rosezin R, Linn E, Bruchhaus R and Waser R2011 Materials, technologies, and circuit concepts fornanocrossbar-based bipolar RRAM Appl. Phys. A102 791–809

[12] Seo K et al 2011 Analog memory and spike-timing-dependentplasticity characteristics of a nanoscale titanium oxidebilayer resistive switching device Nanotechnology 22 1–5

[13] Cruz-Albrecht J M, Yung M W and Srinivasa N 2012Energy-efficient neuron, synapse and STDP integratedcircuits IEEE Trans. Biomed. Circuits Syst. 6 246–56

[14] Kim K-H, Gaba S, Wheeler D, Cruz-Albrecht J M, Hussain T,Srinivasa N and Lu W 2012 A functional hybrid memristorcrossbar-array/CMOS system for data storage andneuromorphic applications Nano Lett. 12 389–95

[15] Kuzum D, Jeyasingh R, Yu S and Wong H 2012 Low-energyrobust neuromorphic computation using synaptic devicesIEEE Trans. Electron Devices 59 3489–94Q.10

[16] Simons H 1962 The architecture of complexity Proc. Am.Phil. Soc. 106 467–82

[17] Strogatz S 2001 Exploring complex networks Nature410 268–76

[18] Beggs J M and Plenz D 2003 Neuronal avalanches inneocortical circuits J. Neurosci. 23 11167–77

[19] Levina A, Herrmann J M and Geisel T 2007 Dynamicalsynapses causing self-organized criticality in neuralnetworks Nature Phys. 3 857–60

[20] Murphy B K and Miller K D 2009 Balanced amplification: anew mechanism of selective amplification of neural activitypatterns Neuron 61 635–48

[21] Chialvo D R 2010 Emergent complex neural dynamics NaturePhys. 6 744–50

[22] Shew W L, Yang H, Yu S, Roy R and Plenz D 2011Information capacity and transmission are maximized inbalanced cortical networks with neuronal avalanchesJ. Neurosci. 31 55–63

[23] Jensen H J 1998 Self-Organized Criticality: EmergentComplex Behavior in Physical and Biological Systemsvol 10 (Cambridge: Cambridge university press)

[24] Sporns O, Chialvo D R, Kaiser M and Hilgetag C C 2004Organization, development and function of complex brainnetworks Trends Cogn. Sci. 8 418–25

[25] Buonomano D V and Maass W 2009 State-dependentcomputations: spatiotemporal processing in corticalnetworks Nature Rev. Neurosci. 10 113–25

[26] Hebb D O 1949 The Organization of Behavior; ANeuropsychological Theory (New York: Wiley)

[27] Freeman W J, Burke B C and Holmes M D 2003 Aperiodicphase re-setting in scalp EEG of beta–gamma oscillationsby state transitions at alpha–theta rates Hum. Brain Mapp.19 248–72

[28] Chklovskii D B, Mel B W and Svoboda K 2004 Corticalrewiring and information storage Nature 431 782–8

[29] Kozma R, Puljic M, Balister P, Bollobas B and Freeman W J2005 Phase transitions in the neuropercolation model ofneural populations with mixed local and non-localinteractions Biol. Cybern. 92 367–79

[30] Avizienis A V, Sillin H O, Martin-Olmos C, Shieh H H,Aono M, Stieg A Z and Gimzewski J K 2012Neuromorphic atomic switch networks PLoS One 7 e42772

[31] Stieg A Z, Avizienis A V, Sillin H O, Martin-Olmos C,Aono M and Gimzewski J K 2012 Emergent criticality incomplex turing B-type atomic switch networks Adv. Mater.24 286–93

[32] Hasegawa T, Ohno T, Terabe K, Tsuruoka T, Nakayama T,Gimzewski J K and Aono M 2010 Learning abilitiesachieved by a single solid-state atomic switch Adv. Mater.22 1831–4

[33] Ohno T, Hasegawa T, Tsuruoka T, Terabe K,Gimzewski J K and Aono M 2011 Short-term plasticity andlong-term potentiation mimicked in single inorganicsynapses Nature Mater. 10 591–5

[34] Yang R, Terabe K, Liu G, Tsuruoka T, Hasegawa T,Gimzewski J K and Aono M 2012 On-demand nanodevicewith electrical and neuromorphic multifunction realized bylocal ion migration ACS Nano 6 9515–21

[35] Terabe K, Hasegawa T, Nakayama T and Aono M 2005Quantized conductance atomic switch Nature 433 47–50

[36] Hopfield J J and Tank D W 1985 ‘Neural’ computation ofdecision in optimization problems Biol. Cybern. 52 141–52

[37] Jaeger H 2002 Tutorial on training recurrent neural networks,covering BPPT, RTRL, EKF and the’ echo state network’approach GMD-Forschungszentrum Informationstechnik

[38] Jaeger H 2001 The ‘echo state’ approach to analysing andtraining recurrent neural networks German NationalResearch Center for Information Technology vol 148

[39] Maass W, Natschlager T and Markram H 2002 Real-timecomputing without stable states: a new framework forneural computation based on perturbations Neural Comput.14 2531–60

[40] Schrauwen B, Verstraeten D and Van Campenhout J 2007 Anoverview of reservoir computing: theory, applications andimplementations Q.11

[41] Verstraeten D, Schrauwen B, D’Haene M and Stroobandt D2007 2007 Special issue: an experimental unification ofreservoir computing methods Neural Netw. 20 391–403

10

Nanotechnology 24 (2013) 000000 H O Sillin et al

[42] Lukosevicius M and Jaeger H 2009 Reservoir computingapproaches to recurrent neural network training Comput.Sci. Rev. 3 127–49

[43] Busing L, Schrauwen B and Legenstein R 2010 Connectivity,dynamics, and memory in reservoir computing with binaryand analog neurons Neural Comput. 22 1272–311

[44] Fernando C and Sojakka S 2003 Pattern recognition in abucket Advances in Artificial LifeQ.12

[45] Schurmann F, Meier K and Schemmel J 2004 Edge of chaoscomputation in mixed-mode VLSI-a hard liquid Advancesin Neural Information Processing Systems vol 17,pp 1201–8

[46] Paquot Y, Duport F, Smerieri A, Dambre J, Schrauwen B,Haelterman M and Massar S 2012 Optoelectronic reservoircomputing Sci. Rep.Q.13

[47] Cohen G Z, Pershin Y V and Di Ventra M 2012 Second andhigher harmonics generation with memristive systems Appl.Phys. Lett. 100 133109

[48] Oskoee E N and Sahimi M 2011 Electric currents in networksof interconnected memristors Phy. Rev. E 83 031105

[49] Kuhn A, Argoul F, Muzy J F and Arneodo A 1994 Structuralanalysis of electroless deposits in the diffusion-limitedregime Phys. Rev. Lett. 73 2998–3001

[50] Shochet O and Ben-jacob E 1993 Coexistence ofmorphologies in diffusive patterning Phys. Rev. E48 R4168–71

[51] Avizienis A V, Martin-Olmos C, Sillin H O, Aono M,Gimzewski J K and Stieg A Z 2013 Morphological

transitions from dendrites to nanowires in the electrolessdeposition of silver Cryst. Growth Des. 13 465–9

[52] Kundu M, Terabe K, Hasegawa T and Aono M 2006 Effect ofsulfurization conditions and post-deposition annealingtreatment on structural and electrical properties of silversulfide films J. Appl. Phys. 99 103501

[53] Xu Z, Bando Y, Wang W, Bai X and Golberg D 2010Real-time in situ HRTEM-resolved resistance switching ofAg2S nanoscale ionic conductor ACS Nano 4 2515–22

[54] Chua L 1971 Memristor—the missing circuit element IEEETrans. Circuits Syst. 18 507–19

[55] Strukov D B, Snider G S, Stewart D R and Williams R S 2008The missing memristor found Nature 453 80–3

[56] Keshner M S 1982 1/f noise Proc. IEEE 70 212–8[57] Lazar A, Pipa G and Triesch J 2009 SORN: a self-organizing

recurrent neural network Front. Comput. Neurosci. 3 Q.14

[58] Langton C G 1990 Computation at the edge of chaos: phasetransitions and emergent computation Physica D 42 12–37

[59] Friston K 2008 Hierarchical models in the brain PLoSComput. Biol. 4 e1000211

[60] Meunier D, Lambiotte R, Fornito A, Ersche K D andBullmore E T 2009 Hierarchical modularity in human brainfunctional networks Front. Neurosci. 3 1–12

[61] Gleiser P M and Spoormaker V I 2010 Modelling heirarchicalstructure in functional brain networks Phil. Trans. A Math.Phys. Eng. Sci. 368 5633–44

11

Queries for IOP paper 461325

Journal: NanoAuthor: H O Sillin et alShort title: A theoretical and experimental study of

neuromorphic atomic switch networks forreservoir computing

Page 1

Query 1:Author: Please check the author names and affiliationscarefully.

Page 1

Query 2:Author: Please be aware that the colour figures in this articlewill only appear in colour in the Web version. If you requirecolour in the printed journal and have not previously arrangedit, please contact the Production Editor now.

Page 1

Query 3:Author: In several places I have standardized the Englishsomewhat. Please check that I have not inadvertently alteredthe meaning.

Page 2

Query 4:Author : Please check ‘synapses cm−2’ given here.

Page 3

Query 5:Author: Please check the edits made in the sentence‘Simulation efforts ...’, and correct if necessary.

Page 6

Query 6:Author: The caption to figure 4 has been reworded for clarity.Please check that the meaning is still correct.

Page 8

Query 7:Author: This sentence has been reworded for clarity. Pleasecheck that the meaning is still correct.

Page 10

Query 8:Author: Please check whether any commas need to be insertedhere.

Page 10

Query 9:Author: Please check the details for any journal references thatdo not have a blue link as they may contain some incorrectinformation. Pale purple links are used for references to arXive-prints.

Page 10

Query 10:Author: [15, 40]: Please check second author name inserted.

Page 10

Query 11:Author: [40]: Any more details?

Page 11

Query 12:Author: [44, 45]: Please provide place and publisher.

Page 11

Query 13:Author: [46]: Please provide volume and page/article number.

Page 11

Query 14:Author: [57]: Please provide page/article number.