Draft:All-optical nonlinear activation function

An all-optical nonlinear activation function is a photonic device or physical process that implements a nonlinear mapping between an optical input and an optical output without using an electronic signal-processing stage to perform the activation operation. In an optical neural network, the activation function is normally applied after a weighted summation or another linear optical operation. It introduces the nonlinearity required for a multilayer network to represent transformations that cannot be reduced to a single linear operation.

Optical systems can perform linear operations such as matrix multiplication, interference, convolution, and weighted summation using passive or programmable optical components. Implementing compact, low-power and cascadable nonlinear operations directly in the optical domain is more difficult because the nonlinear response of many optical materials is weak at the power levels used in integrated photonic circuits. All-optical activation devices have therefore been investigated as a means of reducing repeated optical-to-electrical and electrical-to-optical conversions between network layers.

Operating principle

For an artificial neuron, the output is commonly represented as

$y_{j}=f\left(\sum _{i}w_{ji}x_{i}+b_{j}\right),$

where $x_{i}$ are the input values, $w_{ji}$ are the weights, $b_{j}$ is a bias term, and $f$ is a nonlinear activation function. In a photonic implementation, the inputs and outputs may be encoded in optical power, amplitude, phase, wavelength, pulse timing, or combinations of these quantities.

An all-optical activation device transforms the incoming optical signal through an intensity-dependent or field-dependent physical response. Mechanisms used for this purpose include two-photon absorption, saturable absorption, free-carrier dispersion, the Kerr effect, optical bistability, gain saturation, phase transitions, and nonlinear interactions in semiconductor lasers or atomic media.

Depending on the device and its operating point, the resulting transfer characteristic may approximate a rectified linear unit, sigmoid, threshold, radial-basis, saturating, or other nonlinear function. Exact replication of a software activation function is not always required. A measured physical transfer curve can instead be included in hardware-aware training or numerical evaluation of the neural network.

The term all-optical distinguishes these devices from optical–electrical–optical activation units in which a photodetector converts the signal into an electrical quantity, electronic circuitry processes it, and a modulator or laser generates a new optical output. Some all-optical devices may still use an electrical or thermal control signal for calibration or static tuning, while the activation of each data signal remains an optical process.

Implementations

Atomic and free-space systems

Nonlinear activation has been demonstrated using electromagnetically induced transparency in an atomic medium. In 2019, Zuo and colleagues constructed an all-optical neural-network experiment in which spatial light modulators and Fourier lenses performed linear operations, while laser-cooled rubidium atoms supplied the nonlinear optical response. Atomic systems can provide strong and controllable nonlinear responses, although their experimental apparatus is generally less compact than an integrated photonic device.

Waveguide, interferometer and resonator devices

Integrated implementations use optical waveguides, Mach–Zehnder interferometers, microring resonators and photonic cavities to increase the interaction between light and a nonlinear material. Resonant devices can enhance an otherwise weak material response by increasing the optical field within a small region, although their behavior can also be sensitive to wavelength, fabrication variation and temperature.

In 2020, Jha, Huang and Prucnal experimentally demonstrated reconfigurable nonlinear transfer functions using a cavity-loaded Mach–Zehnder interferometer fabricated on a silicon photonics platform. The device used free-carrier dispersion and could be configured to produce several activation-curve shapes.

Microring resonators have also been combined with nonlinear semiconductor structures. A proposed germanium–silicon hybrid microring used two-photon absorption and the resulting free-carrier effects to obtain an intensity-dependent transmission response at a lower threshold than an unmodified silicon structure.

Metamaterial-enhanced nonlinearity

Metamaterial structures can be integrated with conventional optical waveguides to strengthen light–matter interaction. In 2024, Honda, Shoji and Amemiya demonstrated a silicon waveguide incorporating arrays of titanium–gold split-ring resonators. The resonant metamaterial produced a slow-light response that increased the effective interaction of the optical field with the silicon waveguide.

At sufficiently high optical intensity, two-photon absorption causes the waveguide transmission to become nonlinear because two photons are absorbed in a single electronic transition. The resonant slow-light region strengthens this intensity-dependent interaction. The authors reported an effective two-photon-absorption coefficient of 424 cm/GW, approximately 1,200 times the value they used for an unmodified silicon waveguide. When the resulting activation characteristic was incorporated into a handwritten-digit recognition model, the study reported an inference accuracy of 98.36%.

Two-dimensional and phase-change materials

Two-dimensional materials can provide strong nonlinear absorption in a small interaction area. A 2024 device integrated molybdenum ditelluride with optical waveguides and used saturable and reverse saturable absorption to produce nonlinear transmission. The device was studied across visible and near-infrared wavelengths and was evaluated through neural-network simulations.

Phase-change materials have also been proposed for programmable activation devices. Their optical constants change between material states, allowing the transmission curve of a waveguide or resonator to be modified and retained without continuous tuning power. A 2022 proposal used a silicon microring loaded with phase-change material to obtain programmable nonlinear transfer characteristics.

Semiconductor gain and laser devices

Semiconductor optical amplifiers and lasers can produce nonlinear responses through gain saturation, optical injection, carrier dynamics or optical bistability. These devices may also provide gain, which can compensate for losses accumulated in earlier photonic layers.

A theoretical 2020 design used a semiconductor Fano laser as a nonlinear activation and spiking element. Optical control of the Fano mirror produced a threshold response and suppression between the device's on and off states. Laser-based activation devices can support regeneration and fan-out, but require an active gain medium and introduce additional fabrication and control requirements.

Performance characteristics

The usefulness of an all-optical activation function depends on both its device-level transfer curve and its behavior as part of a larger network. Commonly evaluated characteristics include:

• the shape and programmability of the nonlinear transfer function;
• optical-power or energy threshold;
• response time and operating bandwidth;
• insertion loss and extinction ratio;
• signal-to-noise ratio and dynamic range;
• compatibility with wavelength-division or phase-sensitive operation;
• cascadability and the ability to drive subsequent network layers;
• fan-out to multiple downstream neurons;
• device footprint and fabrication compatibility;
• thermal stability and fabrication tolerance; and
• endurance and retention for devices using nonvolatile materials.

These characteristics can involve competing design requirements. Resonators and slow-light structures can reduce the optical power needed to produce a nonlinear response, but may reduce usable optical bandwidth and increase sensitivity to resonance drift. Broadband absorptive materials may operate across a wider wavelength range, but absorption can also reduce the available optical power for subsequent layers. Active semiconductor devices can restore signal power but add energy consumption, noise and device complexity.

A nonlinear device must also preserve sufficient optical power and signal quality for the following layer. Losses accumulated through multiple passive activation units may require amplification, while noise introduced by active components can propagate through the network. Cascadability therefore cannot be established solely from the shape of an isolated device's input–output curve.

Research status

All-optical nonlinear activation functions are studied for image classification, signal processing, photonic reservoir computing, spiking neural networks and other neuromorphic-photonic systems. Experimental studies have demonstrated individual activation devices, small networks and physical transfer functions used in neural-network models.

As of the mid-2020s, many photonic neural networks continued to perform linear operations optically while implementing nonlinear activation and control electronically. Fully integrated, multilayer all-optical systems remained limited by optical loss, weak material nonlinearities, signal restoration, fabrication variation, thermal drift, fan-out and the integration of different material platforms. Research has consequently focused on stronger nonlinear materials, resonant enhancement, heterogeneous photonic integration, hardware-aware training and devices whose transfer functions can be reconfigured for different network architectures.

See also

• Activation function
• Optical neural network
• Neuromorphic computing
• Nonlinear optics
• Optical computing
• Photonic integrated circuit
• Silicon photonics

References

Category:Optical computing Category:Artificial neural networks Category:Nonlinear optics Category:Photonics Category:Neuromorphic engineering