How to construct Kolmogorov-Arnold Network based on MATLAB's Deep Learning Toolbox

Kolmogorov-Arnold Networks (KANs) are theoretically grounded alternatives to conventional Multilayer Perceptrons (MLPs). KANs are built upon the Kolmogorov-Arnold representation theorem, a profound mathematical result stating that any multivariate continuous function can be expressed through a finite composition of univariate continuous functions and binary additions. This theoretical guarantee takes concrete form in the equation:

where and are univariate continuous functions. The structure corresponds to a two-layer network: the first layer applies the inner functions , while the second layer applies the outer functions , with layer widths n and , respectively. Each univariate function using B-spline curves with learnable coefficients.

This leads to a fundamentally different network architecture compared to MLPs. As illustrated in the following figure, traditional MLPs apply fixed nonlinear activations to weighted sums at each node, with learning concentrated in edge weights. In contrast, KANs invert this paradigm: nodes perform simple summations, and the learning occurs on the edges, where each connection is associated with a trainable B-spline activation function.

Can anyone has some advices about how to construct this KAN using Deep Learning Toolbox ?

Answer 1

[Chuguang Pan]

After referring to the open-source Github repository pykan, I have successfully implemented a custom KAN layer based on MATLAB's Deep Learning Toolbox and the Define Custom Deep Learning Layer with Learnable Parameters Example. The custom KAN layer accepts a "CB" dlarray and returns a "CB" dlarray.

>> checkLayer(kanLayer,layout)

Skipping code generation compatibility tests. To check validity of the layer for code generation, specify the CheckCodegenCompatibility and ObservationDimension options.

Running nnet.checklayer.TestLayerWithoutBackward

.......... .......... ........

Done nnet.checklayer.TestLayerWithoutBackward

__________

Test Summary:

28 Passed, 0 Failed, 0 Incomplete, 6 Skipped.

Time elapsed: 0.35918 seconds.

[Imola Fodor]

Hello, would you be kind enough to share the script? Thank you

[Chuguang Pan]

@Imola Fodor. I am preparing to submit the relevant scripts to File Exchange, the related files will be presented in the File Exchange.

[Imola Fodor]

Sounds good, now you know there is interest in the implementation :)

[Chuguang Pan]

@Imola Fodor. You can obtain the source code from file exchange now. https://ww2.mathworks.cn/matlabcentral/fileexchange/183210-kan-layer-based-on-matlab-deep-learning-toolbox

[Imola Fodor]

Awesome, thanks!