A SIMPLE, FAST ALGORITHM FOR CONTINUAL LEARNING FROM HIGH-DIMENSIONAL DATA

Neil Ashtekar; Vasant Honavar

A SIMPLE, FAST ALGORITHM FOR CONTINUAL LEARNING FROM HIGH-DIMENSIONAL DATA

Neil Ashtekar
, Vasant Honavar

Research output: Contribution to conference › Paper › peer-review

Abstract

As an alternative to resource-intensive deep learning approaches to the continual learning problem, we propose a simple, fast algorithm inspired by adaptive resonance theory (ART). To cope with the curse of dimensionality and avoid catastrophic forgetting, we apply incremental principal component analysis (IPCA) to the model’s previously learned weights. Experiments show that this approach approximates the performance achieved using static PCA and is competitive with continual deep learning methods. Our implementation is available on https://github.com/neil-ash/ART-IPCA.

Original language	English (US)
State	Published - 2023
Event	1st Tiny Papers at 11th International Conference on Learning Representations, Tiny Papers @ ICLR 2023 - Kigali, Rwanda Duration: May 5 2023 → May 5 2023

Conference

Conference	1st Tiny Papers at 11th International Conference on Learning Representations, Tiny Papers @ ICLR 2023
Country/Territory	Rwanda
City	Kigali
Period	5/5/23 → 5/5/23

All Science Journal Classification (ASJC) codes

Linguistics and Language
Language and Linguistics
Computer Science Applications
Education

Cite this

@conference{bc3c0b3e4fbb4c388ef83a184eeb7cc4,

title = "A SIMPLE, FAST ALGORITHM FOR CONTINUAL LEARNING FROM HIGH-DIMENSIONAL DATA",

abstract = "As an alternative to resource-intensive deep learning approaches to the continual learning problem, we propose a simple, fast algorithm inspired by adaptive resonance theory (ART). To cope with the curse of dimensionality and avoid catastrophic forgetting, we apply incremental principal component analysis (IPCA) to the model{\textquoteright}s previously learned weights. Experiments show that this approach approximates the performance achieved using static PCA and is competitive with continual deep learning methods. Our implementation is available on https://github.com/neil-ash/ART-IPCA.",

author = "Neil Ashtekar and Vasant Honavar",

note = "Publisher Copyright: {\textcopyright} 2023 1st Tiny Papers Track at ICLR 2023 - Tiny Papers @ ICLR 2023. All rights reserved.; 1st Tiny Papers at 11th International Conference on Learning Representations, Tiny Papers @ ICLR 2023 ; Conference date: 05-05-2023 Through 05-05-2023",

year = "2023",

language = "English (US)",

}

TY - CONF

T1 - A SIMPLE, FAST ALGORITHM FOR CONTINUAL LEARNING FROM HIGH-DIMENSIONAL DATA

AU - Ashtekar, Neil

AU - Honavar, Vasant

PY - 2023

Y1 - 2023

N2 - As an alternative to resource-intensive deep learning approaches to the continual learning problem, we propose a simple, fast algorithm inspired by adaptive resonance theory (ART). To cope with the curse of dimensionality and avoid catastrophic forgetting, we apply incremental principal component analysis (IPCA) to the model’s previously learned weights. Experiments show that this approach approximates the performance achieved using static PCA and is competitive with continual deep learning methods. Our implementation is available on https://github.com/neil-ash/ART-IPCA.

AB - As an alternative to resource-intensive deep learning approaches to the continual learning problem, we propose a simple, fast algorithm inspired by adaptive resonance theory (ART). To cope with the curse of dimensionality and avoid catastrophic forgetting, we apply incremental principal component analysis (IPCA) to the model’s previously learned weights. Experiments show that this approach approximates the performance achieved using static PCA and is competitive with continual deep learning methods. Our implementation is available on https://github.com/neil-ash/ART-IPCA.

UR - https://www.scopus.com/pages/publications/105026752507

UR - https://www.scopus.com/pages/publications/105026752507#tab=citedBy

M3 - Paper

AN - SCOPUS:105026752507

T2 - 1st Tiny Papers at 11th International Conference on Learning Representations, Tiny Papers @ ICLR 2023

Y2 - 5 May 2023 through 5 May 2023

ER -

A SIMPLE, FAST ALGORITHM FOR CONTINUAL LEARNING FROM HIGH-DIMENSIONAL DATA

Abstract

Conference

All Science Journal Classification (ASJC) codes

Other files and links

Fingerprint

Cite this