AIAny
Icon for item

PROLA: Principal-Orthogonal Low-rank Adaptation for Predictive Spatiotemporal Weather Downscaling

Predicts future high-resolution weather fields from coarse inputs. Its low-rank adaptation divides a fixed parameter budget between pretrained principal directions and their orthogonal complement.

Introduction

Weather downscaling usually improves spatial detail for a given time; forecasting adds the harder requirement of evolving that detail into the future. This work treats both as one problem and finds that a carefully structured low-rank update can outperform broader adaptation while training fewer than 4 million parameters in its main setting.

Key Findings
  • The method splits a fixed low-rank budget between directions already prominent in pretrained weights and an orthogonal branch for new corrections. This lets adaptation reuse learned structure without restricting all updates to it.
  • On global weather downscaling, it reports lower RMSE than the evaluated low-rank baselines across five variables at both 3-hour and 6-hour leads. It also surpasses full fine-tuning on selected variables and leads, despite using 3.92 million trainable parameters—under 0.4% of the full fine-tuning parameter count.
  • The reported gains extend to ERA5-to-CERRA regional downscaling and regional precipitation forecasting, suggesting the approach is not limited to one global benchmark.
  • Gradient-guided rescaling adjusts rank-1 optimizer updates without increasing the trainable rank, targeting more effective use of a constrained adaptation budget.
Method and Scope

The prediction pipeline separates spatial refinement from temporal prediction, using a shared pretrained weather backbone, fixed numerical scaffolds, and lightweight residual heads. PROLA changes how that backbone is adapted; it is not a replacement forecasting architecture. The experiments compare it with methods including LoRA, PiSSA, OPLoRA, and AdaLoRA under their reported settings.

Who It Suits—and the Trade-off

A useful read for researchers adapting pretrained geophysical models under tight training or parameter budgets, especially when both forecast horizon and spatial resolution matter. The evidence is empirical and tied to the paper’s evaluated datasets, variables, and protocols; it does not establish gains for every weather model or deployment setting. Readers seeking a general-purpose forecasting system rather than a parameter-efficient adaptation method should look elsewhere.

Information

  • Websitearxiv.org
  • OrganizationsKorea Advanced Institute of Science and Technology (KAIST)
  • AuthorsMinseo Yoon, Minseong Bae, Sojin Lee, Hyunwoo J. Kim
  • Published date2026/11/08

More Items

Examines whether Transformers are necessary when LLMs write model weights without gradient training. When allowed to design the architecture, the tested LLMs compile facts into sparse tables, reaching up to 54.4% accuracy on ARC.

Connects single-chain protein folding to a general-purpose language model, using its residue representations to drive a pretrained structure generator. Current results show weaker performance on longer chains and reduced scores on a general-knowledge test after protein training.

Reduces Transformer parameter count by repeatedly applying shared blocks, then combines their updates with geometric decay and learned attention. On ClimbMix, it beats standard baselines in validation loss with fewer Transformer parameters.