Skip to main content
Compute lag features across all series, or within groups of series, using SQL-style RANGE semantics.
Most lag transforms in mlforecast are computed independently per series: a rolling mean for series A only ever sees A’s own history. That works well when every series carries enough signal on its own, but it breaks down in two common situations:
  • Short-history series. A newly launched product, store, or SKU has too little history for per-series rolling statistics to stabilise.
  • Cross-series signals. Total demand across a brand, region, or category at a given timestamp is often a stronger feature than any one series’ lag.
Pooled lag transforms address both. By passing global_=True (across all series) or groupby=["col", ...] (within a static-feature group) to any rolling, expanding, seasonal, or exponentially weighted transform, you ask mlforecast to compute the statistic over a bucket of series aggregated by timestamp. The result is a single value per timestamp per bucket that every series in the bucket then receives as a feature.

​
Data setup

import warnings

import numpy as np
import pandas as pd

from mlforecast import MLForecast
from mlforecast.lag_transforms import (
    ExpandingMean,
    ExponentiallyWeightedMean,
    RollingMean,
)
from mlforecast.utils import generate_daily_series
We’ll start from a small synthetic panel and attach a static brand column so we have something to group by. 
series = generate_daily_series(
    n_series=6,
    min_length=60,
    max_length=60,
    equal_ends=True,
    static_as_categorical=False,
    seed=0,
)
brand_map = {f"id_{i}": "A" if i < 3 else "B" for i in range(6)}
series["brand"] = series["unique_id"].map(brand_map)
series.head()

​
Global features with global_=True

Set global_=True on any built-in lag transform to compute the statistic across all series, aggregated by timestamp. Every series receives the same value at a given timestamp. 
fcst = MLForecast(
    models=[],
    freq="D",
    lag_transforms={
        1: [RollingMean(window_size=7, global_=True)],
    },
)
prep = fcst.preprocess(series, static_features=["brand"])
prep[prep["ds"] == prep["ds"].min()].head(6)
Notice that global_rolling_mean_lag1_window_size7 is identical for every unique_id at a given ds: it’s the rolling mean of the pooled observations across all series, lagged by one day. The feature name is automatically prefixed with global_ to make the pooling explicit.

​
Group features with groupby=[...]

Use groupby to compute the statistic within each level of one or more static features. Series in the same group share the feature value at each timestamp; series in different groups get different values. Any column used in groupby must be declared as a static feature when fitting. 
fcst = MLForecast(
    models=[],
    freq="D",
    lag_transforms={
        1: [RollingMean(window_size=7, groupby=["brand"])],
    },
)
prep = fcst.preprocess(series, static_features=["brand"])
prep[prep["ds"] == prep["ds"].min()].sort_values(["brand", "unique_id"]).head(6)
Series within brand A share one rolling-mean value; series within brand B share another. The column name is prefixed with groupby_brand_ to record which static feature drove the pooling.

​
RANGE semantics: staggered series and gaps

Pooled transforms use RANGE BETWEEN ... PRECEDING semantics, the same model SQL window functions use. The window is defined by timestamp distance, not row position, and only actual observations are aggregated — no synthetic zeros are injected for series that haven’t started yet. Concretely, with RollingMean(window_size=2, global_=True) over the data below:
unique_iddsy
a11.0
a22.0
a33.0
b220.0
b330.0
Series b starts at ds=2; it does not contribute a phantom zero at ds=1. At each timestamp the window looks back over the last two days of real values across all series, applied at lag 1. 
staggered = pd.DataFrame(
    {
        "unique_id": ["a", "a", "a", "b", "b"],
        "ds": pd.to_datetime(
            ["2024-01-01", "2024-01-02", "2024-01-03", "2024-01-02", "2024-01-03"]
        ),
        "y": [1.0, 2.0, 3.0, 20.0, 30.0],
    }
)
fcst = MLForecast(
    models=[],
    freq="D",
    lag_transforms={1: [RollingMean(window_size=2, global_=True, min_samples=1)]},
)
fcst.preprocess(staggered, dropna=False)
Because pooled transforms assume a continuous, gap-free time grid within each series, you should validate your data (the default) before relying on these features. We come back to this in the validate_data section below.

​
min_samples in pooled mode

There’s an important semantic divergence between local and pooled modes:
  • Local mode (per series): min_samples is capped at window_size by coreforecast. It controls how many non-NaN values in the window the series needs.
  • Pooled mode (global_=True or groupby=...): min_samples counts the total non-NaN observations across all series in the bucket, with no capping at window_size.
This makes it useful as a coverage threshold. For example, RollingMean(window_size=1, min_samples=2, groupby=['brand']) only produces a value when at least two series in the brand contributed an observation in the window — useful to suppress noise from sparsely populated groups. The default when min_samples=None also differs by mode. Everywhere else it defaults to window_size, but in local partition mode (partition_by without global_/groupby, covered below) it defaults to 1: the window spans window_size calendar steps while only same-partition observations count toward min_samples, so requiring a full window is rarely attainable — RollingMean(7, partition_by=['promo']) on a panel with interleaved promo days would otherwise be almost entirely NaN. The default of 1 matches SQL RANGE window semantics, where the result is NULL only for empty windows. When partition_by is combined with global_ or groupby, the default remains window_size, counted across all series in the (group, partition) bucket — pass min_samples explicitly if you want a different threshold. 
fcst = MLForecast(
    models=[],
    freq="D",
    lag_transforms={
        1: [RollingMean(window_size=1, min_samples=2, groupby=["brand"])],
    },
)
prep = fcst.preprocess(series, static_features=["brand"], dropna=False)
prep[
    [
        "unique_id",
        "ds",
        "brand",
        "groupby_brand_rolling_mean_lag1_window_size1_min_samples2",
    ]
].head(6)

​
Aggregating rows per timestamp with time_agg

By default a pooled transform treats every observation as an individual sample: RollingMean(window_size=7, groupby=['brand']) pools all rows of the brand that fall in the window and averages them, weighting a timestamp by how many series reported on it. Sometimes you instead want to first collapse all rows sharing a timestamp into a single value — for example the brand’s daily total — and then apply the transform over that per-timestamp series. time_agg does exactly that. Set it to one of 'sum', 'count', 'mean', 'min', or 'max' and each (bucket, timestamp) is reduced to one value before the rolling/expanding/seasonal/EWM statistic runs:
  • time_agg='sum' — rolling mean of the brand’s daily sums (total demand)
  • time_agg='count' — e.g. expanding max of the number of active series per day
  • time_agg='max' — rolling max of the daily peak across the brand
time_agg requires global_=True or groupby=[...] (optionally combined with partition_by). It is rejected in local or partition_by-only mode, where each (bucket, timestamp) already has a single row and the aggregation would be a no-op. With time_agg=None, each observation remains an individual pooled sample. ExponentiallyWeightedMean is the exception: its time_agg defaults to 'mean', and None is not accepted. 
fcst = MLForecast(
    models=[],
    freq="D",
    lag_transforms={
        1: [RollingMean(window_size=3, groupby=["brand"], time_agg="sum")],
    },
)
prep = fcst.preprocess(series, static_features=["brand"], dropna=False)
prep[
    [
        "unique_id",
        "ds",
        "brand",
        "groupby_brand_rolling_mean_lag1_window_size3_time_aggsum",
    ]
].head(6)
Two semantics worth calling out:
  • With time_agg, min_samples counts observed timestamps in the window (each timestamp contributes at most one aggregated value), not rows. RollingMean(window_size=3, min_samples=3, groupby=['brand'], time_agg='sum') needs three distinct timestamps of history, regardless of how many series reported on each.
  • RollingMean(..., time_agg='mean') differs from RollingMean(...) without time_agg: the former is an unweighted mean of the per-timestamp means, the latter a row-weighted pooled mean (they coincide only when every timestamp has the same number of observations). On ExponentiallyWeightedMean, time_agg='mean' is the default because pooled EWM consumes each timestamp’s bucket-aggregate mean exactly once.

​
Disabling validation: the validate_data=False warning

Pooled transforms are correctness-sensitive to timestamp gaps because they rely on the actual timestamps when computing RANGE windows. If you bypass validation (validate_data=False) and your data has gaps, the features will be silently wrong. To make this hazard explicit, mlforecast emits a UserWarning when you combine validate_data=False with any pooled transform: 
fcst = MLForecast(
    models=[],
    freq="D",
    lag_transforms={1: [RollingMean(window_size=7, global_=True)]},
)
with warnings.catch_warnings(record=True) as caught:
    warnings.simplefilter("always")
    fcst.preprocess(series, static_features=["brand"], validate_data=False)
[str(w.message) for w in caught if issubclass(w.category, UserWarning)]
If you genuinely need to skip validation (e.g. on a very large pre-cleaned dataset), make sure your time grid is continuous and gap-free before doing so.

​
Pooling with partition_by

partition_by=[col, ...] splits a pooled bucket further along a dynamic column — typically a feature whose value changes over time (e.g. promo, regime, store_format flips). Each unique combination of partition values gets its own bucket; rolling/expanding statistics are computed per bucket. Unlike groupby, partition_by columns do not need to be static. They can vary across timestamps for the same unique_id. At prediction time you must supply their future values via X_df. Three combinations are supported:
CombinationBucket key
partition_by=[...] alone(unique_id, *partition_values) — local partition, one bucket per series-partition pair
global_=True, partition_by=[...](*partition_values) — nonlocal, aggregates across all series sharing the same partition values
groupby=[...], partition_by=[...](*group_values, *partition_values) — nonlocal, aggregates within each group, separately per partition
Unlike global_ and groupby (which are mutually exclusive), partition_by composes with either one — or stands alone. Let’s add a time-varying promo column to our panel and walk through the three combinations. 
rng = np.random.default_rng(0)
series["promo"] = rng.choice([0, 1], size=len(series), p=[0.7, 0.3])
series.head()

​
Local partition: per-(id, partition_vals) buckets

When partition_by is used alone (no global_, no groupby), each (unique_id, partition_value) combination becomes its own bucket. The rolling window is computed within that bucket only — so a series sees a different rolling history depending on which promo regime it is currently in. 
fcst = MLForecast(
    models=[],
    freq="D",
    lag_transforms={1: [RollingMean(window_size=7, partition_by=["promo"])]},
)
fcst.preprocess(series).head(8)
The feature name is prefixed partby_promo_ to indicate the partition. Each row’s feature is the 7-day rolling mean of y lagged by 1, restricted to rows of the same series with the same promo value.

​
Global + partition_by: cross-series aggregates within each partition

Combine with global_=True to aggregate across all series, but separately for each promo value: 
fcst = MLForecast(
    models=[],
    freq="D",
    lag_transforms={
        1: [RollingMean(window_size=7, global_=True, partition_by=["promo"])]
    },
)
fcst.preprocess(series).head(8)
Every series with promo=0 at a given timestamp sees the same global_partby_promo_... value; every series with promo=1 sees a different (also shared) value.

​
Groupby + partition_by: group aggregates within each partition

Combine with groupby=[...] to aggregate per group (e.g. per brand), separately for each partition value: 
fcst = MLForecast(
    models=[],
    freq="D",
    lag_transforms={
        1: [RollingMean(window_size=7, groupby=["brand"], partition_by=["promo"])]
    },
)
fcst.preprocess(series, static_features=["brand"]).head(8)
The bucket key here is (brand, promo). There are at most n_brands × n_promo_values buckets.

​
Dynamic partition values at prediction time

Because partition columns are not static, you must supply their future values when forecasting. Pass an X_df containing unique_id, ds, and every partition column for every horizon step: 
from sklearn.linear_model import LinearRegression

fcst = MLForecast(
    models=[LinearRegression()],
    freq="D",
    lag_transforms={
        1: [RollingMean(window_size=7, global_=True, partition_by=["promo"])]
    },
)
# `brand` plays no role in this global_ + partition_by example, so we drop it
# rather than feed a raw string column to LinearRegression
fcst.fit(series.drop(columns="brand"))

last_date = series["ds"].max()
unique_ids = sorted(series["unique_id"].unique())
future_promo = pd.DataFrame(
    [
        {
            "unique_id": uid,
            "ds": last_date + pd.Timedelta(days=h),
            "promo": int(rng.integers(0, 2)),
        }
        for uid in unique_ids
        for h in range(1, 4)
    ]
)
fcst.predict(h=3, X_df=future_promo).head()
A few notes on dynamic prediction:
  • If X_df introduces a partition value never seen during fit, mlforecast creates a fresh bucket on the fly. The first few predictions for that bucket may be NaN if its history is shorter than window_size/min_samples.
  • Subset prediction (predict(ids=[...])) is rejected when nonlocal partition_by is in play: the missing series would still contribute to the bucket aggregates, so partial subsets are ambiguous.
  • Partition columns are automatically excluded from static_features even when static_features=None. They must come from X_df at predict time.

​
Understanding ordinals and parent calendars

A partition_by bucket only contains the rows where the partition value is active for a series. When the partition is sparse — e.g. promo=1 is only on at timestamps [1, 3, 5] while the parent calendar (the global or group scope) is [1, 2, 3, 4, 5] — the bucket’s ordinals come from the parent calendar:
  • Bucket has observations at parent positions [0, 2, 4], not [0, 1, 2].
  • RollingMean(window_size=2) at parent position 4 looks back to positions [3, 4]. The bucket has no observation at position 3, so only position 4 contributes to the window.
This preserves SQL RANGE BETWEEN ... semantics across gaps: the window is defined in time, not in number of rows. Without parent-calendar ordinals, a partition with gaps would silently collapse to row-based semantics, mixing observations across non-adjacent timestamps.

​
Aligned-end requirement

All pooled modes — global_, groupby, and partition_by (including local, alone) — require every series to end at the same timestamp. This is checked at fit and predict time and raises if violated: 
ValueError: Pooled lag transforms require all series to end at the same timestamp (recursive prediction advances all series in lockstep).
Local partition_by shares this requirement because parent calendars (per-id, for local mode) must advance in lockstep during recursive prediction. If your series have ragged ends, pre-align them (pad to a common last timestamp, or drop series that end early) before fitting.

​
Exponentially Weighted Mean with partitions

ExponentiallyWeightedMean with partition_by has subtly different semantics from a regular EWM:
  • The decay is applied across the bucket’s observed timestamp aggregates (one contribution per timestamp, regardless of how many rows the aggregate covered).
  • Timestamps where the partition bucket has no observation are skipped — they do not contribute to the running EWM and do not advance the decay step.
This means EWM treats each observed timestamp uniformly and ignores gaps. mlforecast emits a UserWarning when you construct an ExponentiallyWeightedMean with partition_by to make this explicit. 
with warnings.catch_warnings(record=True) as caught:
    warnings.simplefilter("always")
    tfm = ExponentiallyWeightedMean(alpha=0.3, partition_by=["promo"])
[str(w.message) for w in caught if issubclass(w.category, UserWarning)]

​
keep_last_n and pooled history

keep_last_n controls how much history is carried into the recursive prediction loop. It applies to pooled transforms too, per state (one state per (mode, groupby, partition_by) combination):
  • A state whose transforms are all finite-window — Lag, any Rolling* or SeasonalRolling*, and Offset/Combine built from them — is trimmed to its last max(keep_last_n, W_state) parent-calendar ordinals, where W_state is the state’s widest window. The dropped prefix can never enter a window, so predictions are unchanged.
  • A state containing any unbounded transform — Expanding* or ExponentiallyWeightedMean — keeps its full history. Unlike the local (coreforecast) path, pooled expanding/EWM carry no running accumulator: they recompute over the entire aggregate vectors at every step, so trimming them would change predictions.
This applies whether keep_last_n is set explicitly or inferred — when you leave it unset it is inferred as the largest window across all transforms — mirroring how the per-series arrays used by local transforms are trimmed.

​
Divergence from local transforms

Pooled trimming is floored at the state’s widest window (W_state). A local rolling transform survives an explicit keep_last_n smaller than its window because coreforecast keeps a separate per-transform window buffer. Pooled mode has no such buffer — the per-timestamp aggregates are the buffer — so a keep_last_n below a pooled window is raised to that window for the affected state, keeping predictions correct. When keep_last_n is left to be inferred it already equals the largest window, so the floor never changes anything.

​
Occurrence lookups with LookupLag

LookupLag returns the target from the previous matching occurrence within each (unique_id, partition_by...) bucket, regardless of the calendar gap between occurrences. This is useful for moving events — e.g. “last year’s Easter value” — where the relevant previous target is not a fixed calendar lag. partition_by is required and defines the buckets. Because it is a pooled transform, the partition columns may change over time and must be supplied via X_df at prediction. Since the lookup is computed inside mlforecast, it stays aligned with any target transforms (e.g. LocalStandardScaler). 
from mlforecast.lag_transforms import LookupLag

fcst = MLForecast(
    models=[],
    freq='D',
    lag_transforms={1: [LookupLag(partition_by=['promo'])]},
)
prep = fcst.preprocess(series, static_features=['brand'])
prep.head()

​
Constraints to keep in mind

  • global_=True and groupby=[...] are mutually exclusive on the same transform. Use two separate transforms if you want both feature families. partition_by composes with either one, or stands alone.
  • Columns in groupby must be declared in static_features when you call fit / preprocess. Columns in partition_by must not be — they are dynamic and supplied via X_df at prediction.
  • min_samples=0 in pooled mode produces NaN for timestamps with no observations in the window and triggers a warning at construction time — prefer min_samples=1 if you want “as soon as there is any observation”.
  • All pooled modes — global_, groupby, and partition_by (including local, alone) — require series to end at the same timestamp.
  • Subset prediction (predict(ids=[...])) is rejected when nonlocal pooled transforms are in play: omitted series would still contribute to the bucket aggregates.
  • All supported rolling, expanding, seasonal-rolling, and exponentially weighted transforms accept global_, groupby, and partition_by. Offset and Combine delegate to their wrapped transforms.
  • Performance: RollingQuantile, ExpandingQuantile and the SeasonalRolling* transforms have no aggregate-cache fast path in pooled modes. They fall back to a row-level pass whose cost grows with unique timestamps × bucket rows at fit, and aggregates are rebuilt at every recursive prediction step. For large panels prefer the mean/std/min/max/EWM transforms, which use cached per-timestamp aggregates.
  • Memory: finite-window pooled states (Lag/Rolling*/SeasonalRolling*) are trimmed under keep_last_n just like the per-series arrays (see the keep_last_n and pooled history section above); a state containing an Expanding*/ExponentiallyWeightedMean transform keeps the full training history because it recomputes over all of it at predict. Each predict call also backs up the pooled state per model to isolate recursive mutations. Expect higher peak memory with unbounded pooled transforms on large panels.
  • time_agg ('sum'/'count'/'mean'/'min'/'max') pre-aggregates rows sharing a timestamp within each bucket before the transform runs; it requires global_ or groupby and counts observed timestamps for min_samples. All pooled-capable transforms support it, including the quantile and seasonal ones (via the row-level path).

​
End-to-end example

Pooled transforms behave like any other lag transform in the full MLForecast lifecycle: fit, predict, and cross_validation all carry the pooled features through automatically. 
from sklearn.linear_model import LinearRegression

fcst = MLForecast(
    models=[LinearRegression()],
    freq="D",
    lags=[1, 7],
    lag_transforms={
        1: [
            RollingMean(window_size=7, global_=True),
            RollingMean(window_size=7, groupby=["brand"]),
            ExpandingMean(groupby=["brand"]),
            ExponentiallyWeightedMean(alpha=0.3, global_=True),
        ],
    },
)
# `promo` was only needed for the partition_by examples above; this end-to-end
# model groups by `brand` and doesn't use it, so drop it to keep predict X_df-free
fcst.fit(series.drop(columns="promo"), static_features=["brand"])
fcst.predict(h=7).head(6)

​
Where to next

  • The general Lag transformations guide covers the built-in transforms, Combine, Offset, and custom numba-based transforms.
  • The mlforecast.lag_transforms API reference documents every parameter, including global_, groupby, and min_samples, for each transform class.