Data Analytics report

# Why GP's TimesFM backtest went flat

Technical diagnosis of the nearly flat GP daily forecast, its accuracy metrics, GPU status, and model-improvement options.

Aug 12, 2026, 6:30 AM UTC

## Technical summary

The flat orange line is genuine model output, and this GP configuration should not be treated as a useful trading forecast. Across 598 Source for 598 Source: Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 File: prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json hidden daily closes, TimesFM's MAE was BDT 32.27 Source for 32.27 Source: Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 File: prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json versus BDT 31.81 Source for 31.81 Source: Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 File: prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json for simply repeating the last known close, so skill was -1.46%Source for -1.46%Source: Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 File: prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json. Direction matched only 46.82%Source for 46.82%Source: Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 File: prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json of bars, R-squared was -0.028 Source for -0.028 Source: Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 File: prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json, and the nominal 80%Source for 80%Source: Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 File: prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json interval covered only 4.52%Source for 4.52%Source: Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 File: prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json of actual closes.

The GPU did not cause the poor shape. This saved run used the CPU because the Tesla P100 was incompatible with the installed PyTorch kernels. GPU execution would primarily change speed; it would not add information, retrain the weights, or turn this long-horizon path into a more accurate one.

Source for Technical summary Source: Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 File: prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json

Displayed accuracy

88.63 Source for 88.63 Source: GP run summary calculation Tables: read_json_auto, run, dispersion

The displayed score is 100 minus sMAPE; it is not the probability that the forecast direction or price is correct.

sMAPE 11.37 Source for 11.37 Source: GP run summary calculation Tables: read_json_auto, run, dispersion
Source for Displayed accuracy The displayed score is 100 minus sMAPE; it is not the probability that the forecast direction or price is correct.Source: GP run summary calculation Tables: read_json_auto, run, dispersion

Loads the saved run's headline metrics and calculates actual-versus-predicted dispersion from its backtest rows.

    WITH run AS (SELECT * FROM read_json_auto('prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json')), dispersion AS (SELECT var_samp(b.predicted::DOUBLE) / var_samp(b.actual::DOUBLE) AS variance_ratio, stddev_samp(b.predicted::DOUBLE) AS predicted_std_bdt, stddev_samp(b.actual::DOUBLE) AS actual_std_bdt FROM run, UNNEST(backtest) AS t(b)) SELECT r.metrics.accuracy_score AS accuracy_score_pct, r.metrics.smape_percent AS smape_pct, r.metrics.skill_vs_naive_percent AS skill_vs_naive_pct, r.metrics.mae AS timesfm_mae_bdt, r.metrics.naive_mae AS naive_mae_bdt, r.metrics.directional_accuracy_percent AS direction_accuracy_pct, r.metrics.interval_80_coverage_percent AS coverage_80_pct, 80.0 AS coverage_nominal_pct, d.variance_ratio, d.predicted_std_bdt, d.actual_std_bdt, r.metrics.r_squared FROM run r CROSS JOIN dispersion d

Skill vs last price

-1.46 Source for -1.46 Source: GP run summary calculation Tables: read_json_auto, run, dispersion

Negative skill means TimesFM's MAE is worse than repeating the last observed close for every holdout bar.

TimesFM MAE 32.27 Source for 32.27 Source: GP run summary calculation Tables: read_json_auto, run, dispersion Baseline MAE 31.81 Source for 31.81 Source: GP run summary calculation Tables: read_json_auto, run, dispersion
Source for Skill vs last price Negative skill means TimesFM's MAE is worse than repeating the last observed close for every holdout bar.Source: GP run summary calculation Tables: read_json_auto, run, dispersion

Loads the saved run's headline metrics and calculates actual-versus-predicted dispersion from its backtest rows.

    WITH run AS (SELECT * FROM read_json_auto('prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json')), dispersion AS (SELECT var_samp(b.predicted::DOUBLE) / var_samp(b.actual::DOUBLE) AS variance_ratio, stddev_samp(b.predicted::DOUBLE) AS predicted_std_bdt, stddev_samp(b.actual::DOUBLE) AS actual_std_bdt FROM run, UNNEST(backtest) AS t(b)) SELECT r.metrics.accuracy_score AS accuracy_score_pct, r.metrics.smape_percent AS smape_pct, r.metrics.skill_vs_naive_percent AS skill_vs_naive_pct, r.metrics.mae AS timesfm_mae_bdt, r.metrics.naive_mae AS naive_mae_bdt, r.metrics.directional_accuracy_percent AS direction_accuracy_pct, r.metrics.interval_80_coverage_percent AS coverage_80_pct, 80.0 AS coverage_nominal_pct, d.variance_ratio, d.predicted_std_bdt, d.actual_std_bdt, r.metrics.r_squared FROM run r CROSS JOIN dispersion d

Direction matched

46.82 Source for 46.82 Source: GP run summary calculation Tables: read_json_auto, run, dispersion

Share of holdout bars for which the predicted up, down, or unchanged sign matched the actual sign.

Source for Direction matched Share of holdout bars for which the predicted up, down, or unchanged sign matched the actual sign.Source: GP run summary calculation Tables: read_json_auto, run, dispersion

Loads the saved run's headline metrics and calculates actual-versus-predicted dispersion from its backtest rows.

    WITH run AS (SELECT * FROM read_json_auto('prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json')), dispersion AS (SELECT var_samp(b.predicted::DOUBLE) / var_samp(b.actual::DOUBLE) AS variance_ratio, stddev_samp(b.predicted::DOUBLE) AS predicted_std_bdt, stddev_samp(b.actual::DOUBLE) AS actual_std_bdt FROM run, UNNEST(backtest) AS t(b)) SELECT r.metrics.accuracy_score AS accuracy_score_pct, r.metrics.smape_percent AS smape_pct, r.metrics.skill_vs_naive_percent AS skill_vs_naive_pct, r.metrics.mae AS timesfm_mae_bdt, r.metrics.naive_mae AS naive_mae_bdt, r.metrics.directional_accuracy_percent AS direction_accuracy_pct, r.metrics.interval_80_coverage_percent AS coverage_80_pct, 80.0 AS coverage_nominal_pct, d.variance_ratio, d.predicted_std_bdt, d.actual_std_bdt, r.metrics.r_squared FROM run r CROSS JOIN dispersion d

80% band coverage

4.52 Source for 4.52 Source: GP run summary calculation Tables: read_json_auto, run, dispersion

Only a small fraction of actual closes fell inside the model's nominal Q10-Q90 interval.

Nominal target 80 Source for 80 Source: GP run summary calculation Tables: read_json_auto, run, dispersion
Source for 80% band coverage Only a small fraction of actual closes fell inside the model's nominal Q10-Q90 interval.Source: GP run summary calculation Tables: read_json_auto, run, dispersion

Loads the saved run's headline metrics and calculates actual-versus-predicted dispersion from its backtest rows.

    WITH run AS (SELECT * FROM read_json_auto('prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json')), dispersion AS (SELECT var_samp(b.predicted::DOUBLE) / var_samp(b.actual::DOUBLE) AS variance_ratio, stddev_samp(b.predicted::DOUBLE) AS predicted_std_bdt, stddev_samp(b.actual::DOUBLE) AS actual_std_bdt FROM run, UNNEST(backtest) AS t(b)) SELECT r.metrics.accuracy_score AS accuracy_score_pct, r.metrics.smape_percent AS smape_pct, r.metrics.skill_vs_naive_percent AS skill_vs_naive_pct, r.metrics.mae AS timesfm_mae_bdt, r.metrics.naive_mae AS naive_mae_bdt, r.metrics.directional_accuracy_percent AS direction_accuracy_pct, r.metrics.interval_80_coverage_percent AS coverage_80_pct, 80.0 AS coverage_nominal_pct, d.variance_ratio, d.predicted_std_bdt, d.actual_std_bdt, r.metrics.r_squared FROM run r CROSS JOIN dispersion d

Predicted / actual variance

0.1%Source for 0.1%Source: GP run summary calculation Tables: read_json_auto, run, dispersion

The forecast variance is almost zero relative to the actual holdout variance, which is why the orange path looks flat.

Predicted standard deviation 1.12 Source for 1.12 Source: GP run summary calculation Tables: read_json_auto, run, dispersion Actual standard deviation 36.36 Source for 36.36 Source: GP run summary calculation Tables: read_json_auto, run, dispersion
Source for Predicted / actual variance The forecast variance is almost zero relative to the actual holdout variance, which is why the orange path looks flat.Source: GP run summary calculation Tables: read_json_auto, run, dispersion

Loads the saved run's headline metrics and calculates actual-versus-predicted dispersion from its backtest rows.

    WITH run AS (SELECT * FROM read_json_auto('prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json')), dispersion AS (SELECT var_samp(b.predicted::DOUBLE) / var_samp(b.actual::DOUBLE) AS variance_ratio, stddev_samp(b.predicted::DOUBLE) AS predicted_std_bdt, stddev_samp(b.actual::DOUBLE) AS actual_std_bdt FROM run, UNNEST(backtest) AS t(b)) SELECT r.metrics.accuracy_score AS accuracy_score_pct, r.metrics.smape_percent AS smape_pct, r.metrics.skill_vs_naive_percent AS skill_vs_naive_pct, r.metrics.mae AS timesfm_mae_bdt, r.metrics.naive_mae AS naive_mae_bdt, r.metrics.directional_accuracy_percent AS direction_accuracy_pct, r.metrics.interval_80_coverage_percent AS coverage_80_pct, 80.0 AS coverage_nominal_pct, d.variance_ratio, d.predicted_std_bdt, d.actual_std_bdt, r.metrics.r_squared FROM run r CROSS JOIN dispersion d

Holdout R-squared

-0.03 Source for -0.03 Source: GP run summary calculation Tables: read_json_auto, run, dispersion

A negative value means the forecast explains less holdout variation than predicting the holdout mean in the R-squared calculation.

Source for Holdout R-squared A negative value means the forecast explains less holdout variation than predicting the holdout mean in the R-squared calculation.Source: GP run summary calculation Tables: read_json_auto, run, dispersion

Loads the saved run's headline metrics and calculates actual-versus-predicted dispersion from its backtest rows.

    WITH run AS (SELECT * FROM read_json_auto('prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json')), dispersion AS (SELECT var_samp(b.predicted::DOUBLE) / var_samp(b.actual::DOUBLE) AS variance_ratio, stddev_samp(b.predicted::DOUBLE) AS predicted_std_bdt, stddev_samp(b.actual::DOUBLE) AS actual_std_bdt FROM run, UNNEST(backtest) AS t(b)) SELECT r.metrics.accuracy_score AS accuracy_score_pct, r.metrics.smape_percent AS smape_pct, r.metrics.skill_vs_naive_percent AS skill_vs_naive_pct, r.metrics.mae AS timesfm_mae_bdt, r.metrics.naive_mae AS naive_mae_bdt, r.metrics.directional_accuracy_percent AS direction_accuracy_pct, r.metrics.interval_80_coverage_percent AS coverage_80_pct, 80.0 AS coverage_nominal_pct, d.variance_ratio, d.predicted_std_bdt, d.actual_std_bdt, r.metrics.r_squared FROM run r CROSS JOIN dispersion d

## The forecast collapsed toward a level instead of tracking price movement

The hidden actual close ranged from BDT 222.20 to BDT 379.20 Source for 379.20 Source: Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 File: prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json, a BDT 157.00 Source for 157.00 Source: Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 File: prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json span. The prediction stayed between BDT 286.56 Source for 286.56 Source: Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 File: prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json and BDT 293.51 Source for 293.51 Source: Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 File: prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json, only a BDT 6.96 Source for 6.96 Source: Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 File: prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json span. Its variance was 0.095%Source for 0.095%Source: Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 File: prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json of actual variance, and predicted-versus-actual correlation was -0.30 Source for -0.30 Source: Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 File: prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json. Zooming in reveals small model movements; zooming out correctly shows that those movements are tiny relative to the stock's realized range.

Source for The forecast collapsed toward a level instead of tracking price movement Source: Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 File: prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json

## Longer evaluation does not reveal durable skill

The comparison below scores prefixes of the same one-origin forecast. TimesFM stays extremely close to the last-price baseline and finishes worse at 598 bars. This is a sensitivity check, not independent rolling validation, but it confirms that the large displayed accuracy score is not evidence of baseline-beating performance.

Source for Longer evaluation does not reveal durable skill Source: Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 File: prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json
MAE by evaluation horizon
Source for MAE by evaluation horizon Source: GP horizon sensitivity calculation Tables: read_json_auto, run, ordered, prefixes, selected

Calculates TimesFM and constant-last-price MAE over selected prefixes of the same ordered holdout forecast.

    WITH run AS (SELECT * FROM read_json_auto('prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json')), ordered AS (SELECT row_number() OVER (ORDER BY b.time) AS horizon_bars, b.actual::DOUBLE AS actual, b.predicted::DOUBLE AS predicted FROM run, UNNEST(backtest) AS t(b)), prefixes AS (SELECT horizon_bars, avg(abs(predicted - actual)) OVER (ORDER BY horizon_bars ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS timesfm_mae_bdt, avg(abs(286.6 - actual)) OVER (ORDER BY horizon_bars ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS last_price_mae_bdt FROM ordered), selected AS (SELECT *, (1 - timesfm_mae_bdt / NULLIF(last_price_mae_bdt, 0)) * 100 AS skill_vs_naive_pct FROM prefixes WHERE horizon_bars IN (30,60,96,128,192,256,300,384,512,598)) SELECT horizon_bars, 'TimesFM' AS series, timesfm_mae_bdt AS mae_bdt, 'solid' AS line_style, skill_vs_naive_pct FROM selected UNION ALL SELECT horizon_bars, 'Last-price baseline' AS series, last_price_mae_bdt AS mae_bdt, 'dashed' AS line_style, skill_vs_naive_pct FROM selected ORDER BY horizon_bars, series DESC

MAE by evaluation horizon data
Evaluated holdout bars | MAE | Forecast | Skill vs baseline
--- | --- | --- | ---
30 | 5.25BDT | TimesFM | -4.03%
30 | 5.05BDT | Last-price baseline | -4.03%
60 | 26.49BDT | TimesFM | -1.19%
60 | 26.18BDT | Last-price baseline | -1.19%
96 | 38.05BDT | TimesFM | -1.09%
96 | 37.63BDT | Last-price baseline | -1.09%
128 | 39.54BDT | TimesFM | -1.04%
128 | 39.13BDT | Last-price baseline | -1.04%
192 | 41.36BDT | TimesFM | -0.07%
192 | 41.33BDT | Last-price baseline | -0.07%
256 | 41.33BDT | TimesFM | 0.47%
256 | 41.53BDT | Last-price baseline | 0.47%
300 | 39.95BDT | TimesFM | 0.8%
300 | 40.28BDT | Last-price baseline | 0.8%
384 | 34.35BDT | TimesFM | 1.23%
384 | 34.78BDT | Last-price baseline | 1.23%
512 | 30.84BDT | TimesFM | 0.19%
512 | 30.89BDT | Last-price baseline | 0.19%
598 | 32.27BDT | TimesFM | -1.46%
598 | 31.81BDT | Last-price baseline | -1.46%

## The 88.63%Source for 88.63%Source: Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 File: prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json headline is a percentage-error transform, not forecast reliability

The application defines `accuracy` as 100 minus sMAPE. Prices around BDT 250-300 can therefore produce a visually high score even when the model misses path direction and volatility. For model selection, use baseline skill, MAE or MASE, directional performance, and interval calibration together; do not promote this score as a probability of correctness.

Source for The 88.63% headline is a percentage-error transform, not forecast reliability Source: Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 File: prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json

## Full-holdout metric audit

Source for Full-holdout metric audit Source: GP metric audit rows File: diagnostics/gp_metric_audit.sql

Shapes the reviewed full-holdout metrics and benchmark interpretations into an audit table.

    SELECT * FROM (VALUES (1, 'Displayed accuracy', '88.63%', 'Defined as 100 - sMAPE', 'Not a probability of a correct price or direction'), (2, 'Mean absolute error', 'BDT 32.27', 'Last-price MAE: BDT 31.81', 'Worse than the naive forecast'), (3, 'Skill vs last price', '-1.46%', 'Positive is better', 'No baseline-beating skill'), (4, 'Directional accuracy', '46.82%', '50% is a rough binary reference', 'No useful directional evidence in this run'), (5, 'R-squared', '-0.028', 'Zero is the holdout-mean reference', 'Explains no useful holdout variation'), (6, 'Q10-Q90 coverage', '4.52%', 'Nominal: 80%', 'Severely under-dispersed uncertainty'), (7, 'Price range', 'Predicted BDT 6.96', 'Actual BDT 157.00', 'Forecast path is genuinely flat at the full scale')) AS t(rank, metric, observed, benchmark, diagnosis) ORDER BY rank

Full-holdout metric audit
# | Metric | Observed | Benchmark or meaning | Diagnosis
--- | --- | --- | --- | ---
1 Source for 1 Source: GP metric audit rows File: diagnostics/gp_metric_audit.sql | Displayed accuracy | 88.63% | Defined as 100 - sMAPE | Not a probability of a correct price or direction
2 Source for 2 Source: GP metric audit rows File: diagnostics/gp_metric_audit.sql | Mean absolute error | BDT 32.27 | Last-price MAE: BDT 31.81 | Worse than the naive forecast
3 Source for 3 Source: GP metric audit rows File: diagnostics/gp_metric_audit.sql | Skill vs last price | -1.46% | Positive is better | No baseline-beating skill
4 Source for 4 Source: GP metric audit rows File: diagnostics/gp_metric_audit.sql | Directional accuracy | 46.82% | 50% is a rough binary reference | No useful directional evidence in this run
5 Source for 5 Source: GP metric audit rows File: diagnostics/gp_metric_audit.sql | R-squared | -0.03 | Zero is the holdout-mean reference | Explains no useful holdout variation
6 Source for 6 Source: GP metric audit rows File: diagnostics/gp_metric_audit.sql | Q10-Q90 coverage | 4.52% | Nominal: 80% | Severely under-dispersed uncertainty
7 Source for 7 Source: GP metric audit rows File: diagnostics/gp_metric_audit.sql | Price range | Predicted BDT 6.96 | Actual BDT 157.00 | Forecast path is genuinely flat at the full scale

## The test design is much harsher than the command suggests

The command loaded 1,195 daily bars and the default 50/50 split gave the model 597 context bars, then asked it for all 598 hidden bars in one open-loop path from January 23, 2024 to August 11, 2026. The `--future-steps 300` option controls the separate forward forecast; it does not limit the backtest to 300 bars. The implementation calls the backtest with the entire holdout length and never feeds hidden actuals back into later predictions.

That design is leakage-safe, but it is not representative of a strategy that refreshes a 5-, 12-, 20-, or 30-day forecast whenever a new close arrives. A very long raw-price forecast tends to accumulate error and regress toward a central level, making an almost-flat path unsurprising.

## CPU explains runtime, not accuracy

The saved run used `google/timesfm-2.0-500m-pytorch` on CPU with normalization enabled and the median quantile as the point forecast. The Tesla P100 has CUDA capability `sm_60`, while the installed PyTorch build contains kernels starting at `sm_70`, so the application correctly fell back to CPU. A compatible GPU would make inference faster and is useful for training, but it would not fix this forecast's information, target, horizon, or evaluation design.

The project also uses archived TimesFM 2.0. Google's current TimesFM repository documents TimesFM 2.5, including longer context, improved quantile support, exogenous-regressor support, and a LoRA fine-tuning path. Upgrade and benchmark it; do not assume the newer checkpoint automatically fixes GP. Google's TimesFM model overview also explains that long forecasts are generated in output patches, so later predictions depend on earlier generated values and error can accumulate.

## What this diagnosis establishes—and what it does not

This result establishes that this saved GP run and evaluation configuration failed useful baseline and calibration checks. It does not establish that TimesFM is universally poor, nor that another horizon, target, checkpoint, or rolling test will fail. The evidence is one stock and one forecast origin; the current target is the raw close; no sector, index, volume, corporate-action, or macro regressors were supplied; and no transaction-cost trading strategy was evaluated.

## Fix evaluation before fine-tuning

1. Add rolling-origin backtesting. At each historical cutoff, forecast the horizon actually used in decisions—start with 5, 12, 20, and 30 trading days—then advance the cutoff and aggregate results. Keep a final untouched period. A new `--backtest-horizon` option would make this explicit; `--future-steps` currently does not.
2. Benchmark simple forecasts. Compare TimesFM with last price, drift, and a suitable statistical baseline. Report MAE or MASE, skill versus baseline, direction, and Q10-Q90 coverage by horizon and market regime. Demote `100 - sMAPE` from the main accuracy label.
3. Test a stationary target. Compare raw close with log returns or price deltas, reconstruct prices for scoring, and preserve chronological splits. Long raw price levels are non-stationary and encourage level forecasts.
4. Benchmark TimesFM 2.5 zero-shot before training. It is the maintained line and supports longer context and exogenous regressors. Require repeated out-of-sample improvement rather than relying on one graph.
5. Use covariates without leakage. Candidate inputs include DSEX or DS30, sector movement, volume and turnover, calendar features, corporate-action flags, rates, and FX. Any future covariate must be known in advance or separately forecast as a scenario.
6. Fine-tune only with pooled data. One GP history of 1,195 observations is too small for a convincing 500M-model adaptation. Use sliding windows across many DSE equities and regimes, match training horizons to production, and use LoRA or QLoRA with strict ticker/time holdouts. Google's official fine-tuning guide documents this path for TimesFM 2.5.

## Promote a model only after it beats the baseline repeatedly

A replacement should show positive out-of-sample skill across multiple rolling folds and several DSE symbols, direction performance that is stable rather than a one-run fluctuation, and uncertainty coverage reasonably close to its nominal level. If the intended use is trading, the last gate is a transaction-cost- and slippage-aware strategy test. Better-looking forecast lines are not enough.

## Further questions

- What holding period will the forecast support: 1 day, 5 days, 12 days, or a longer allocation horizon?
- Are historical prices adjusted for dividends, splits, rights issues, and other corporate actions?
- Which DSE symbols and market regimes should form the pooled training and final evaluation universe?
- Which exogenous variables are available at prediction time without future leakage?

## Sources

1. Saved GP prediction run gp-1d-20260812T053147Z-f9ed7068 prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json
2. GP run summary calculation diagnostics/gp_run_summary.sql · duckdb · 2026-08-12T06:15:00Z

   Loads the saved run's headline metrics and calculates actual-versus-predicted dispersion from its backtest rows.

   SQL query

       WITH run AS (SELECT * FROM read_json_auto('prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json')), dispersion AS (SELECT var_samp(b.predicted::DOUBLE) / var_samp(b.actual::DOUBLE) AS variance_ratio, stddev_samp(b.predicted::DOUBLE) AS predicted_std_bdt, stddev_samp(b.actual::DOUBLE) AS actual_std_bdt FROM run, UNNEST(backtest) AS t(b)) SELECT r.metrics.accuracy_score AS accuracy_score_pct, r.metrics.smape_percent AS smape_pct, r.metrics.skill_vs_naive_percent AS skill_vs_naive_pct, r.metrics.mae AS timesfm_mae_bdt, r.metrics.naive_mae AS naive_mae_bdt, r.metrics.directional_accuracy_percent AS direction_accuracy_pct, r.metrics.interval_80_coverage_percent AS coverage_80_pct, 80.0 AS coverage_nominal_pct, d.variance_ratio, d.predicted_std_bdt, d.actual_std_bdt, r.metrics.r_squared FROM run r CROSS JOIN dispersion d

3. GP horizon sensitivity calculation diagnostics/gp_horizon_mae.sql · duckdb · 2026-08-12T06:15:00Z

   Calculates TimesFM and constant-last-price MAE over selected prefixes of the same ordered holdout forecast.

   SQL query

       WITH run AS (SELECT * FROM read_json_auto('prediction-runs/gp-1d-20260812T053147Z-f9ed7068.json')), ordered AS (SELECT row_number() OVER (ORDER BY b.time) AS horizon_bars, b.actual::DOUBLE AS actual, b.predicted::DOUBLE AS predicted FROM run, UNNEST(backtest) AS t(b)), prefixes AS (SELECT horizon_bars, avg(abs(predicted - actual)) OVER (ORDER BY horizon_bars ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS timesfm_mae_bdt, avg(abs(286.6 - actual)) OVER (ORDER BY horizon_bars ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS last_price_mae_bdt FROM ordered), selected AS (SELECT *, (1 - timesfm_mae_bdt / NULLIF(last_price_mae_bdt, 0)) * 100 AS skill_vs_naive_pct FROM prefixes WHERE horizon_bars IN (30,60,96,128,192,256,300,384,512,598)) SELECT horizon_bars, 'TimesFM' AS series, timesfm_mae_bdt AS mae_bdt, 'solid' AS line_style, skill_vs_naive_pct FROM selected UNION ALL SELECT horizon_bars, 'Last-price baseline' AS series, last_price_mae_bdt AS mae_bdt, 'dashed' AS line_style, skill_vs_naive_pct FROM selected ORDER BY horizon_bars, series DESC

4. GP metric audit rows diagnostics/gp_metric_audit.sql · duckdb · 2026-08-12T06:15:00Z

   Shapes the reviewed full-holdout metrics and benchmark interpretations into an audit table.

   SQL query

       SELECT * FROM (VALUES (1, 'Displayed accuracy', '88.63%', 'Defined as 100 - sMAPE', 'Not a probability of a correct price or direction'), (2, 'Mean absolute error', 'BDT 32.27', 'Last-price MAE: BDT 31.81', 'Worse than the naive forecast'), (3, 'Skill vs last price', '-1.46%', 'Positive is better', 'No baseline-beating skill'), (4, 'Directional accuracy', '46.82%', '50% is a rough binary reference', 'No useful directional evidence in this run'), (5, 'R-squared', '-0.028', 'Zero is the holdout-mean reference', 'Explains no useful holdout variation'), (6, 'Q10-Q90 coverage', '4.52%', 'Nominal: 80%', 'Severely under-dispersed uncertainty'), (7, 'Price range', 'Predicted BDT 6.96', 'Actual BDT 157.00', 'Forecast path is genuinely flat at the full scale')) AS t(rank, metric, observed, benchmark, diagnosis) ORDER BY rank

5. Local TimesFM forecasting implementation dohasecuritiesstockai/timesfm_forecasting/pipeline.py · source-code

   Local forecasting pipeline, TimesFM backend, and metric definitions inspected for split, horizon, device, normalization, and scoring behavior.