The Mithai Shop Challenge
+19 DAYS
Diwali shifted from 20 October 2025 to 8 November 2026. Would a forecast follow the new date—or repeat the old pattern?
23N AI LABS / RESEARCH NOTE 01V2 · 2026
The calendar moves.
Does the forecast?
A plain-language guide to a synthetic study of festival demand, forecast timing, and production decisions. Discussion brief; not peer-reviewed or based on retailer sales.
23N AI LABS · RESEARCH01 / 15
01 / THE QUESTIONWHY THIS STUDY
Big events move.
Demand moves with them.
A forecast trained on the past may remember last year’s date more strongly than next year’s calendar.
QUESTION
When a festival shifts, can a model put the demand peak on the new date?
Then: does a better-timed forecast help choose how much to make?
When a festival shifts, can a model put the demand peak on the new date?
Then: does a better-timed forecast help choose how much to make?
The study tests these questions in a controlled simulation. It does not measure a real shop.
23N AI LABS · RESEARCH02 / 15
02 / THE TEST SHOPSIMULATED AHMEDABAD
A shop built in code.
Realistic ingredients. Artificial transactions.
3kaju katli · motichoor ladoo · gulab jamun
2019–26daily demand and sales
≤30independently randomized shops in the base setting
The simulator includes weekdays, seasonal patterns, festival lifts, advance corporate orders, weddings, bulk orders, noise, and stock limits.
IMPORTANT · The shops, sales, prices, costs, and savings are simulated. No retailer records were used.
23N AI LABS · RESEARCH03 / 15
03 / A DATA TRAPSALES ≠ DEMAND
Sold out is not the same as “no one wanted more.”
When stock runs out, the till only records what the shop could sell.
200 kgcustomers want
150 kgshop has in stock
150 kgsales recorded
The missing 50 kg disappears from sales history.
The study tests models that account for this censoring—sales capped by available inventory—because learning from sales alone can make real demand look smaller.
23N AI LABS · RESEARCH04 / 15
04 / THE COMPARISON13 CONFIGURATIONS
Thirteen ways to forecast.
Three model families, statistical baselines, and a practical shopkeeper rule.
PRETRAINED MODELS
Yug-1
TimesFM 2.5
Chronos-Bolt
STATISTICAL
Seasonal naive
ETS
Calendar regression
OPERATING RULES
Shopkeeper multiplier
Calibrated shopkeeper
Inventory-aware regression
ZERO-SHOT = the pretrained models forecast without being trained on this shop’s history.
Some methods also received the same external festival-calendar adjustment.
23N AI LABS · RESEARCH05 / 15
05 / FAIR COMPARISONCOMMON CONDITIONS
Same simulator.
Same planning window.
Each method forecast 56 days ahead, with up to 2,048 days of history.
56days in each forecast horizon
2,048maximum history days
13evaluated configurations
Festival correction used known future event dates and patterns in past observed sales. It was an added application step—not a built-in feature of Yug-1, TimesFM, or Chronos.
COMPARISON RULE
Shared constraints and shared simulated demand make the methods comparable inside this experiment.
Shared constraints and shared simulated demand make the methods comparable inside this experiment.
23N AI LABS · RESEARCH06 / 15
06 / RESULT ONEHISTORY-ONLY PEAK TIMING
Several models peaked early.
When Diwali moved 19 days, history-only forecasts often leaned toward the previous festival date.
| Method | Distinct peak | Median timing error | Phantom surge |
|---|---|---|---|
| Yug-1 | 48% | −17 days | +29% |
| TimesFM 2.5 | 33% | −16 days | +15% |
| Chronos-Bolt | 23% | −19 days | +17% |
| Seasonal naive | 100% | −20 days | +45% |
Negative timing error means early. “Distinct peak” is the share with a clear peak; timing is reported only for those cases. Phantom surge means excess forecast weight near the old festival date, adjusted for overall bias.
23N AI LABS · RESEARCH07 / 15
07 / RESULT TWOADD THE KNOWN DATE
A calendar correction moved peaks back on time.
Across tested calendar-informed methods, distinct peaks appeared in 88–96% of cases.
−1 to 0median days from actual peak
+4–11%phantom surge near old dates
All 4tested families improved WAPE
Explicit festival dates helped Yug-1, TimesFM, Chronos, and ETS in the tested scenarios.
TAKEAWAY · Supplying a relevant event date helped more than switching among the tested model architectures.
These findings are specific to the simulated 19-day shift and simulator assumptions.
23N AI LABS · RESEARCH08 / 15
08 / ERROR REDUCTIONWAPE · PERCENTAGE POINTS
Calendar information reduced forecast error.
Improvements were positive in the base setting and all three stress scenarios.
| With calendar | Base | R1 | R2 | R3 |
|---|---|---|---|---|
| Yug-1 | 10.3 | 5.6 | 9.4 | 4.7 |
| TimesFM 2.5 | 6.2 | 2.9 | 6.0 | 2.6 |
| Chronos-Bolt | 8.0 | 4.2 | 7.3 | 3.7 |
| ETS | 10.4 | 6.7 | 9.6 | 5.9 |
Values are percentage-point reductions in WAPE—not final WAPE scores. R1 changes festival shape year to year; R2 adds local shocks; R3 combines both.
23N AI LABS · RESEARCH09 / 15
09 / RESULT THREEFORECASTS DRIVE A PLAN
Better predictions do not always mean a better plan.
The decision layer has its own trade-offs: waste, lost sales, capacity, and shelf life.
| Method | Simulated regret / 56-day plan |
|---|---|
| Censor-aware calendar regression | ₹47,602 |
| Calendar regression | ₹50,855 |
| TimesFM + calendar | ₹52,847 |
| Yug-1 + calendar | ₹52,882 |
| Yug-1 history only | ₹87,051 |
| TimesFM history only | ₹82,656 |
Lower regret is better. Simulated INR per product plan; not actual revenue, profit, or savings. Yug-1 + calendar versus TimesFM + calendar differed by ₹35; paired 95% interval −₹1,914 to +₹1,603, so that comparison is inconclusive.
23N AI LABS · RESEARCH10 / 15
10 / RESULT FOURCHECK THE UNCERTAINTY
One random seed can flatter a model.
Repeated samples revealed meaningful variation in forecast quality.
68–80%festival-period coverage for a nominal 80% interval
41.1–42.6%festival WAPE across five Yug-1 seeds
15 / 16business comparisons favored Yug-1 under the original seed
The original seed was unusually favorable. That is a reason to report seed sensitivity—not proof of a software defect.
GOOD PRACTICE · Report repeated runs, uncertainty, and the cases where a model does not produce a distinct peak.
23N AI LABS · RESEARCH11 / 15
11 / REPRODUCIBILITYPUBLIC VERIFICATION RECORD
The checks are inspectable.
Independent verification recomputed the released metrics.
42archived synthetic forecast files
241,920forecast rows
18 / 18metric checks matched
The verification record found no non-finite or crossed-quantile rows. Code, notebooks, pinned model revisions, result summaries, and audit scripts are public. The raw forecast archives (about 367 MB) are not included in the public repository.
SOURCE · github.com/23nlabs/mithai-shop-challenge
REPRODUCE · Reproduction instructions
DETAILS · Technical appendix
REPRODUCE · Reproduction instructions
DETAILS · Technical appendix
23N AI LABS · RESEARCH12 / 15
12 / LIMITSWHAT THIS DOES NOT SHOW
This is a simulator result.
Not a shop trial.
The result depends on assumptions about customers, prices, stock, and festival effects.
NO REAL SALES
No retailer transactions or realized savings.
NO REAL-WORLD CERTAINTY
Intervals describe simulated shops and model seeds.
NEXT QUESTIONS
Test native calendar inputs, more seeds, real consented data, and different planning policies.
This is a controlled demonstration of how calendar knowledge and inventory rules can change forecast evaluation.
23N AI LABS · RESEARCH13 / 15
13 / THE SHORT VERSIONWHAT TO REMEMBER
Give the forecast the date.
Then test the plan.
In this simulation, festival dates mattered. Stockouts hid demand. Repeated runs changed the picture. And the lowest-error forecast did not automatically make the best production decision.
One practical next step
Compare the calendar correction with native event inputs—and validate against consented real shop data.
No model winner should be inferred beyond the scenarios and policies tested here.
23N AI LABS · RESEARCH14 / 15
14 / QUICK GLOSSARYPLAIN-LANGUAGE TERMS
Four terms, without the fog.
Time-series foundation model
A pretrained model designed to forecast values over time.
Zero-shot
Forecasting without training the model on this shop’s history.
WAPE
Total absolute error divided by total demand. Lower is better.
Censoring
When stock runs out, sales stop showing how much customers still wanted.
Coverage
How often actual demand falls within a forecast’s stated range.
Regret
The simulated gap from a feasible plan made with perfect hindsight. Lower is better.
23N AI LABS · RESEARCH15 / 15