Creating a GDP Nowcast, Part 1: How I Built It and Why It Failed

TL;DR: I attempted to build a real-time GDP Nowcast from scratch using monthly indicators (Jobs, Production, Retail) to predict US growth before the official release. My "Beta" model achieved strong historical accuracy (65% R^2) but failed live testing, revealing a critical "Inflation Trap" where nominal sales data inverted the growth signal during the 2022-2024 price shocks. Features: Automated R-Stata pipeline, FRED API integration, robust relative pathing, and stationarity diagnostics.

Every software product has a "Beta" version—the rough draft that works perfectly in the lab but breaks the moment you hand it to a real user.

Have you ever had the following frustration: "We can't wait for the government report. If I could build us a tool that tells us what the economy is doing right now."

This is the story of my Beta GDP Nowcast. It is a story about how I built a data pipeline from scratch, achieved an incredible 65% accuracy in backtesting, and then watched the model completely fail in the live environment.

This post documents the process, the code, and the critical lesson I learned.
(Spoiler: The fix is coming in Part 2).

The Mission: Solving the "Data Lag"

In the world of finance, speed is everything. Hedge funds pay millions of dollars for "Nowcasting" models—algorithms that take fast-moving daily data and use them to guess the slow-moving GDP number.

My goal was to replicate this "Wall Street" technology using my own laptop, open-source data, and a bit of grit.

Step 1: Building the Machine (The R Pipeline)

Before I could do any economics, I had to do "Data Engineering." I faced a classic "Apples and Oranges" problem:

The Target: GDP (Released once every 3 months).
The Clues: Industrial Production, Retail Sales, Payrolls, Housing (Released every month).

You can't just plug these into a spreadsheet. They don't match up.

I wrote an R script using fredr to pull the data and tidyverse to "harmonize" it. The goal was to aggregate monthly data into quarterly averages so everything speaks the same language.

The Code: Cleaning and Harmonizing Frequencies

clean_data <- raw_data_all %>%
  # 1. Round dates down to the start of the quarter
  mutate(quarter = floor_date(date, unit = "quarter")) %>%

  # 2. Group by Quarter and Variable
  group_by(quarter, variable_name) %>%

  # 3. Calculate the Quarterly Average
  summarize(value = mean(value, na.rm = TRUE)) %>%

  # 4. Pivot from "Long" to "Wide" (Spreadsheet format)
  pivot_wider(names_from = variable_name, values_from = value) %>%

  # 5. Export for Stata
  write_dta("02_Clean_Data/gdp_nowcast.dta")

After debugging API keys and date formats, I finally had it: a clean, perfect dataset spanning 25 years of US economic history.

Step 2: The "Engines" of the Economy

For this Beta test, I selected four "engines" to power the model:

Making Things (INDPRO): Industrial Production.
Buying Things (RSAFS): Retail Sales (Nominal).
Working (PAYEMS): Nonfarm Payrolls.
Building Things (HOUST): Housing Starts.

With my dataset ready, I switched to Stata to run the econometrics.

Step 3: The "Optical Illusion" of Success

I ran a regression to see if these four "engines" actually predict GDP. I compared a simple baseline (just factories) against my full model.

The Stata Code: The "Horse Race"

Stata

* Model 1: Baseline (Manufacturing Only)
regress GDP Ind_Production, robust

* Model 2: The Beta Nowcast (All Engines)
regress GDP Ind_Production Retail_Sales Payrolls Housing_Starts, robust
estimates store Full_Model

* Compare Accuracy
estimates table Baseline Full_Model, star stats(r2_a)

The results were shocking. The accuracy (R^2) skyrocketed from 7% to 65%.

----------------------------------------------
    Variable |   Baseline       Full_Model    
-------------+--------------------------------
Ind_Produc~n |  .23933602       .18774394**   
Retail_Sales |                 -.10692069***  
    Payrolls |                  1.1934947***  
Housing_St~s |                  .00683511     
       _cons |  2.0031253***      1.10761*    
-------------+--------------------------------
        r2_a |  .07183827       .65537652     
           N |        103             103     
----------------------------------------------
      Legend: * p<0.05; ** p<0.01; *** p<0.001

I watched the graph appear on my screen. The blue line (my AI model) tracked the gray line (actual GDP) almost perfectly for 20 years. It caught the 2008 crash. It caught the COVID rebound.

I thought I had a finished product. I was wrong.

The Beta Failure: The "Inflation Trap"

Overconfidence is a dangerous thing wherever you go. I decided to run the ultimate test: asking the model to predict today (Late 2025).

The Stata Code: Generating the Forecast

Stata

estimates restore Full_Model
predict GDP_Nowcast, xb

* Show the most recent performance
list quarter GDP GDP_Nowcast in -4/l

Reality: The US economy is currently booming (GDP ~4%).
My Beta Model: It predicted a slump (GDP ~1.1%).

     +------------------------------+
     | quarter       GDP   GDP_No~t |
     |------------------------------|
100. |  2024q4   1.85223   3.702773 |
101. |  2025q1   -.64848   2.468095 |
102. |  2025q2   3.83803   2.161778 |
103. |  2025q3   4.34278   1.096498 |
     +------------------------------+

I stared at the screen. My model was off by a mile on the current moment. How could it be so right about the past and so wrong about the present?

I dove back into the regression results, looking for the bug. And then I saw it.

------------------------------------------------------------------------------
         GDP | Coefficient  Std. err.      t    P>|t|     [95% conf. interval]
-------------+----------------------------------------------------------------
Retail_Sales |  -.1069207   .018467    -5.79   0.000    -.1435678   -.0702735
------------------------------------------------------------------------------

Retail Sales had a negative coefficient (-0.10).

In plain English, my model had "learned" that Higher Sales = Lower Growth.

The Diagnosis:

I had fed the model "Nominal Retail Sales"—the raw dollar amount spent at stores.

In normal times, more spending means more goods sold.
But in high inflation times (like 2022-2024), spending goes up just because prices are higher, even if people are buying fewer actual things (so real GDP goes down).

My Beta model looked at the high-inflation period of the 2020s and got confused. It saw spending skyrocketing while the real economy struggled, so it concluded that "Spending is bad."

What's Next: Version 2.0

This failure is exactly why we run Beta tests. A high "R-squared" doesn't mean your model understands the world; it just means it found a pattern. And if the world changes—like a shift from high inflation to low inflation—that pattern breaks.

I am currently working on Version 2.0.

The fix is clear: I need to strip the inflation noise out of the data. In my next post, I will be re-engineering the pipeline to use Real Retail Sales (inflation-adjusted). I have a hypothesis that once we make this swap, the negative coefficient will flip, and the forecast will catch up to reality.

Technical Appendix

Data Sources:

Federal Reserve Economic Data (FRED): https://fred.stlouisfed.org/
Specific Series: Real GDP (GDPC1), Industrial Production (INDPRO), Retail Sales (RSAFS), Nonfarm Payrolls (PAYEMS), Housing Starts (HOUST).

Software:

RStudio: Data cleaning, frequency conversion (Monthly to Quarterly), and API fetching.
Stata 19.5: Time-series definition (tsset), Stationarity tests (dfuller), and OLS forecasting.

Key Packages:

R: fredr, tidyverse, lubridate, haven
Stata: estout

Replication:

Full code and data available on my GitHub: https://github.com/junseoul0163/GDP_Nowcast