fix: support commodity/forex/crypto tickers and never invent prices (#781)

Analyzing a symbol Yahoo Finance does not recognize (e.g. XAUUSD+) could
produce an invented price instead of an error. The agent now either prices
the correct instrument or clearly reports that data is unavailable.

Ticker support:
- Commodities/forex/crypto resolve to the symbol Yahoo actually serves, so
  you can enter the common form and it just works:
    XAUUSD / XAUUSD+ / GOLD  -> GC=F   (gold)
    USOIL                    -> CL=F   (WTI crude)
    EURUSD                   -> EURUSD=X
    BTCUSD                   -> BTC-USD
    SPX500 / NAS100          -> ^GSPC / ^NDX
  Native Yahoo symbols (AAPL, GC=F, ^GSPC) keep working unchanged. New
  instruments are added by extending the alias table.

Reliability:
- Unknown or delisted symbols now return a clear "data unavailable" result
  the agent reports verbatim, instead of a value the model fills in.
- A failed fetch no longer leaves a broken symbol cached until the cache is
  cleared by hand.
This commit is contained in:
Yijia-Xiao committed 2026-05-31 22:38:47 +00:00
1 parent 2f85be624e
commit 1ff3f07a73
10 files changed
+460 -53

No files matched your search

+24 -8
View File
@@ -9,6 +9,7 @@ from typing import Annotated
import os
from .config import get_config
from .utils import safe_ticker_component
from .symbol_utils import normalize_symbol, NoMarketDataError
logger = logging.getLogger(__name__)
@@ -68,9 +69,11 @@ def load_ohlcv(symbol: str, curr_date: str) -> pd.DataFrame:
subsequent calls the cache is reused. Rows after curr_date are
filtered out so backtests never see future prices.
"""
# Reject ticker values that would escape the cache directory when
# Resolve broker/forex symbols (XAUUSD+ -> GC=F) to Yahoo's convention,
# then reject values that would escape the cache directory when
# interpolated into the cache filename (e.g. ``../../tmp/x``).
safe_symbol = safe_ticker_component(symbol)
canonical = normalize_symbol(symbol)
safe_symbol = safe_ticker_component(canonical)
config = get_config()
curr_date_dt = pd.to_datetime(curr_date)
@@ -87,19 +90,32 @@ def load_ohlcv(symbol: str, curr_date: str) -> pd.DataFrame:
f"{safe_symbol}-YFin-data-{start_str}-{end_str}.csv",
)
# A cached file may be empty if a prior fetch failed (unknown symbol,
# transient rate limit). Treat an empty/columnless cache as a miss and
# re-fetch rather than serving the poisoned file forever.
data = None
if os.path.exists(data_file):
data = pd.read_csv(data_file, on_bad_lines="skip", encoding="utf-8")
else:
data = yf_retry(lambda: yf.download(
symbol,
cached = pd.read_csv(data_file, on_bad_lines="skip", encoding="utf-8")
if not cached.empty and "Close" in cached.columns:
data = cached
if data is None:
downloaded = yf_retry(lambda: yf.download(
canonical,
start=start_str,
end=end_str,
multi_level_index=False,
progress=False,
auto_adjust=True,
))
data = _ensure_date_column(data.reset_index())
data.to_csv(data_file, index=False, encoding="utf-8")
downloaded = _ensure_date_column(downloaded.reset_index())
# Only cache real data — never persist an empty frame.
if downloaded.empty or "Close" not in downloaded.columns:
raise NoMarketDataError(
symbol, canonical, "Yahoo Finance returned no rows"
)
downloaded.to_csv(data_file, index=False, encoding="utf-8")
data = downloaded
data = _clean_dataframe(data)