Data Provider Coverage — Beyond Markets Analysis Platform
Authoritative reference combining all prior coverage documents and current subscriptions. Scoped to Beyond Markets as an analysis report platform — not a real-time trading execution bot. Real-time streaming, L2 order book, broker execution, and intraday tick data are out of scope and excluded from gap analysis.
Last updated: 2026-07-10 (Alpha Vantage removed — enabled but never invoked; commodities/crypto downgraded to partial via FMP batch quotes. ETF.com etf_flows retired — scrape now HTTP 403, no working ETF-flow source; ETF fund-flow coverage now a gap.)
Current Subscriptions & Monthly Cost
| Provider | Module key | Plan | Cost/month | Auth |
|---|---|---|---|---|
| Alpaca (market data) | alpaca_mktdata |
Algo Trader Plus (data API) | $99 | ALPACA_API_KEY_PAPER |
| Alpaca (trading) | n/a (routes only) | Paper + Live | $0 | ALPACA_API_KEY_PAPER/LIVE |
| Articles (local archive) | articles |
Internal | $0 | none |
| BEA (Bureau of Economic Analysis) | macro_calendar |
Public | $0 | BEA_API_KEY |
| Benzinga (pro.benzinga.com) | benzinga |
Paid | TBD | BEZINGA_API_KEY |
| BIS | imf |
Public | $0 | none |
| BLS (Bureau of Labor Statistics) | macro_calendar |
Public | $0 | BLS_API_KEY (optional) |
| ECB | imf |
Public | $0 | none |
| EODHD (eodhistoricaldata.com) | eodhd |
Paid | TBD | EODHD_API_KEY |
| Financial Modeling Prep | fmp |
Premium | $49.00 | FMP_API_KEY |
| FINRA REGSHO | finra_short |
Public | $0 | none |
| FinViz Elite | finviz |
Elite | $39.50 | FINVIZ_API_TOKEN |
| Forex Factory | economic_calendar |
Public | $0 | none |
| FRED (St. Louis Fed) | fred |
Public | $0 | FRED_API_KEY |
| GDELT | gdelt |
Free | $0 | none |
| iborrowdesk | iborrowdesk |
Public | $0 | none |
| IMF DataMapper | imf |
Public | $0 | none |
| NewsAPI.ai (Event Registry) | newsapi_ai |
5K | $90.00 | NEWSAPI_AI_KEY |
| NewsAPI.org | newsapi |
Developer | $0 | NEWSAPI_ORG_KEY |
| NewsData.io | newsdata |
Free | $0 | NEWSDATA_API_KEY |
| OpenFIGI | openfigi |
Free | $0 | OPENFIGI_API_KEY (optional) |
| Polygon.io | polygon |
Advanced | $29.00 | POLYGON_API_KEY |
| SEC EDGAR | sec + 13f |
Public | $0 | none |
| SEC FTD (REGSHO fails-to-deliver) | sec_ftd |
Public | $0 | none |
| Seeking Alpha | — not integrated | Premium | $20.83 ($250/yr) | n/a (no public API) |
| TrendSpider | inbound webhook only (webhook_store) |
Premium | $137.00 | TRENDSPIDER_WEBHOOK_SECRET (no public API — push only) |
| Unusual Whales | uw |
Paid | $64.75 | UNUSUAL_WHALES_EMAIL / UNUSUAL_WHALES_PASSWORD (session login) |
Integrated data-feed spend: $272.25 + EODHD
Non-integrated subscriptions (manual / UI tools, no data feed): TrendSpider $137.00 + Seeking Alpha $20.83 = $157.83/month. These are paid for but not wired into the pipeline — see How Providers Are Integrated below for why and how they could be.
How Providers Are Integrated
Every integrated provider is a tool module (tools/data_ingestion_*.py) exposing TOOL_DEFINITIONS + run_tool(name, input), called through the shared SQLite TTL cache. They fall into a few integration patterns:
| Pattern | How it works | Providers |
|---|---|---|
| Authenticated REST API | HTTPS calls with an API key from .env; JSON responses parsed into tool results |
FMP, Polygon, EODHD, NewsAPI.ai, NewsData.io, NewsAPI.org, FinViz Elite (token), Benzinga, OpenFIGI, BEA, BLS, FRED |
| Session-authenticated REST | Headless Playwright login → cached session bearer token, then live REST calls to phx.unusualwhales.com. No offline fallback (tools/uw_session.py + data_ingestion_external_data.py). |
Unusual Whales (uw) |
| Public REST, no auth | Open government / institutional endpoints, no key needed | SEC EDGAR, SEC FTD, FINRA REGSHO, IMF, ECB, BIS, GDELT, iborrowdesk |
| Web scrape | HTML/CSV fetched and parsed (no official API) | Forex Factory (calendar) |
| Downloaded files / local archive | Files fetched on a schedule or stored locally, read from disk | Articles (Economist archive) |
| Broker SDK | Official alpaca-py SDK with per-tenant keys (trading) / env keys (market data) |
Alpaca |
Not integrated (paid, but no data feed wired in):
| Provider | Why not integrated | How it could be integrated |
|---|---|---|
| TrendSpider | Still no queryable API on the Premium plan — nothing can be pulled. But it can push, and that half is now wired in (see Inbound webhooks below). The market data it shows (equities, options, futures, FX, crypto) is already covered by Polygon, EODHD and Alpaca, so only the alerts are worth taking. | Pull-side would need manual CSV export + a one-off importer. Its unique capability (no-code backtesting / variance testing) can't be pulled programmatically at all. |
| Seeking Alpha | No official public API, and scraping is actively blocked. Its analyst ratings, price targets, transcripts and dividends already come from Benzinga / FMP / Polygon. | Its unique Quant Ratings & Factor Grades and earnings-call transcripts would require cookie-authenticated scraping — the same pattern the articles module already uses for the Economist archive. This is the realistic path if transcripts (see gap below) become a priority. |
Inbound webhooks (push, not pull)
Tools with no queryable API can still fire alerts at us. webhook_store.py is the
shared landing zone: a source-keyed table in .data/webhooks.db holding one row per
pushed ticker, expiring after 24 hours by read-time cutoff (same model as the news
store). Each source gets a receiver in routes/webhooks.py.
| Source | Receiver | Read back as | Setup |
|---|---|---|---|
| TrendSpider | POST /api/webhooks/trendspider |
navbar bell / admin only | Settings → Notifications → Webhook for alerts, URL https://beyondmarkets.ai/api/webhooks/trendspider?secret=<TRENDSPIDER_WEBHOOK_SECRET>&symbol={{ticker}}. The URL is account-wide — every alert with Webhook ticked hits it, so filter on the TrendSpider side. |
| TradingView | POST /api/webhooks/tradingview |
GET /api/webhooks/tradingview (admin only) |
Alert → Notifications → Webhook URL, https://beyondmarkets.ai/api/webhooks/tradingview?secret=<TRADINGVIEW_WEBHOOK_SECRET>. Symbol comes from a symbol/ticker field in the JSON alert message, or ?symbol=. |
Neither source is a screener list any more. TrendSpider used to be one, backed by
webhook_store.SCREENER_SOURCES and a 1h window; it measured empty over 157 snapshots
(screener_updates.md) and the list, its read route and the whitelist were removed.
Both sources still write, and both are read back only by the navbar bell and the admin
view — so a chart alert cannot put a ticker in front of the trade proposer.
The symbol is always taken from ?symbol= or a JSON field, never parsed out of the
alert body: TrendSpider sends its free-text Note as the raw body, and BUY / LONG /
RSI all look like tickers. The body is stored verbatim as row context. An alert that
carries no resolvable symbol is rejected with 400 — the store is symbol-keyed, so
there is nothing to record without one.
Coverage by Data Category
Legend: ✅ Fully covered · ⚠️ Partial · ❌ Not covered · — Not applicable (out of scope for analysis platform)
Prices & Market Data
| Data type | Status | Source |
|---|---|---|
| Previous day OHLCV | ✅ | polygon (polygon_prev_close), fmp |
| Historical OHLCV (daily/weekly/monthly) | ✅ | polygon (polygon_aggregates), fmp, finviz |
| Intraday candles (1min–4h) | ✅ | fmp (get_intraday_prices), polygon (5-min, Advanced plan — use ms timestamps) |
| Intraday bars extended hours (pre/post-market) | ✅ | polygon (04:00–20:00 ET), fmp |
| Live price snapshot (batch) | ✅ | alpaca_mktdata (get_stock_snapshots), alpaca (get_snapshots — per-tenant TLS) |
| Market movers (gainers/losers) | ✅ | fmp (get_market_movers), alpaca_mktdata (get_market_movers) |
| Most active stocks | ✅ | alpaca_mktdata (get_most_active) |
| Pre/after-hours price | ✅ | fmp (get_aftermarket_quote) |
| Bid/ask (L1) | ✅ | fmp (get_quote_l1) |
 VWAP |
⚠️ | Available in polygon_aggregates daily bar; no intraday VWAP |
| Company reference data (market cap, shares out, SIC) | ✅ | polygon (polygon_ticker_details), fmp |
| Dividends | ✅ | polygon (polygon_dividends), fmp |
| Stock screener | ✅ | finviz (screener), fmp (screen_stocks) |
| Sector / group performance | ✅ | fmp (get_sector_performance), finviz (group_performance) |
Macro & Economic Data
| Data type | Status | Source |
|---|---|---|
| US interest rates (Fed funds, yield curve) | ✅ | fred, fmp (get_treasury_rates) |
| CPI / PCE / PPI inflation | ✅ | fred (get_inflation) |
| GDP (real & nominal) | ✅ | fred, imf |
| Labor market (unemployment, NFP, JOLTS) | ✅ | fred (get_labor_market) |
| Credit conditions (OAS spreads, lending standards) | ✅ | fred (get_credit_conditions) |
| Money supply (M2) | ✅ | fred (get_money_supply) |
| Housing data | ✅ | fred (get_housing) |
| Consumer sentiment | ✅ | fred (get_consumer_indicators) |
 VIX |
✅ | fred (get_market_indicators), fmp |
| Dollar index (DXY) | ✅ | fmp |
| Forex (EUR, GBP, JPY, CHF, CNY) | ✅ | imf (ECB FX rates), fmp (batch quotes) |
| Commodities (oil, gold, silver, ag) | ⚠️ | fmp (batch quotes accept major commodity symbols; no dedicated feed) |
| Crypto (BTC, ETH, SOL) | ⚠️ | fmp (batch quotes accept major crypto symbols; no dedicated feed) |
| International macro (GDP, CPI, debt, current account) | ✅ | imf (241 countries, WEO projections to 2031) |
| Euro area monetary data | ✅ | imf (ECB: HICP, yield curve, €STR, bank lending) |
| BIS financial stability indicators | ✅ | imf (BIS: credit gaps, debt service, property prices) |
| Macro liquidity regime score | ✅ | macro_liquidity (composite: real rates, CB balance sheet, credit, funding stress) |
| Economic calendar (CPI/NFP/FOMC dates + forecast) | ✅ | economic_calendar (Forex Factory, current week) |
| Multi-week macro calendar look-ahead | ✅ | macro_calendar (macro_calendar_upcoming) — FRED release dates, any look-ahead |
Fundamentals & Valuation
| Data type | Status | Source |
|---|---|---|
| Income statement (revenue, EPS, margins) | ✅ | fmp, sec |
| Balance sheet (assets, debt, cash) | ✅ | fmp, sec |
| Cash flow statement (FCF) | ✅ | fmp, sec |
| Key metrics (P/E, EV/EBITDA, P/S, ROE) | ✅ | fmp (get_key_metrics, get_financial_ratios) |
| DCF valuation | ✅ | fmp (get_dcf_valuation) |
| ESG data | ✅ | fmp (get_esg) |
| XBRL financial facts | ✅ | sec (get_sec_company_facts) |
| Full filing history (10-K, 10-Q, 8-K links) | ✅ | sec (get_sec_submissions), fmp |
| Full-text SEC filing search | ✅ | sec (search_sec_filings) |
Earnings & Analyst Data
| Data type | Status | Source |
|---|---|---|
| Next earnings date | ✅ | fmp (get_analyst_data), finviz |
| EPS / revenue estimates | ✅ | fmp (get_analyst_data) |
| Earnings beat/miss history | ✅ | fmp |
| Analyst upgrades / downgrades | ✅ | fmp (get_analyst_data), finviz, benzinga (benzinga_ratings) |
| Price target changes | ✅ | fmp |
| Consensus rating | ✅ | fmp |
| Estimate revision velocity/breadth | ⚠️ | FMP has individual revisions; no breadth/velocity aggregation |
| Earnings transcripts | ❌ | Not available |
News & Sentiment
| Data type | Status | Source |
|---|---|---|
| Financial / market news search | ✅ | newsapi_ai, newsdata, gdelt, fmp, alpaca_mktdata (get_alpaca_news) |
| Company-specific news | ✅ | newsapi_ai (get_company_news), fmp, finviz, alpaca_mktdata |
| Macro / central bank news | ✅ | newsapi_ai, gdelt |
| Geopolitical news | ✅ | newsapi_ai, gdelt (get_gdelt_geo_news) |
| Article sentiment scoring | ✅ | newsapi_ai (−1.0 to +1.0 per article), newsdata |
| Sentiment timeline / trend | ✅ | gdelt (get_gdelt_sentiment) |
| News coverage volume trend | ✅ | gdelt (get_gdelt_coverage_volume) |
| Top headlines (US) | ✅ | newsapi (disabled by default) |
| Low-latency / real-time news push | ⚠️ | benzinga (pull-based REST; websocket push not yet wired) |
| Local article archive (Economist, Reuters) | ✅ | articles |
Options Data
| Data type | Status | Source |
|---|---|---|
| Options flow / unusual prints | ✅ | uw (ext_get_options_flow) |
| Options sweeps | ✅ | uw (ext_get_options_flow — large prints) |
| Dark pool prints | ✅ | uw (ext_get_dark_pool) — per-ticker only; no market-wide aggregate |
| Put/call ratio | ✅ | uw (ext_get_options_screener, ext_get_net_premium) |
| Market tide (net call vs put premium) | ✅ | uw (ext_get_net_premium) |
| Open interest (flow level) | ✅ | uw (ext_get_oi_changes) |
| Open interest snapshot (full chain) | ⚠️ | OI available in flow/chain data; no full chain OI snapshot by expiry/strike |
| Implied volatility (per trade) | ✅ | uw (in ext_get_options_flow records; IV rank via ext_get_net_premium) |
| IV rank / IV percentile | ⚠️ | eodhd (eodhd_hist_volatility) — HV rank proxy; IV rank needs options add-on |
| IV term structure / skew | ❌ | Options add-on not on current EODHD plan |
| Delta / gamma per position | ⚠️ | Per-trade delta in flow; gamma-exposure (GEX) summary via uw (ext_get_net_premium) |
| Net premium ticks (intraday) | ⚠️ | uw (ext_get_net_premium) — net-premium summary only; no intraday tick series |
| Options screener | ✅ | finviz (options_screen), uw (ext_get_options_screener) |
| Congress trades | ✅ | fmp |
Short Interest & Borrow
| Data type | Status | Source |
|---|---|---|
| Cost to borrow (daily series) | ✅ | iborrowdesk |
| Shares available to borrow | ✅ | iborrowdesk |
| Borrow utilization signal | ✅ | iborrowdesk (High/Medium/Low from fee rate) |
| Daily short volume ratio | ✅ | finra_short (finra_short_volume) — T+1, free |
| Official biweekly short interest | ✅ | finra_short (finra_short_interest) — FINRA REGSHO, biweekly |
| Short interest % of float (screener) | ⚠️ | finviz (screener field, delayed snapshot) |
| Short interest time series | ⚠️ | finra_short gives biweekly history; no daily trend |
| Near-live short interest estimates | ❌ | Requires ORTEX or S3 Partners (paid) |
| Failure-to-deliver (FTD) data | ✅ | sec_ftd (sec_ftd_get, sec_ftd_trend) — SEC REGSHO bimonthly files |
Institutional & Insider Activity
| Data type | Status | Source |
|---|---|---|
| 13F holdings (current quarter) | ✅ | 13f (SEC EDGAR, 45-day lag inherent) |
| 13F new/added/reduced/closed positions | ✅ | 13f (manager_new_positions) |
| Per-position type (long vs put vs call) | ✅ | generate_13f_positions.py script (CSV output) |
| Insider buying / selling (Form 4) | ✅ | fmp (get_insider_trades), sec, finviz |
| Insider transaction detail (role, size, type) | ✅ | fmp |
| 13D/13G activist filings | ⚠️ | sec (search_sec_filings) — unstructured text search; no clean API |
| Institutional ownership % | ✅ | fmp (get_company_profile), finviz |
| Fund ownership changes (quarter-over-quarter) | ✅ | 13f (via comparison of current vs prior quarter) |
| Fund-level drill-down (per manager, all holdings) | ✅ | 13f (generate_13f_positions.py CSV) |
ETF & Fund Flows
| Data type | Status | Source |
|---|---|---|
| ETF fund flow (daily/weekly/monthly/YTD) | ❌ | No working source — ETF.com scrape returns HTTP 403 (etf_flows retired); FMP/EODHD don't publish ETF flows/AUM history |
| Top ETFs by inflow/outflow | ❌ | Same — no flows source |
| ETF holdings / sector weightings | ⚠️ | fmp (requires Ultimate plan for full flows; basic holdings available) |
| Sector ETF relative performance | ✅ | fmp, finviz (group_performance) |
| Leveraged ETF flows | ❌ | Not available |
Technical Indicators
| Data type | Status | Source |
|---|---|---|
| EMA (9/12/21/50/200) | ✅ | fmp (get_technical_indicator) |
| SMA (50/200) | ✅ | fmp |
 RSI |
✅ | fmp |
 ATR |
✅ | fmp |
| ADX / DMI | ✅ | fmp |
| Williams %R | ✅ | fmp |
| Standard deviation | ✅ | fmp |
 MACD |
⚠️ | Derivable from FMP EMAs — no direct MACD endpoint |
| VWAP (intraday) | ⚠️ | Daily VWAP available in polygon_aggregates; intraday not available |
What Is Still Needed
These are the remaining gaps that would meaningfully improve report quality for an analysis platform, ranked by impact. Items irrelevant to report generation (real-time streaming, execution APIs, L2 order book) are excluded.
Priority 1 — Remaining high impact gaps
True IV rank / term structure — EODHD Options add-on or ORATS (~$40–70/month)
The current EODHD plan provides historical (realised) volatility rank via eodhd_hist_volatility — a useful proxy, but not true implied volatility rank. IV rank and IV term structure require either upgrading the EODHD subscription to include the Options add-on, or subscribing to ORATS Core (~$40/month). This is the most impactful remaining gap for options analysis sections.
Earnings call transcripts — any provider
Currently the system can identify earnings dates, beats/misses, and revisions, but cannot read what management actually said. Transcripts would dramatically improve the qualitative section of equity reports. We now hold a Seeking Alpha Premium subscription (which includes transcripts), so the cheapest path is to integrate it via cookie-authenticated scraping — the same pattern the articles module already uses — at no extra subscription cost. Alternatives: Intrinio (API, ~$50/month). The articles module infrastructure already exists for this pattern.
Priority 2 — Nice to have
sec-api.io for cleaner Form 4 / 13D feeds (~$50–150/month)
The current data_ingestion_sec.py hand-rolls XML parsing of EDGAR filings, which is brittle against format variations. sec-api.io delivers Form 4 insider transactions, 13D/13G activist filings, and all SEC filings as clean structured JSON. For an analysis platform the real-time aspect is secondary — the structured clean data is the value.
ORTEX for near-live short interest (~$50–100/month)
finra_short covers official biweekly short interest and daily short volume ratio. ORTEX would add near-live short interest % of float as a time series. Useful for more precise squeeze analysis but FINRA covers the essentials.
~~Benzinga API~~ — now integrated (benzinga module). Covers news, analyst ratings, earnings calendar, dividends, and guidance.
Summary
| Category | Status |
|---|---|
| Price / OHLC history | ✅ Complete |
| Macro series & calendar | ✅ Complete (FRED multi-week calendar + BLS + BEA) |
| Fundamentals & valuation | ✅ Complete |
| Earnings & analyst estimates | ✅ Complete |
| News & sentiment | ✅ Complete (pull-based; Benzinga adds structured finance events) |
| Options flow & dark pool | ✅ Complete |
| Short interest & borrow | ✅ Complete (official FINRA + iborrowdesk) |
| Institutional 13F holdings | ✅ Complete |
| Insider trading | ✅ Complete |
| ETF fund flows | ❌ Gap — ETF.com scrape now HTTP 403 (etf_flows retired); no working source |
| Technical indicators | ✅ Complete |
| HV rank (realised vol proxy) | ✅ eodhd (eodhd_hist_volatility) — HV rank/percentile from EOD prices |
| IV rank / term structure (true IV) | ⚠️ Partial — requires EODHD Options add-on or ORATS |
| Earnings call transcripts | ❌ Gap — Seeking Alpha Premium now owned; integrate via cookie-auth scrape (no extra cost) or Intrinio API |
| Multi-week macro calendar | ✅ macro_calendar (macro_calendar_upcoming) — FRED release dates |
| Failure-to-deliver data | ✅ sec_ftd (sec_ftd_get, sec_ftd_trend) |
| Near-live short interest % of float | ⚠️ Partial — FINRA biweekly covers it; ORTEX for near-live |
| Structured insider/activist filings | ⚠️ Partial — EDGAR works; sec-api.io cleaner |