Problem Description
Solution
To get historical data for a futures contract that accounts for rollovers (often called "Continuous Futures"), you must configure the DataNormalizationMode and DataMappingMode when adding the future subscription.
In QuantConnect, the "Canonical Symbol" (the symbol returned by add_future) represents the continuous contract. When you request history on this symbol, the LEAN engine automatically stitches together data from individual expiring contracts based on your specified mapping and normalization settings.
Key Concepts
- DataNormalizationMode.BACK_ADJUSTED: This is the most common setting for backtesting. It adjusts historical prices of previous contracts to eliminate the "gap" that occurs when rolling from one contract to another (due to the carry cost/premium). This creates a smooth price series for indicators.
- DataMappingMode: This determines when the system rolls to the next contract (e.g., based on Open Interest or the Last Trading Day).
Implementation Example
The following algorithm demonstrates how to configure a Gold Future (GC) to be continuous and how to request the stitched historical data.
# region imports
from AlgorithmImports import *
# endregion
class ContinuousFuturesHistoryAlgorithm(QCAlgorithm):
def initialize(self):
self.set_start_date(2023, 1, 1)
self.set_end_date(2024, 1, 1)
self.set_cash(100000)
# 1. Add the Future
# We use the canonical symbol (Futures.Metals.GOLD)
future = self.add_future(
Futures.Metals.GOLD,
resolution=Resolution.DAILY,
extended_market_hours=True
)
# 2. Configure the Filter
# We need to ensure the universe has contracts to map.
# This selects contracts expiring within 0 to 182 days.
future.set_filter(0, 182)
# 3. Set Data Normalization (Crucial for "Accounting for Rollovers")
# BACK_ADJUSTED eliminates price jumps at rollovers, ideal for TA.
# Use RAW if you want actual trade prices (but you will see gaps).
future.data_normalization_mode = DataNormalizationMode.BACK_ADJUSTED
# 4. Set Data Mapping
# OPEN_INTEREST is the standard for liquidity; it rolls when the next
# contract has higher open interest than the current one.
future.data_mapping_mode = DataMappingMode.OPEN_INTEREST
self._continuous_symbol = future.symbol
def on_end_of_algorithm(self):
# 5. Request History
# We request history on the self._continuous_symbol (the Canonical Symbol).
# LEAN will return a DataFrame stitched together based on the settings above.
history_df = self.history(
self._continuous_symbol,
start=self.start_date,
end=self.end_date,
resolution=Resolution.DAILY
)
if not history_df.empty:
# Log the first 5 and last 5 rows to verify data range
self.log(f"History Head:\n{history_df.head()}")
self.log(f"History Tail:\n{history_df.tail()}")
# Example: Accessing the close column
# The index will be (Symbol, Time), so we might need to unstack or loc
closes = history_df['close']
self.log(f"Average Close Price: {closes.mean()}")
else:
self.log("No history found.")
Explanation of Settings
future.set_filter(0, 182): This is required. The engine needs a pool of underlying contracts to choose from. If you don't set a filter, the engine won't load the specific contracts required to build the continuous history.DataNormalizationMode.BACK_ADJUSTED:- Without this: If the current contract is $2000 and the next is $2010, the chart would jump $10 instantly on the rollover date.
- With this: The system works backward, subtracting that $10 difference from all historical data points, ensuring the transition is seamless for moving averages and other indicators.
DataMappingMode.OPEN_INTEREST: This ensures you are always looking at the most liquid contract. Other options includeLAST_TRADING_DAYorFIRST_DAY_MONTH.
Q&A
Q: Can I get the specific contract symbol that was used for a specific date in the history?
A: Yes. When you request self.history on a canonical symbol, the resulting DataFrame usually has a MultiIndex. However, for continuous futures, the index is the Canonical Symbol. To see the underlying mapped contract, you can look at the SymbolChangedEvents in a backtest, or request self.history on the specific contracts if you know them. In the slice object during on_data, slice.symbol_changed_events will notify you when the continuous contract rolls to a new underlying ticker.
Q: What if I want the actual execution prices but still want a continuous stream?
A: Set future.data_normalization_mode = DataNormalizationMode.RAW. This will stitch the data together based on your mapping mode (e.g., Open Interest), but it will not adjust the prices. You will see price gaps on the days the contract rolls. This is preferred for PnL calculations but bad for technical indicators.
Q: Why is my history empty?
A: This usually happens if set_filter is not configured or is too restrictive (e.g., looking for contracts expiring in 0 days). Ensure the filter window is wide enough to find the next contract (e.g., 0, 182 days).
Q: Does this work for intraday data?
A: Yes. Change Resolution.DAILY to Resolution.MINUTE or Resolution.HOUR in both the add_future call and the history call. Note that requesting long periods of minute data takes significantly longer to process.