Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: multiple subscriptions in DataQueueHandler #32

Conversation

Romazes
Copy link
Contributor

@Romazes Romazes commented Oct 15, 2024

Description

Problem

When attempting to subscribe to a large number of symbols (over 1,000), an exception was raised: url is too long.

Solution

This PR addresses the issue by dividing the symbol subscriptions into smaller chunks of 100 symbols each. Each chunk is handled as a separate task, allowing efficient updates from TradeStation without exceeding URL length limits.

Related Issue

closes #31

Motivation and Context

The goal is to enable users to subscribe to a large number of symbols for market analysis based on the returned data. This will help them create more effective algorithms in Lean by providing broader market coverage.

Requires Documentation Change

N/A

How Has This Been Tested?

Test Lean Algrotihm #1
public class BasicTemplateAlgorithm : QCAlgorithm, IRegressionAlgorithmDefinition
{
    Dictionary<Symbol, int> _amountBySymbol = new();
    public override void Initialize()
    {
        AddOption("QQQ").SetFilter(o => o.IncludeWeeklys().Strikes(-1000, -5).Expiration(1, 10));
    }
    public override void OnData(Slice slice)
    {
        foreach (var item in slice)
        {
            _amountBySymbol[item.Key] = _amountBySymbol.GetValueOrDefault(item.Key, 0) + 1;
        }
    }
    public override void OnEndOfAlgorithm()
    {
        Log(new string('-', 10));
        Log($"AMOUNT OF SYMBOLS RECEIVED: {_amountBySymbol.Count}");
        Log(new string('-', 10));
    }
}

Result:
image

Test Lean Algrotihm #2
public override void Initialize()
{
    QQQ = AddOption("QQQ");
    QQQ.SetFilter(o => o.IncludeWeeklys().Strikes(-1000, -5).Expiration(1, 10));
    _addNewOption = Time.AddSeconds(30);
    _removeOldOption = Time.AddMinutes(1);
}
public override void OnData(Slice slice)
{
    foreach (var item in slice)
    {
        _amountBySymbol[item.Key] = _amountBySymbol.GetValueOrDefault(item.Key, 0) + 1;
    }
    if (Time > _addNewOption)
    {
        _addNewOption = _addNewOption.AddMinutes(10);
        AddOption("AAPL").SetFilter(o => o.IncludeWeeklys().Strikes(-100, 300).Expiration(1, 10));
    }
    if (Time > _removeOldOption)
    {
        _removeOldOption = _removeOldOption.AddMinutes(10);
        var optionChain = OptionChain(QQQ.Symbol);
        foreach (var optionContract in optionChain)
        {
            RemoveOptionContract(optionContract.Symbol);
        }
        RemoveSecurity(QQQ.Symbol);
    }
}
Test Algorithm #3

Testing with Algo from #31. We can see that prices are changed.
image

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • Refactor (non-breaking change which improves implementation)
  • Performance (non-breaking change which improves performance. Please add associated performance test and results)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Non-functional change (xml comments/documentation/etc)

Checklist:

  • My code follows the code style of this project.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.
  • My branch follows the naming convention bug-<issue#>-<description> or feature-<issue#>-<description>

@Romazes Romazes added the bug Something isn't working label Oct 15, 2024
@Romazes Romazes self-assigned this Oct 15, 2024
feat: handle OperationCanceledException
feat: StreamTaskManager with stream functionally
feat: TradeStation Multiple Subscription Manager to subscribe on multiple symbols
remove: handle response in DQH partial class
feat: dev friendly comment why skip OperationCanceledException
…onItem

refactor: dispose in StreamTaskManager
feat: new log about remove subscriptionTask in TradeStationBrokerageMultiStreamSubscriptionManager
feat: add lock to all operation with hash collection in StreamingTaskManager
remove: internall list in task
refactor: RemoveSubscriptionItem in StreamingTaskManager
refactor: UnSubscribe in TradeStationBrokerageMultiStreamSubscriptionManager
Copy link
Member

@Martin-Molinero Martin-Molinero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! 👍

@Martin-Molinero Martin-Molinero merged commit b590eed into QuantConnect:master Oct 21, 2024
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Persistent Connection Lost with Options
2 participants