-
Notifications
You must be signed in to change notification settings - Fork 39
Issues: Lightning-AI/litdata
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Author
Label
Projects
Milestones
Assignee
Sort
Issues list
Existing Cache files leads to permanent DataLoader hang
bug
Something isn't working
help wanted
Extra attention is needed
#398
opened Oct 16, 2024 by
lilavocado
Combine Small StreamingDatasets into 1 Large StreamingDataset
enhancement
New feature or request
#396
opened Oct 11, 2024 by
schopra8
Improve CombinedStreamingDataset to handle multiple subdatasets efficiently
enhancement
New feature or request
#386
opened Oct 2, 2024 by
bhimrazy
The config isn't consistent between chunks
bug
Something isn't working
help wanted
Extra attention is needed
#370
opened Sep 17, 2024 by
AugustDev
How can I shut down automatically distributing data when using StreamingDataset?
enhancement
New feature or request
question
Further information is requested
#368
opened Sep 12, 2024 by
ygtxr1997
Failed to Resume Training w/ CombinedStreamingDataset
bug
Something isn't working
duplicate
This issue or pull request already exists
help wanted
Extra attention is needed
#363
opened Sep 5, 2024 by
schopra8
CombinedStreamingDataset causes NCCL timeout when using multiple nodes
bug
Something isn't working
help wanted
Extra attention is needed
#340
opened Aug 26, 2024 by
hubenjm
Lazyload subsamples if subsample=1.0
enhancement
New feature or request
question
Further information is requested
#339
opened Aug 21, 2024 by
deependujha
StreamingDataset intermittently fails due to lack of index.json
bug
Something isn't working
help wanted
Extra attention is needed
#337
opened Aug 20, 2024 by
plra
Bug: Inconsistent Behavior with StreamingDataloader loading states (specific to CombinedStreamingDataset)
bug
Something isn't working
help wanted
Extra attention is needed
#331
opened Aug 14, 2024 by
bhimrazy
Use different batch sizes in CombinedStreamingDataset
enhancement
New feature or request
help wanted
Extra attention is needed
#327
opened Aug 10, 2024 by
schopra8
Add support for multi sample item in optimize and yielding from the _getitem_ of the StreamingDataset
enhancement
New feature or request
help wanted
Extra attention is needed
#317
opened Aug 8, 2024 by
tchaton
Explore about integrating homomorphic encryption
enhancement
New feature or request
help wanted
Extra attention is needed
#313
opened Aug 7, 2024 by
bhimrazy
Investigate keeping the content of the downloaded chunks in RAM instead of writing it to file.
enhancement
New feature or request
help wanted
Extra attention is needed
#291
opened Aug 1, 2024 by
tchaton
Add training mode compression for zstd
enhancement
New feature or request
help wanted
Extra attention is needed
#283
opened Jul 31, 2024 by
tchaton
Add support for sample windowing
enhancement
New feature or request
help wanted
Extra attention is needed
#282
opened Jul 31, 2024 by
tchaton
RuntimeError: Can't start new thread
bug
Something isn't working
help wanted
Extra attention is needed
#280
opened Jul 31, 2024 by
cgebbe
If data samples contain a Python list with a variable number of elements, type inference will fail
bug
Something isn't working
help wanted
Extra attention is needed
#260
opened Jul 23, 2024 by
senarvi
Add example of saving / resuming DataLoader state with PyTorch Lightning
documentation
Improvements or additions to documentation
#249
opened Jul 21, 2024 by
schopra8
Profiler patches torch without reverting it
bug
Something isn't working
help wanted
Extra attention is needed
#240
opened Jul 17, 2024 by
awaelchli
Integration with DAG framework Prefect
enhancement
New feature or request
help wanted
Extra attention is needed
#226
opened Jul 12, 2024 by
tchaton
Add support for the reduce operator
enhancement
New feature or request
help wanted
Extra attention is needed
#225
opened Jul 12, 2024 by
tchaton
Using a streaming dataloader with an unbalanced dataset yields unexpected batch sizes.
bug
Something isn't working
help wanted
Extra attention is needed
#199
opened Jun 29, 2024 by
esivonxay-cognitiv
Add support for parquet files for storing the chunks
enhancement
New feature or request
help wanted
Extra attention is needed
#191
opened Jun 27, 2024 by
tchaton
LitData doesn't support s3 bucket connection outside server
enhancement
New feature or request
help wanted
Extra attention is needed
#183
opened Jun 25, 2024 by
sanyalsunny111
Previous Next
ProTip!
Add no:assignee to see everything that’s not assigned.