Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ingest - Longer than the specified 1500 #14

Open
xingfanxia opened this issue Mar 22, 2023 · 1 comment
Open

Ingest - Longer than the specified 1500 #14

xingfanxia opened this issue Mar 22, 2023 · 1 comment

Comments

@xingfanxia
Copy link

Is this a problem with ingesting?

❯ python3 ingest.py                                                                                                                                                              ─╯
Created a chunk of size 1738, which is longer than the specified 1500
Created a chunk of size 1698, which is longer than the specified 1500
Created a chunk of size 1568, which is longer than the specified 1500
Created a chunk of size 2845, which is longer than the specified 1500
Created a chunk of size 2181, which is longer than the specified 1500
Created a chunk of size 2042, which is longer than the specified 1500
Created a chunk of size 2387, which is longer than the specified 1500
Created a chunk of size 2305, which is longer than the specified 1500
Created a chunk of size 2076, which is longer than the specified 1500
Created a chunk of size 2316, which is longer than the specified 1500
Created a chunk of size 2400, which is longer than the specified 1500
Created a chunk of size 1520, which is longer than the specified 1500
Created a chunk of size 1718, which is longer than the specified 1500
Created a chunk of size 2526, which is longer than the specified 1500
Created a chunk of size 1874, which is longer than the specified 1500
Created a chunk of size 2946, which is longer than the specified 1500
Created a chunk of size 2362, which is longer than the specified 1500
Created a chunk of size 1753, which is longer than the specified 1500
Created a chunk of size 2821, which is longer than the specified 1500
Created a chunk of size 1579, which is longer than the specified 1500
Created a chunk of size 1824, which is longer than the specified 1500
Created a chunk of size 2026, which is longer than the specified 1500
Created a chunk of size 2981, which is longer than the specified 1500
Created a chunk of size 2865, which is longer than the specified 1500
Created a chunk of size 97940, which is longer than the specified 1500
Created a chunk of size 53894, which is longer than the specified 1500
Created a chunk of size 1821, which is longer than the specified 1500
Created a chunk of size 2861, which is longer than the specified 1500
Created a chunk of size 5088, which is longer than the specified 1500
Created a chunk of size 1865, which is longer than the specified 1500
Created a chunk of size 1538, which is longer than the specified 1500
Created a chunk of size 1542, which is longer than the specified 1500
Created a chunk of size 2046, which is longer than the specified 1500
Created a chunk of size 2558, which is longer than the specified 1500
Created a chunk of size 1882, which is longer than the specified 1500
Created a chunk of size 13716, which is longer than the specified 1500
Created a chunk of size 2183, which is longer than the specified 1500
Created a chunk of size 3526, which is longer than the specified 1500
Created a chunk of size 1702, which is longer than the specified 1500
Created a chunk of size 2313, which is longer than the specified 1500
Created a chunk of size 1775, which is longer than the specified 1500
Created a chunk of size 5809, which is longer than the specified 1500
Created a chunk of size 4280, which is longer than the specified 1500
Created a chunk of size 3446, which is longer than the specified 1500
Created a chunk of size 2207, which is longer than the specified 1500
Created a chunk of size 2502, which is longer than the specified 1500
Created a chunk of size 2465, which is longer than the specified 1500
Created a chunk of size 16323, which is longer than the specified 1500
Created a chunk of size 2277, which is longer than the specified 1500
Created a chunk of size 8885, which is longer than the specified 1500
Created a chunk of size 3056, which is longer than the specified 1500
Created a chunk of size 14512, which is longer than the specified 1500
Created a chunk of size 2896, which is longer than the specified 1500
Created a chunk of size 10896, which is longer than the specified 1500
Created a chunk of size 1504, which is longer than the specified 1500
Created a chunk of size 3937, which is longer than the specified 1500
Created a chunk of size 1643, which is longer than the specified 1500
Created a chunk of size 2604, which is longer than the specified 1500
Created a chunk of size 4257, which is longer than the specified 1500
Created a chunk of size 2635, which is longer than the specified 1500
Created a chunk of size 1907, which is longer than the specified 1500
Created a chunk of size 2010, which is longer than the specified 1500
Created a chunk of size 1832, which is longer than the specified 1500
Created a chunk of size 1517, which is longer than the specified 1500
Created a chunk of size 2795, which is longer than the specified 1500
Created a chunk of size 1795, which is longer than the specified 1500
Created a chunk of size 1862, which is longer than the specified 1500
Created a chunk of size 2764, which is longer than the specified 1500
Created a chunk of size 1596, which is longer than the specified 1500
Created a chunk of size 3146, which is longer than the specified 1500
Created a chunk of size 1655, which is longer than the specified 1500
Created a chunk of size 1563, which is longer than the specified 1500
Created a chunk of size 1547, which is longer than the specified 1500
Created a chunk of size 1570, which is longer than the specified 1500
Created a chunk of size 1688, which is longer than the specified 1500
Created a chunk of size 1626, which is longer than the specified 1500
Created a chunk of size 2224, which is longer than the specified 1500
Created a chunk of size 3120, which is longer than the specified 1500
Created a chunk of size 2197, which is longer than the specified 1500
@lmsanch
Copy link

lmsanch commented Apr 18, 2023

I am having the same issue. Not sure if I need to specify a different chunk size in the code and/or if this affects performance, etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants