Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak when clickhouse raise error on data send. #406

Closed
pulina opened this issue Dec 12, 2023 · 0 comments · Fixed by #407
Closed

Memory leak when clickhouse raise error on data send. #406

pulina opened this issue Dec 12, 2023 · 0 comments · Fixed by #407

Comments

@pulina
Copy link
Contributor

pulina commented Dec 12, 2023

Describe the bug
When clickhouse rise error when sending for column Array(FixedString(16)) memory is not freed. Note that this problem occurs only for not compressed connection and is correlated with #405

To Reproduce

#!/usr/bin/env python

from clickhouse_driver import Client
import tracemalloc

DATABASE = "***"
USER = "***"
PASSWORD = "***"
HOST = "***"
TABLE = "test_table"
DROP_TABLE_QUERY = f"""
DROP TABLE IF EXISTS `{DATABASE}`.{TABLE}
"""
CREATE_TABLE_QUERY = f"""
CREATE TABLE `{DATABASE}`.{TABLE}
                (
                    `col1` Array(FixedString(16))
                )
                ENGINE = MergeTree
                ORDER BY (col1)

"""
INSERT_QUERY = f"""
INSERT INTO `{DATABASE}`.{TABLE} (col1) VALUES
"""


tracemalloc.start()

data = [{"col1": [b'\r\xf7\x9c\xa1\xd7\xe4]\xee\x15\xeer["\xed^\xcc'] * 10000}] * 10000
try:
    with Client(
        host=HOST,
        port=39000,
        user=USER,
        password=PASSWORD,
        database=DATABASE,
        # compression="zstd", # uncomment to spot the difference
    ) as client:
        client.execute(DROP_TABLE_QUERY)
        client.execute(CREATE_TABLE_QUERY)
        client.execute(INSERT_QUERY, data)
except:
    raise
finally:
    print(
        "Memory usage, Pick memory usage ",
        tracemalloc.get_traced_memory()[0] / 1024 / 1024,
        tracemalloc.get_traced_memory()[1] / 1024 / 1024,
    )

Expected behavior
Memory should be freed regardless of error.

Versions

Versions

Verivifed for current master and 0.2.6
CH version: 23.3.2 (but is probably irrelevant)
Python 3.10.12

@pulina pulina changed the title Memory leek when clickhouse raise error on data send. Memory leak when clickhouse raise error on data send. Dec 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant