fix client.insert_dataframe() for tables with tuple columns with acceptance of numpy.ndarrays
#426
Conversation
|
@xzkostyan , does it look better now? |
stankudrow
changed the title
numpy.ndarrays
|
@xzkostyan , the |
| CHECK_NUMPY_TYPES = False | ||
|
|
||
|
|
||
| def _check_sequence_to_be_an_expected_iterable(seq): |
There was a problem hiding this comment.
Numpy/pandas helpers should be located in clickhouse_driver/numpy/helpers.py.
I'd keep two different versions of chunks: one for non-numpy and another one for numpy.
| column_values = dataframe[column].values | ||
| for idx, col_vals in enumerate(column_values): | ||
| if isinstance(col_vals, dict): | ||
| column_values[idx] = tuple(col_vals.values()) |
There was a problem hiding this comment.
What about speed and memory consumption here? Did you test the new code with large dataframes?
There was a problem hiding this comment.
This part is to be refined later.
stankudrow
force-pushed
the
fix-client-insert-dataframe-with-dicts
branch
from
9bf3ad8
to
7f0cc87
Compare
| from tests.numpy.testcase import NumpyBaseTestCase | ||
|
|
||
|
|
||
| @skipIf(not PANDAS_IMPORTED, reason="pandas cannot be imported") |
There was a problem hiding this comment.
Please see how simple numpy/pandas tests are implemented: tests/numpy/test_generic.py. Skipping logic (@skipIf) is hidden under the hood. Variable PANDAS_IMPORTED is redundant in this case.
fixes
reconsiders the PR fix
client.insert_dataframe()for tuple columns in tables #425Packages:
pytest.mark.parametrize)Checklist:
flake8and fix issues;pytestno tests failed. See the dev docs.