Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: After enabling dynamic schema, if there is no $meta field, the import of Parquet file format will fail. #32729

Closed
1 task done
zhuwenxing opened this issue Apr 30, 2024 · 3 comments
Closed
1 task done

[Bug]: After enabling dynamic schema, if there is no $meta field, the import of Parquet file format will fail. #32729

zhuwenxing opened this issue Apr 30, 2024 · 3 comments
Assignees
@zhuwenxing
Labels
kind/bug Issues or changes related a bug triage/accepted Indicates an issue or PR is ready to be actively worked on.
Milestone
2.4.2

Comments

@zhuwenxing
Copy link
Contributor

Is there an existing issue for this?

  • I have searched the existing issues

Environment

- Milvus version:master
- Deployment mode(standalone or cluster):
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

Even if the dynamic schema is enabled, users may not have dynamic columns in their data, so $meta can be optional.

Expected Behavior

failed_reason': "schema not equal, err=field '$meta' not in arrow schema: importing data failed: importing data failed", 'progress_percent': '0'}

Steps To Reproduce

No response

Milvus Log

failed ci job:
https://jenkins.milvus.io:18080/blue/organizations/jenkins/Milvus%20HA%20CI/detail/PR-32706/3/pipeline/

Anything else?

No response
@zhuwenxing zhuwenxing added kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Apr 30, 2024
@zhuwenxing
Copy link
Contributor Author

/assign @bigsheeper

@bigsheeper bigsheeper mentioned this issue Apr 30, 2024
Merged
@yanliang567
Copy link
Contributor

/unassign

@yanliang567 yanliang567 added this to the 2.4.1 milestone Apr 30, 2024
@yanliang567 yanliang567 added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Apr 30, 2024
sre-ci-robot pushed a commit that referenced this issue May 7, 2024
@bigsheeper
fix: Make the dynamic column optional in parquet import (#32738)
4de063a 
issue: #32729

Signed-off-by: bigsheeper <[email protected]>
bigsheeper added a commit to bigsheeper/milvus that referenced this issue May 7, 2024
@bigsheeper
fix: Make the dynamic column optional in parquet import (milvus-io#32738
1d55334 
)

issue: milvus-io#32729

Signed-off-by: bigsheeper <[email protected]>
@bigsheeper bigsheeper mentioned this issue May 7, 2024
Merged
@yanliang567 yanliang567 modified the milestones: 2.4.1, 2.4.2 May 7, 2024
sre-ci-robot pushed a commit that referenced this issue May 7, 2024
@bigsheeper
fix: Make the dynamic column optional in parquet import (#32738) (#32802
54e00db 
)

issue: #32729

pr: #32738

Signed-off-by: bigsheeper <[email protected]>
@bigsheeper
Copy link
Contributor

/assign @zhuwenxing
/unassign

please help to verify

@sre-ci-robot sre-ci-robot assigned zhuwenxing and unassigned bigsheeper May 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@zhuwenxing zhuwenxing

Labels
kind/bug Issues or changes related a bug triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Milestone
2.4.2
Development

No branches or pull requests
3 participants
@zhuwenxing @bigsheeper @yanliang567