Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve error messages #58

Open
bhou opened this issue Jul 22, 2019 · 3 comments
Open

Improve error messages #58

bhou opened this issue Jul 22, 2019 · 3 comments
Labels
enhancement New feature or request
Projects

Comments

@bhou
Copy link
Contributor

bhou commented Jul 22, 2019

In some cases, java exceptions are displayed directly in error message notification. Need to provide more user-friendly messages.

@millboh
Copy link
Collaborator

millboh commented Aug 23, 2019

We need to display the amount of skipped lines due to errors and the n first errors. An advanced parameters in all steps should be "max_errors". Technically speaking, we already have an error_handler in most steps. We can refactor it in more abstract classes to handle errors with this advanced setting. By default i suggest max_errors should set to 0 .

@millboh millboh added the enhancement New feature or request label Aug 23, 2019
@millboh millboh added this to To do in 0.11.0 via automation Aug 23, 2019
@bhou
Copy link
Contributor Author

bhou commented Aug 24, 2019

@millboh Putting a max_errors for each step sounds like a bit overkill. We can add a global settings max_errors, and pass it in dag request. For the default value, 0 seems too strict, and it will break some existing projects.

@millboh
Copy link
Collaborator

millboh commented Aug 25, 2019

@bhou I suggested max_errors to be step specific because it actually depends on the volume of data processed. For an excel source 0 might fit but if you process TB of data you'll need a different max_error value.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
0.11.0
  
To do
Development

No branches or pull requests

2 participants