Changes in celery after update to release 1.0.4

[legumbot]

I made use of the update_local.sh script to update my release 1.0.3 installation to 1.0.4.
After the script was completed I can log into the web applications, but failed to run the basic tasks, like the "LoadTerms" task. It will just sit in the queue with a pending status.

I noticed that in updating to release 1.0.4, the way celery is started becomes different from 1.0.3. Previously the way to start was:
celery multi start 2 -A apps.task -f celery.log -l info -B

but now in 1.0.4 it became:
celery multi start 2 -A apps -f celery.log -l info -B

Is the changes in removing reference to task.py intended and right? Documentation in 1.0.4 is not updated yet and still refer to "apps.task" (pg 7 of the Admin Guide manual) Checking on "registered tasks" and "active tasks" returned failure status led me to believe that the issue has to do with celery. Please see error log below.

If that is not the issue, can you please help to advice what to look for if an admin task remains pending with 0% progress? I was able to complete this task before upgrading to 1.0.4.

(ve) contraxsuite@llb-ideacentre-510S-08IKL:/contraxsuite/lexpredict-contraxsuite/contraxsuite_services$ celery inspect registered -A apps
Traceback (most recent call last):
File "/contraxsuite/ve/bin/celery", line 11, in
sys.exit(main())
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/main.py", line 14, in main
_main()
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/celery.py", line 326, in main
cmd.execute_from_commandline(argv)
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/celery.py", line 488, in execute_from_commandline
super(CeleryCommand, self).execute_from_commandline(argv)))
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/base.py", line 281, in execute_from_commandline
return self.handle_argv(self.prog_name, argv[1:])
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/celery.py", line 480, in handle_argv
return self.execute(command, argv)
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/celery.py", line 412, in execute
).run_from_argv(self.prog_name, argv[1:], command=argv[0])
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/base.py", line 285, in run_from_argv
sys.argv if argv is None else argv, command)
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/base.py", line 367, in handle_argv
*self.parse_options(prog_name, argv, command))
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/base.py", line 403, in parse_options
self.parser = self.create_parser(prog_name, command)
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/base.py", line 413, in create_parser
epilog=self._format_epilog(self.epilog),
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/control.py", line 167, in epilog
self.list_commands(indent=4, help=True, app=self.app)
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/control.py", line 56, in list_commands
choices = cls._choices_by_group(app)
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/control.py", line 152, in _choices_by_group
app.loader.import_default_modules()
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/loaders/base.py", line 111, in import_default_modules
return [self.import_task_module(m) for m in self.default_modules]
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/loaders/base.py", line 111, in
return [self.import_task_module(m) for m in self.default_modules]
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/loaders/base.py", line 97, in import_task_module
return self.import_from_cwd(module)
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/loaders/base.py", line 106, in import_from_cwd
package=package,
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/utils/imports.py", line 101, in import_from_cwd
return imp(module, package=package)
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/loaders/base.py", line 100, in import_module
return importlib.import_module(module, package=package)
File "/contraxsuite/ve/lib/python3.5/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 986, in _gcd_import
File "", line 969, in _find_and_load
File "", line 958, in _find_and_load_unlocked
File "", line 673, in _load_unlocked
File "", line 665, in exec_module
File "", line 222, in _call_with_frames_removed
File "/contraxsuite/lexpredict-contraxsuite/contraxsuite_services/apps/task/tasks.py", line 76, in
from tika import parser
File "/contraxsuite/ve/lib/python3.5/site-packages/tika/parser.py", line 19, in
from .tika import parse1, callServer, ServerEndpoint
File "/contraxsuite/ve/lib/python3.5/site-packages/tika/tika.py", line 142, in
fileHandler = logging.FileHandler(log_file)
File "/usr/lib/python3.5/logging/init.py", line 1008, in init
StreamHandler.init(self, self._open())
File "/usr/lib/python3.5/logging/init.py", line 1037, in _open
return open(self.baseFilename, self.mode, encoding=self.encoding)
PermissionError: [Errno 13] Permission denied: '/tmp/tika.log'

(ve) contraxsuite@llb-ideacentre-510S-08IKL:/contraxsuite/lexpredict-contraxsuite/contraxsuite_services$ celery inspect active -A apps
Traceback (most recent call last):
File "/contraxsuite/ve/bin/celery", line 11, in
sys.exit(main())
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/main.py", line 14, in main
_main()
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/celery.py", line 326, in main
cmd.execute_from_commandline(argv)
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/celery.py", line 488, in execute_from_commandline
super(CeleryCommand, self).execute_from_commandline(argv)))
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/base.py", line 281, in execute_from_commandline
return self.handle_argv(self.prog_name, argv[1:])
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/celery.py", line 480, in handle_argv
return self.execute(command, argv)
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/celery.py", line 412, in execute
).run_from_argv(self.prog_name, argv[1:], command=argv[0])
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/base.py", line 285, in run_from_argv
sys.argv if argv is None else argv, command)
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/base.py", line 367, in handle_argv
*self.parse_options(prog_name, argv, command))
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/base.py", line 403, in parse_options
self.parser = self.create_parser(prog_name, command)
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/base.py", line 413, in create_parser
epilog=self._format_epilog(self.epilog),
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/control.py", line 167, in epilog
self.list_commands(indent=4, help=True, app=self.app)
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/control.py", line 56, in list_commands
choices = cls._choices_by_group(app)
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/control.py", line 152, in _choices_by_group
app.loader.import_default_modules()
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/loaders/base.py", line 111, in import_default_modules
return [self.import_task_module(m) for m in self.default_modules]
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/loaders/base.py", line 111, in
return [self.import_task_module(m) for m in self.default_modules]
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/loaders/base.py", line 97, in import_task_module
return self.import_from_cwd(module)
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/loaders/base.py", line 106, in import_from_cwd
package=package,
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/utils/imports.py", line 101, in import_from_cwd
return imp(module, package=package)
File "/contraxsuite/ve/lib/python3.5/site-packages/celery/loaders/base.py", line 100, in import_module
return importlib.import_module(module, package=package)
File "/contraxsuite/ve/lib/python3.5/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 986, in _gcd_import
File "", line 969, in _find_and_load
File "", line 958, in _find_and_load_unlocked
File "", line 673, in _load_unlocked
File "", line 665, in exec_module
File "", line 222, in _call_with_frames_removed
File "/contraxsuite/lexpredict-contraxsuite/contraxsuite_services/apps/task/tasks.py", line 76, in
from tika import parser
File "/contraxsuite/ve/lib/python3.5/site-packages/tika/parser.py", line 19, in
from .tika import parse1, callServer, ServerEndpoint
File "/contraxsuite/ve/lib/python3.5/site-packages/tika/tika.py", line 142, in
fileHandler = logging.FileHandler(log_file)
File "/usr/lib/python3.5/logging/init.py", line 1008, in init
StreamHandler.init(self, self._open())
File "/usr/lib/python3.5/logging/init.py", line 1037, in _open
return open(self.baseFilename, self.mode, encoding=self.encoding)
PermissionError: [Errno 13] Permission denied: '/tmp/tika.log'
(ve) contraxsuite@llb-ideacentre-510S-08IKL:/contraxsuite/lexpredict-contraxsuite/contraxsuite_services$

[emichaelsomm]

I have the same issue. From: legumbot [mailto:[email protected]] Sent: Thursday, December 14, 2017 1:59 AM To: LexPredict/lexpredict-contraxsuite <[email protected]> Cc: Subscribed <[email protected]> Subject: [LexPredict/lexpredict-contraxsuite] Changes in celery after update to release 1.0.4 (#16) I made use of the update_local.sh script to update my release 1.0.3 installation to 1.0.4. After the script was completed I can log into the web applications, but failed to run the basic tasks, like the "LoadTerms" task. It will just sit in the queue with a pending status. I noticed that in updating to release 1.0.4, the way celery is started becomes different from 1.0.3. Previously the way to start was: celery multi start 2 -A apps.task -f celery.log -l info -B but now in 1.0.4 it became: celery multi start 2 -A apps -f celery.log -l info -B Is the changes in removing reference to task.py intended and right? Documentation in 1.0.4 is not updated yet and still refer to "apps.task" (pg 7 of the Admin Guide manual) Checking on "registered tasks" and "active tasks" returned failure status led me to believe that the issue has to do with celery. Please see error log below. If that is not the issue, can you please help to advice what to look for if an admin task remains pending with 0% progress? I was able to complete this task before upgrading to 1.0.4. (ve) contraxsuite@llb-ideacentre-510S-08IKL:/contraxsuite/lexpredict-contraxsuite/contraxsuite_services$ celery inspect registered -A apps Traceback (most recent call last): File "/contraxsuite/ve/bin/celery", line 11, in sys.exit(main()) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/main.py", line 14, in main _main() File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/celery.py", line 326, in main cmd.execute_from_commandline(argv) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/celery.py", line 488, in execute_from_commandline super(CeleryCommand, self).execute_from_commandline(argv))) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/base.py", line 281, in execute_from_commandline return self.handle_argv(self.prog_name, argv[1:]) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/celery.py", line 480, in handle_argv return self.execute(command, argv) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/celery.py", line 412, in execute ).run_from_argv(self.prog_name, argv[1:], command=argv[0]) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/base.py", line 285, in run_from_argv sys.argv if argv is None else argv, command) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/base.py", line 367, in handle_argv *self.parse_options(prog_name, argv, command)) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/base.py", line 403, in parse_options self.parser = self.create_parser(prog_name, command) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/base.py", line 413, in create_parser epilog=self._format_epilog(self.epilog), File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/control.py", line 167, in epilog self.list_commands(indent=4, help=True, app=self.app) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/control.py", line 56, in list_commands choices = cls._choices_by_group(app) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/control.py", line 152, in _choices_by_group app.loader.import_default_modules() File "/contraxsuite/ve/lib/python3.5/site-packages/celery/loaders/base.py", line 111, in import_default_modules return [self.import_task_module(m) for m in self.default_modules] File "/contraxsuite/ve/lib/python3.5/site-packages/celery/loaders/base.py", line 111, in return [self.import_task_module(m) for m in self.default_modules] File "/contraxsuite/ve/lib/python3.5/site-packages/celery/loaders/base.py", line 97, in import_task_module return self.import_from_cwd(module) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/loaders/base.py", line 106, in import_from_cwd package=package, File "/contraxsuite/ve/lib/python3.5/site-packages/celery/utils/imports.py", line 101, in import_from_cwd return imp(module, package=package) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/loaders/base.py", line 100, in import_module return importlib.import_module(module, package=package) File "/contraxsuite/ve/lib/python3.5/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 986, in _gcd_import File "", line 969, in _find_and_load File "", line 958, in _find_and_load_unlocked File "", line 673, in _load_unlocked File "", line 665, in exec_module File "", line 222, in _call_with_frames_removed File "/contraxsuite/lexpredict-contraxsuite/contraxsuite_services/apps/task/tasks.py", line 76, in from tika import parser File "/contraxsuite/ve/lib/python3.5/site-packages/tika/parser.py", line 19, in from .tika import parse1, callServer, ServerEndpoint File "/contraxsuite/ve/lib/python3.5/site-packages/tika/tika.py", line 142, in fileHandler = logging.FileHandler(log_file) File "/usr/lib/python3.5/logging/init.py", line 1008, in init StreamHandler.init(self, self._open()) File "/usr/lib/python3.5/logging/init.py", line 1037, in _open return open(self.baseFilename, self.mode, encoding=self.encoding) PermissionError: [Errno 13] Permission denied: '/tmp/tika.log' (ve) contraxsuite@llb-ideacentre-510S-08IKL:/contraxsuite/lexpredict-contraxsuite/contraxsuite_services$ celery inspect active -A apps Traceback (most recent call last): File "/contraxsuite/ve/bin/celery", line 11, in sys.exit(main()) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/main.py", line 14, in main _main() File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/celery.py", line 326, in main cmd.execute_from_commandline(argv) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/celery.py", line 488, in execute_from_commandline super(CeleryCommand, self).execute_from_commandline(argv))) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/base.py", line 281, in execute_from_commandline return self.handle_argv(self.prog_name, argv[1:]) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/celery.py", line 480, in handle_argv return self.execute(command, argv) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/celery.py", line 412, in execute ).run_from_argv(self.prog_name, argv[1:], command=argv[0]) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/base.py", line 285, in run_from_argv sys.argv if argv is None else argv, command) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/base.py", line 367, in handle_argv *self.parse_options(prog_name, argv, command)) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/base.py", line 403, in parse_options self.parser = self.create_parser(prog_name, command) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/base.py", line 413, in create_parser epilog=self._format_epilog(self.epilog), File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/control.py", line 167, in epilog self.list_commands(indent=4, help=True, app=self.app) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/control.py", line 56, in list_commands choices = cls._choices_by_group(app) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/bin/control.py", line 152, in _choices_by_group app.loader.import_default_modules() File "/contraxsuite/ve/lib/python3.5/site-packages/celery/loaders/base.py", line 111, in import_default_modules return [self.import_task_module(m) for m in self.default_modules] File "/contraxsuite/ve/lib/python3.5/site-packages/celery/loaders/base.py", line 111, in return [self.import_task_module(m) for m in self.default_modules] File "/contraxsuite/ve/lib/python3.5/site-packages/celery/loaders/base.py", line 97, in import_task_module return self.import_from_cwd(module) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/loaders/base.py", line 106, in import_from_cwd package=package, File "/contraxsuite/ve/lib/python3.5/site-packages/celery/utils/imports.py", line 101, in import_from_cwd return imp(module, package=package) File "/contraxsuite/ve/lib/python3.5/site-packages/celery/loaders/base.py", line 100, in import_module return importlib.import_module(module, package=package) File "/contraxsuite/ve/lib/python3.5/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 986, in _gcd_import File "", line 969, in _find_and_load File "", line 958, in _find_and_load_unlocked File "", line 673, in _load_unlocked File "", line 665, in exec_module File "", line 222, in _call_with_frames_removed File "/contraxsuite/lexpredict-contraxsuite/contraxsuite_services/apps/task/tasks.py", line 76, in from tika import parser File "/contraxsuite/ve/lib/python3.5/site-packages/tika/parser.py", line 19, in from .tika import parse1, callServer, ServerEndpoint File "/contraxsuite/ve/lib/python3.5/site-packages/tika/tika.py", line 142, in fileHandler = logging.FileHandler(log_file) File "/usr/lib/python3.5/logging/init.py", line 1008, in init StreamHandler.init(self, self._open()) File "/usr/lib/python3.5/logging/init.py", line 1037, in _open return open(self.baseFilename, self.mode, encoding=self.encoding) PermissionError: [Errno 13] Permission denied: '/tmp/tika.log' (ve) contraxsuite@llb-ideacentre-510S-08IKL:/contraxsuite/lexpredict-contraxsuite/contraxsuite_services$ — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#16>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ANN2bYX7u9lxTY1JQDL0pea5Z1T5QplUks5tAPFOgaJpZM4RBzuD>.

[ericlex]

@legumbot @emichaelsomm - We will investigate this and get back to you shortly. Thanks for your patience as we work through some of these setup / upgrade issues.

[dwmcqueen]

Did anyone discover a work around? I have the same issue on a clean 1.0.5 install using the deploy script.