How to set up queue_job in odoo sh?

I followed the instruction on the ubuntu server the queue_job working very well, after that, I try to set up the queue_job in odoo sh, unfortunately, its does's work, when i go throw the odoo log, one error message show like this:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/requests/adapters.py", line 440, in send
timeout=timeout
File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/local/lib/python3.6/dist-packages/urllib3/util/retry.py", line 388, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=8069): Max retries exceeded with url: /queue_job/session (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f25a8441550>: Failed to establish a new connection: [Errno 111] Connection refused',))

Anyone can tell me how to fix this in odoo sh, thanks a lot.

@Cedric-Pigeon

@Cedric-Pigeon @guewen
Any idea on this? Or is Odoo.sh not yet supported?

@elliotxin

@elliotxin
I got it running, add this to the Odoo config file:

[queue_job]
channels = root:4
host="https://yourenv.odoo.com"
port=443
http_auth_user="youruser"
http_auth_password="yourpassword"

Good luck!

@Cedric-Pigeon

@Cedric-Pigeon @guewen
Think this issue can be closed. This works.

Thanks for the feedback!

Hello @CasVissers ,

How do you locate the odoo config file in odoo.sh?

Greetings

@acysos

@acysos
nano .config/odoo/odoo.conf

@CasVissers thank you

@CasVissers Did you have to do anything else except modifying the config file? Does the amount of database workers make a difference? I can't get it working on my odoo.sh (see #244 )

Hi there, I'm Fabien Meghazi, core developer and responsible of the Odoo.sh platform.
I'm sorry to hijack this thread but I'd like to point out that queue_job (in it's current design) is not compatible with Odoo.sh which uses a hybrid model of multiple threaded workers.

queue_job can somewhat work on Odoo.sh when using only one worker but the problem is that for each job it will trigger as many http calls as there are running workers. When an instance is creating a lot of jobs, this translates to a significative drop of performance due to concurrent updates.

We recently listed the projects hosted on Odoo.sh using this module and we are about to contact the problematic ones.

Greets.

Thanks for informing that and I don't consider it as a hijacking, but a good appreciation that closes the discussion.

@amigrave

Hi @amigrave,

Thanks for the notice. Do you know any way we could make it work on odoo.sh?

queue_job can somewhat work on Odoo.sh when using only one worker but the problem is that for each job it will trigger as many http calls as there are running workers. When an instance is creating a lot of jobs, this translates to a significative drop of performance due to concurrent updates.

This looks like a misconfiguration issue. The number of parallel jobs (so number of workers busy for jobs at the same time) can be configured using ODOO_QUEUE_JOB_CHANNELS=root:1 or equivalent in the configuration file, the default being 1. Or am I misunderstanding what you mean by "for each job it will trigger as many http calls as there are running workers"?

EDIT: I got what you say.

Is it a peculiarity due to the way the hybrid model works, like, it starts several "jobrunner" workers, because you start several odoo main processes which all fork their own "jobrunner" worker? (if that's the case, it probably can be prevented in queue_job's code).

@guewen

Is it a peculiarity due to the way the hybrid model works, like, it starts several "jobrunner" workers, because you start several odoo main processes which all fork their own "jobrunner" worker? (if that's the case, it probably can be prevented in queue_job's code).

@guewen Indeed, you nailed it. Unfortunately there's no easy way to make it work. One could think that you just need to let one worker spawn the jobrunner and somehow prevent the other workers to spawn the jobrunner, but this would not work because Odoo.sh recycles workers on some events and there's no "master" process, hence you could end up in a situation where you don't have the jobrunner alive anymore.

In order to make queue_job work on Odoo.sh I guess it would need to support this hybrid mode of multiple threaded workers and setup a communication between the workers so they can decide who is the master (a bit like a derivative of the registry signaling) and in case the master dies another worker becomes the master. I understand that such a thing can't be foreseen in the development of queue_job because:

it seems overly complex
I can't guarantee the current hybrid model won't change in the future
this Odoo.sh hybrid mode is not "standard"

We already discussed internally about having this hybrid mode standardized in Odoo (I love the idea because it would be less code to maintain in Odoo.sh) but nothing has been started yet in that direction so far and some challenged should be tackled first, such as the longpolling routes which are specially handled by Odoo.sh.

Note: This is just a hot reaction, maybe there's an easier way I haven't tough of to make queue_job work on Odoo.sh but if later I think about an easier way I'll post it here.

@amigrave

@amigrave Thanks for the very informative and detailed answer!

Actually, the fact that we have to spawn only one jobrunner when we have multiple nodes is kind of annoying outside odoo.sh too, so we could tackle this problem for other uses cases. (At camptocamp, we deploy several nodes on production and the unique jobrunner defeats the improved availability of such setup, regarding jobs).

An idea which doesn't seem that difficult for me, would be that the jobrunner could acquire an advisory lock at loading, and the other nodes would only wait until they can acquire it, the first in the race gets to be the new master. Hot reaction too, I may be overlooking details :)

Anyway, I'll think about this and thanks again for sharing your much helping thoughts.

Support multi-nodes with lock on jobrunner

squash! Support multi-nodes with lock on jobrunner

Support multi-nodes with lock on jobrunner

@CasVissers-360ERP

@CasVissers-360ERP

I tryed your solution and I got this traceback in the log system :

_

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/odoo/src/user/extra-addons/queue_job/jobrunner/runner.py", line 237, in urlopen
response = requests.get(url, timeout=1, auth=auth)
File "/usr/lib/python3/dist-packages/requests/api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python3/dist-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='%22https', port=80): Max retries exceeded with url: //name_of_my_app.odoo.com/%22:443/queue_job/runjob?db=name_of_my_database&job_uuid=94b5d0a2-4c5d-4e26-baf5-7b3524f03c73 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8c3f801400>: Failed to establish a new connection: [Errno -2] Name or service not known'))

_

This is my odoo config parameter 👍

[queue_job]
channels = root:4
host = "https://name_of_my_app.odoo.com"
port = 443
http_auth_user = "admin"
http_auth_password = "admin_password"

Do you have any idea ?

@ArgosVeto

@ArgosVeto you should have a look at #256, and give feedback (that I need as I'm not using odoo.sh).

Support multi-nodes with lock on jobrunner

I share my tests about this:

dbfilter=
server_wide_modules=web,queue_job

[queue_job]
scheme=https
port=443
host=mydb-2350951.dev.odoo.com

And service apparently start:

2021-04-07 19:10:47,634 769 INFO ? odoo.addons.queue_job.jobrunner.runner: starting
2021-04-07 19:10:47,634 769 INFO ? odoo.addons.queue_job.jobrunner.runner: initializing database connections
2021-04-07 19:10:47,646 769 INFO ? odoo.addons.queue_job.jobrunner.runner: queue job runner ready for db mydb-2350951
2021-04-07 19:10:47,646 769 INFO ? odoo.addons.queue_job.jobrunner.runner: database connections ready

BTW jobs dont change to enqueued state, so I updated via SQL to enqueued state, and then run a get request to /queue_job/runjob? from python console and jobs works.

Support multi-nodes with lock on jobrunner

Navigation Menu

Repository navigation

How to set up queue_job in odoo sh? #169

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

How to set up queue_job in odoo sh? #169

Description

Activity

CasVissers-360ERP commented on Mar 25, 2020

CasVissers-360ERP commented on Mar 27, 2020

CasVissers-360ERP commented on May 7, 2020

guewen commented on May 7, 2020

acysos commented on May 8, 2020

CasVissers-360ERP commented on May 8, 2020

acysos commented on May 8, 2020

NMisko commented on Jul 1, 2020

amigrave commented on Sep 16, 2020

pedrobaeza commented on Sep 16, 2020

guewen commented on Sep 16, 2020

amigrave commented on Sep 16, 2020

guewen commented on Sep 16, 2020

ArgosVeto commented on Jan 19, 2021

guewen commented on Jan 19, 2021

ovnicraft commented on Apr 7, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions