Repository navigation
[ADD] Capture output log - #7
ventor-dev wants to merge 3 commits into
Conversation
0feb1f8 to
ffc1df6
Compare
d6dccc2 to
afbddbf
Compare
| "scheme": odoo_config.get("queue_job_scheme", "http"), | ||
| "host": odoo_config.get("queue_job_host", "localhost"), | ||
| "port": odoo_config.get("queue_job_port", 8069), | ||
| "capture_output": _bool(os.environ.get("QUEUE_JOB__CAPTURE_OUTPUT")), |
There was a problem hiding this comment.
Should be false by default
There was a problem hiding this comment.
And this code is deprecated - we introduced [queue_job] section (easy to fix).
| handler.setFormatter( | ||
| logging.Formatter("%(asctime)s %(levelname)s %(name)s: %(message)s") | ||
| ) | ||
| handler.setLevel(logging.DEBUG) |
There was a problem hiding this comment.
Unfortunately, it breaks Odoo standard logging and clients will get hundreds or even thousands of DEBUG lines in their logs (it will be a huge problem for Odoo.sh users):
|
|
||
| @route('/queue-job/log/export/<int:wizard_id>', type='http', auth='user') | ||
| def export_log_txt(self, wizard_id): | ||
| wizard = request.env['queue.job.log.wizard'].browse(wizard_id) |
There was a problem hiding this comment.
AI tells that it would be nice to call wizard.check_access('read') to avoid 500 server error and raise correct 403 error (I didn't check it but sounds nice)
| /> | ||
| </header> | ||
| <sheet> | ||
| <field name="log_text" invisible="1"/> |
There was a problem hiding this comment.
It may affect performance - Odoo will load (and pass to UI) this field every time user opens job, isn't it? There may be hundreds / thousands of lines (kilobytes or even megabytes of text data)
| string="Captured Log", | ||
| readonly=True, | ||
| help="Output captured (stdout + stderr + root logger) during job execution.", | ||
| ) |
There was a problem hiding this comment.
(Not critical if we make this option optional but worth to know): a user having 100k jobs/month with even 100 KB of captured output each is ~10 GB of logs per 30 days (retention time)... It will be a problem (even if some users accidentally forgot to disable this feature).
There was a problem hiding this comment.
I run initial import three times in our Shopify connector and queue_job table uses about 7 MB (27 jobs only)...
| 'summary': '''Lightweight background jobs module used as a technical dependency for VentorTech integration modules. | ||
| Built on top of Job Queue module from OCA (https://github.com/OCA/queue). | ||
| ''', | ||
| 'version': '19.0.1.0.5', |
There was a problem hiding this comment.
Version bump (minor)
|
|
||
| def _try_perform_job(self, env, job): | ||
| """Try to perform the job.""" | ||
| job.set_started() |
There was a problem hiding this comment.
By the way, do we clear logs on job requeue action? E.g. if logging was disabled, we need to clear saved logs to avoid confusion
| """ | ||
| job.env.clear() | ||
| with Registry(job.env.cr.dbname).cursor() as new_cr: | ||
| job.env = api.Environment(new_cr, SUPERUSER_ID, {}) |
There was a problem hiding this comment.
Won't it drop user and context which were used in original Environment object? Previous version re-used original environment object simply replacing cursor (which looks better)
|
|
||
| old_stdout, old_stderr = sys.stdout, sys.stderr | ||
| sys.stdout = _Tee(old_stdout, buffer) # type: ignore[assignment] | ||
| sys.stderr = _Tee(old_stderr, buffer) # type: ignore[assignment] |
There was a problem hiding this comment.
AI suggested that we can have issues because default sys.stdout is more complex object (TextIOWrapper?) with fewer methods (in comparison with _Tee class placeholder). Most likely it won't lead to issues right now (while using our connectors) but good to know that it is source of possible issues
|
|
||
| @contextmanager | ||
| def capture_output() -> Iterator[StringIO]: | ||
| """Collect stdout, stderr, and root-logger output for the ``with`` block. |
There was a problem hiding this comment.
I noticed one more side effect (bad one): while some job is running (Started state) all user actions will be logged in DEBUG mode (e.g. if I browse some pages or click buttons I can see SQL queries in the logs). This is major issue.
2 participants
No description provided.