Activity

1. LowriJenkins

   self-assigned this

   on Sep 21, 2026
2. LowriJenkins

   added this to Data streaming for HRPD-X & SANDALS-IIon Sep 21, 2026
3. LowriJenkins commented on Sep 30, 2026

   

   LowriJenkins

   on Sep 30, 2026

   Last edited by LowriJenkins

   Author

   More actions

   ADR - 12 - Forwarder aioca vs PyEpics

   Status

   Pending

   Context

   The forwarder currently in use to send information from PVs to the file-writer relies on caproto to handle reading channel access PVs. caproto has a bug that prevents reconnection to a pv that has dropped and therefore needs to be replaced by a different library. This could be achieved in a couple of different ways. PyEpics is relativley similar to caproto and supports threaded operation, so a refactor to replace caproto with PyEpics and refactor any sections heavily tied to caproto could be undertaken. On the otherhand, caproto could also be replaced with aioca, unlike caproto and PyEpics, aioca is an asycnhronus system, this would require far greater changes, likley a full re-write rather than just re-factoring but async is usually better suited to IO tasks that are slow or ones that have a large number of connections (such as setting up a large number of PV monitors.) A full rewrite would also present the oppurtunity to prune some content from the forwarder which is present only for the ESS and would not be used at ISIS.

   Decision

   We will rewrite the forwarder to make use of async-io:

   • Channel Access will be handled using aioca
   • PV Access will be handled using p4p's async mode.
   • Sections irrelevent to ISIS (tdct, previous versions of serializers, etc.) will not be carried forward.
   • Kafka interactions will be handled by aiokafka
   • The external facing interfaces such as kafka schemas, serialization, and forwarder configs will not change.

   This decision was made to allow for better handling in situations where large numbers of PVs are in need of forwarding, especially when changing configs. While threading may be enough to handle smaller number of PVs it could cause slow-down for large numbers of PVs unless a thread is made for each individual PV to be setup, and doing so would cause high memory usage due to the overhead required per thread. As the performance would be too slow without either threading or async, and as we expect to require a greater number of PVs in future, re-writing to make use of async will save us having to do this later.

   Consequences

   • A new repository will be opened called async-forwarder
   • An Issue/Issues will be created to begin creation of the new async-forwarder.
   • async-forwarder is tested with the same inputs against forwarder to ensure same outputs.
     • async-forwarder is tested against the bug that was the initial cause for this re-write.
4. Tom-Willemsen

   marked forwarder: still having issues with PV reconnection after a gateway config change #80 as a duplicate of this issue on Oct 1, 2026
5. rerpha commented on Oct 1, 2026

   

   rerpha

   on Oct 1, 2026 · Hidden as resolved

   Last edited by rerpha

   show commentMore actions
6. rerpha commented on Oct 6, 2026

   

   rerpha

   on Oct 6, 2026

   Contributor

   More actions

   I have a very high level question - the main issue with caproto in the current ESS forwarder is that it basically failed to reconnect to PVs (blocks) after the gateway had been re-configured on a configuration change. Does aioca actually solve this issue with a minimal example where you monitor a block value and then change config?
   Probably not as easy to test with p4p as the blockserver doesn't serve PVA...
7. Tom-Willemsen commented on Oct 6, 2026

   

   Tom-Willemsen

   on Oct 6, 2026

   Last edited by Tom-Willemsen

   Member

   More actions

   yes - I tested with an aioca monitor on ...:CS:SB:blah, then renamed blah to blah2. It reported a disconnect. Then renamed blah2 back to blah and aioca reconnected and gave updates when the block was subsequently updated.

   👍React with 👍1Reacted by Jack Harper
8. rerpha commented on Oct 7, 2026

   

   rerpha

   on Oct 7, 2026

   Contributor

   More actions

   ok - as another devils advocate question then, did we consider testing with https://github.com/ControlSystemStudio/pvaify and just using the ESS forwarder with PVA?

   i know it adds another component into the system, but that may or may not be more/less risky than a complete rewrite of the forwarder.
9. Tom-Willemsen

   modified the milestones: Autumn 2026, Spring 2027 on Oct 9, 2026