Orchestration Service for Sync

[amoghjalan]

What do we need?

In the current architecture of OSS Dora, the synchronization process operates sequentially, posing challenges in tracking its completion status. The absence of clear visibility into the sync progress impedes effective monitoring and management of the system.

To address this issue, we propose implementing a logging mechanism within the database. Upon receiving a sync request, the system will log the initiation of each sync operation, providing real-time insights into the progress of the synchronization process. Additionally, the database table will store the status of every process along with relevant metadata, enabling comprehensive tracking of ongoing processes.

While this approach meets the needs of our synchronous system, it's crucial to anticipate future requirements. As the system evolves into an asynchronous model with numerous simultaneous sync operations, our current logging mechanism may become inadequate. Therefore, we must explore alternative strategies for monitoring and managing sync progress to ensure continued effectiveness.

Furthermore, our orchestration service should incorporate robust error handling mechanisms. In the event of process failure due to errors, the orchestration service must be capable of identifying and handling such scenarios effectively, ensuring uninterrupted operation and timely resolution of issues.

Acceptance Criteria

• Ability to track start end and error in a process