Uploaded image for project: 'CloverDX'
  1. CloverDX
  2. CLO-13032

Reliable JMX messages between Worker and Server

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: rel-4-9-0
    • Component/s: Server
    • Security Level: Users (General product issues)
    • QA Testing:
      UNDECIDED

      Description

      Release notes : job lifecycle messages between Worker and Server Core and now transported over HTTP. The previously used JMX remote protocol is unreliable.

      Based on our tests and observations, it looks like JMX messages between Worker and Server can get lost under high CPU load. JMX by itself does not guarantee message delivery. Losing important messages leads to issues, e.g. we don't get info that a phase or a graph finished, and don't perform necessary actions. We can detect the lost messages (we have sequence numbers).

       

      We need to find a solution to this. There's several options:

      • detect the lost messages and request them again - needs message buffering (at least on Worker side). This is basically "Make JMS more reliant".
      • wrap JMX in other protocol, e.g. in the usual HTTP call we use now, this is about making the JMS transport more reliable. However, we already know that we are quite overloaded with the HTTP connections.
      • change JMX to other technology - some queue?
      • ?

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                urbanj Jaroslav Urban
                Reporter:
                urbanj Jaroslav Urban
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 4 days, 6 hours
                  4d 6h