The Admin Portal Latency issue has been resolved.

Cause:
There was an uptick in Elastic Load Balancing 500 errors, indicating servers were failing to respond to app requests. It was suggested that "sticky sessions" (which is a process in which a load balancer creates an affinity between a client and a specific network server for the duration of a session) may be the culprit, so after a review of the code, it was determined that we should disable sticky sessions.

Resolution:
After disabling them, we noticed an immediate drop off in the errors reported. The core issue was that we recently enabled auto scaling (9/6) which terminates servers whenever their threshold drops below a certain value in order to preserve costs. Whenever these servers were terminated, if they had any "sticky sessions" still referencing them, they would error out. By removing the "sticky sessions", we let the load balancers redirect traffic to any available server, which resolved the issue.

The Heartland Team will continue to monitor the situation and make sure that core functionality has returned to normal. We apologize for the inconvenience this issues may have caused our dealers and merchants. If any dealers or merchants experience issues with the Admin Portal please reach to the Heartland Restaurant Support Team @ 833-844-4767 Option 1, or by email at hrpossupport@heartland.us.

Admin Portal is currently experiencing some latency for a small percentage of our users. The appropriate teams are aware of the situation and are working for a resolution. The approximate ETA for a resolution is 45 minutes.

We apologize for the inconvenience these issues may have caused our dealers and merchants.