RFO June 22nd and June 23rd 2020
Due to events related to COVID-19, related business shutdowns, and consecutive reopenings, over the last few weeks, Star2Star has observed significant changes in traffic patterns compared to traffic volume and call distribution for its StarCenter 2.0 contact center product.
Between 2:30 pm EDT to 4:00 pm EDT on June 22 and June 23 a subset of Star2Star’s customers subscribing to Star2Star’s contact center product, StarCenter 2.0, were sporadically unable to process queue calls. These customers, all of which were hosted on a specific server stack, were impacted to various degrees based on the configurations of their call distribution strategies. The problem manifested itself as an inability of the system to deliver calls to available agents, as well as wrong agent status display in the MQM. The problem intermittently affected call processing on the particular StarCenter stack for 30 to 45 minutes and was fully resolved when call processing levels returned to normal without internal operational intervention.
Star2Star continuously monitors and adjusts its network capacity and processing resources according to the changes in volume and to predictions of traffic pattern shifts. While Star2Star’s predictions are typically correct, such predictions are not always accurate as it is hard to determine how the traffic volume will change.
Over a period of the last few weeks, Star2Star has observed an unusual amount of call attempts on network elements responsible for the hosting of StarCenter 2.0. The excessive amount of call setups per second has affected signaling and media processing capabilities, resulting in a negative impact on the functionality of one of the StarCenter 2.0 stacks. While we are still determining the exact root cause of this event, Star2Star has developed a strategy that has stabilized the affected StarCenter 2.0 stack.
The following are the steps taken or in process by the Star2Star team to mitigate outages on this stack:
- Deployed a configuration change last night, June 29, 2020. This configuration change will help with processing elements at a higher scale.
- Review incoming call routing strategy and queue configuration for existing StarCenter 2.0 customers. The result of this will be specific recommendations for customers and partners on how to simplify certain configs, which is expected to help ease the load on the underlying infrastructure
- Ongoing review and acceleration of migration schedules for existing customers from StarCenter 2.0 to the new StarCenter 3.0 product.
Update - The reported concerns were resolved at approximately 5.00pm EST on 6/23/2020. We will be furnishing an RFO for June 22nd & June 23rd 2020. We apologize for the inconvenience.
Star2Star Technical Support is receiving reports of issues with the Multi-Queue Monitor, agent status is not updating when a call is complete, and agents may experience log-in concerns. Our Escalations team is actively working on determining root cause and mitigating this concern. We will will provide an update as soon as possible. We apologize for the inconvenience.Notification