Service Interruption Update

Incident Report for RoomKeyPMS

Resolved

Our team identified a corrupted application pool that was preventing the RoomKeyPMS application from completing its initialization cycle. A new application pool was created and brought online, which immediately restored normal performance. After extended monitoring confirmed stability, we closed out this incident.
Posted Dec 09, 2025 - 08:29 PST

Update

Our team applied a patch and rebooted the impacted server on December 2 at 9:00 PM PST. The maintenance took approximately 15 minutes to complete.

At 12:30 AM on December 3, we observed that the initialization cycle of the applications was still taking longer than normal. During our monitoring, we identified a corrupted application pool. A new application pool was created and brought online within 30 minutes, which restored performance to normal levels. Performance remained stable after an additional 15 minutes of monitoring.

We will continue to monitor system performance closely. Further updates will be posted here as needed.
Posted Dec 03, 2025 - 08:46 PST

Update

Our engineers have identified a memory management issue affecting certain application servers under peak load. To prevent similar disruptions and improve overall RoomKeyPMS performance, we will be deploying a patch tonight.

This update will require a quick server reboot, which may result in a brief downtime of up to 5 minutes. The maintenance window is scheduled for Tuesday, December 2, 2025, starting at 9:00 PM PST.
Posted Dec 02, 2025 - 16:04 PST

Monitoring

We experienced intermittent outages during the mornings of Saturday, Nov 29, Sunday, Nov 30, and Monday, Dec 1. Our web servers are configured to automatically restart applications if they appear unresponsive. Over this period, the RoomKeyPMS application required longer than usual to fully load. Because of this extended load time, the servers detected the application as non-responsive and repeatedly attempted to restart it before it could complete its startup process.

This created a restart loop that prevented the application from fully loading and caused service interruptions for customers.

The automatic health-check and restart behavior has been disabled, and our team is now manually monitoring the application to ensure stability. Service has been restored.

A deeper investigation is underway to determine the root cause of the longer than usual load times. We will provide an update as soon as more information is available.

Thank you for your patience, and we apologize for the inconvenience this caused.
Posted Dec 01, 2025 - 14:07 PST

Identified

Over the past weekend and again this morning, some customers experienced service interruptions with the RoomKeyServer application. These outages impacted normal operations, and we sincerely apologize for any disruption this caused.

Our engineering team is actively investigating the issue. While the root cause has not yet been identified, the platform is currently stable. We are closely monitoring the system and taking steps to prevent further disruption.
Posted Dec 01, 2025 - 08:12 PST

Investigating

We are currently experiencing a system outage affecting RoomKeyPMS services. Beginning Saturday, November 29, 2025, customers have experienced multiple service interruptions, including an outage on Saturday morning and another outage on Sunday morning.

This morning, an attempted fix was deployed to the server but was not successful, and the servers went down again. Some users may be seeing error messages stating that errors were encountered during their last operation.

Our development team is actively investigating the issue and working to restore full service as quickly as possible. We will provide further updates as soon as more information is available.

Thank you for your patience and understanding.
Posted Dec 01, 2025 - 06:36 PST
This incident affected: RoomKeyPMS (Desktop & Web Applications) (East Server, West Server, Lakeview Server, Magnolia Server, Asia Server, Jupiter Server).