Problems accessing the platform
Incident Report for Fluid Attacks
Postmortem

Impact

An unknown number of users experienced issues accessing the platform. The issue started on UTC-5 24-06-12 at 11:27 and was reactively discovered 28.8 minutes (TTD) later by customers who reported through our help desk [1] that they were unable to log in with their accounts, encountering the `Unauthorized access` message. The problem was resolved in 1.6 hours (TTF), resulting in a total impact of 2.4 hours (TTR).

Cause

We have separate secret files for development and production. The secret was rotated in the development file, but this rotation was overlooked in the production file. This caused problems when logging in to the Platform, resulting in the `Unauthorized Access` message [2].

Solution

The secret was rotated in the production file [3].

Conclusion

To prevent similar incidents in the future, we are implementing a solution to merge the two secrets, ensuring that only one needs to be rotated [4]. ROTATION_FAILURE

Posted Jun 13, 2024 - 15:22 GMT-05:00

Resolved
The incident has been resolved, and access to the platform has been fully restored to normal.
Posted Jun 12, 2024 - 14:11 GMT-05:00
Identified
Problems have been detected with some users experiencing difficulties accessing the platform.
Posted Jun 12, 2024 - 13:12 GMT-05:00
This incident affected: Platform.