Impact

An unknown number of users experienced issues accessing groups. The issue started on UTC-5 24-08-23 10:45 and was proactively discovered 57.6 minutes (TTD) later by one of our monitoring tools, indicating some components' failure. The problem was resolved in 1.6 hours (TTF) resulting in a total impact of 2.4 hours (TTR) [1].

Cause

A migration was executed that altered the system's management of user access to groups. During this process, an important access control attribute was unintentionally removed, resulting in users losing access to their groups [2].

Solution

A backup of the database from before the migration was created and used to restore the affected group access settings [3].

Conclusion

to avoid similar issues in the future, the system will be updated to ensure critical data attributes are always required and cannot be left out. This will help prevent problems by ensuring essential information is always included and checked during updates. FAILED_MIGRATION < INCOMPLETE_PERSPECTIVE