Impact

An unknown number of users found problems related to outdated, missing, and incorrectly displayed data in various parts of the Platform, such as ToDo tabs, vulnerability search, indicators, and ToE. The issue started on UTC-5 22-08-02 17:00 and was reactively discovered 15.4 months (TTD) later by a customer who reported through our help desk [1] that he found inconsistencies with findings' indicators. The problem was resolved in 8.1 days (TTF) resulting in a total impact of 15.7 months (TTR).

Cause

The Platform uses advanced filters that rely on a search engine fueled by data from the database through specific software. This implementation's approach proved unreliable, causing multiple failures and periods of service disruption [2].

Solution

The engineering team opted to implement a change in approach, transitioning to AWS Lambda [3].

Conclusion

The issue arose in production due to the initial adoption of an approach that proved suboptimal, failing to seamlessly integrate with our repository's continuous deployment dynamics. With the new implementation operating smoothly, the team is actively investing efforts in refining the technical decision-making process to anticipate and effectively address similar changes in the future. INCOMPLETE_PERSPECTIVE