|
Project |
Name of incident |
Date / Time |
Request from client |
Involved (Internal) |
Involved (External) |
Solution |
Conclusion how to avoid |
|---|---|---|---|---|---|---|---|
|
OneLine: Play2win |
Incident #1 [48h reminders stopped sending on competitions] |
28.11.2024 3:35PM (on live after launch) |
client didn’t know about the issue (after win4win recent reminders bug we decided to fix Play2win bug quietly and track time to another task.) |
Anna Aksonenko (fixed) Artur Bulan (main dev but was on the sick leave) |
nobody |
Request optimization: some checkups moved from code to DB request directly, so the data we receive became smaller and more actual. Reason the issue appeared: Request didn’t include strict timeframe (we were taking all old registrations created earlier than some date and later it lead to the accumulation) and in the code the registrations without partner ID were excluded (but they were not excluded from the request, so we received 1000+ rows, а потім у коді крутили їх виключаючи по одному). |
If it’s a new code – the Code review was required and possibly there the logic problem could be captured. Reminder log should be checked from time to time or additionally some monitoring with Slack notification applied. If the log was checked from time to time it should be possible to notice the gradual increase in reminders number. (This issue couldn’t be captured by the one time testing) |
Leave a Reply
You must be logged in to post a comment.