A spot-on example of scenarios not taken into account in design inadvertently turning something intended to be “joyful” into something “awful”: Eric Meyer really did not want to be reminded of his past year, but still faced Facebook pushing his terrible year's content on him in a “Your Year in Review” feature:
This inadvertent algorithmic cruelty is the result of code that works in the overwhelming majority of cases, reminding people of the awesomeness of their years, showing them selfies at a party or whale spouts from sailing boats or the marina outside their vacation house.
But for those of us who lived through the death of loved ones, or spent extended time in the hospital, or were hit by divorce or losing a job or any one of a hundred crises, we might not want another look at this past year.
The algorithm is even the problem here; it’s the difficulty to grasp that something designers may believe to be great for their target personas might not be for people in other circumstances. What Eric says:
If I could fix one thing about our industry, just one thing, it would be that: to increase awareness of and consideration for the failure modes, the edge cases, the worst-case scenarios.
As he points out in a follow-up post, this can happen to anybody who creates something – this is not about blame for an inconsiderate decision; it’s a reminder that maybe the non-intended consequences of design might be worth more contemplation than its actually intended goals?!