Darkmeyer IssuesThis morning at 10:30am BST we released the eagerly-awaited Sins of the Father quest, along with the city of Darkmeyer. The game was turned off 13 minutes after the update, so needless to say it didn't quite go to plan. The game worlds are back up now, and we can see thousands of you enjoying Morytania's fine swamplands.
We'd like to explain what happened, what we did to resolve it, and why we made those decisions.
Timeline of events:
- 10:00am BST - We started the 30 minute countdown timer for the game update.
- 10:30am BST - The game update was released.
- 10:35am BST - We began to observe issues with player items.
- 10:39am BST - We began a shutdown of all game worlds, accompanied by a three minute countdown timer.
- 10:43am BST - All worlds were confirmed to be offline.
- 10:47am BST - We made the decision to begin a rollback.
- 14:39pm BST - Game worlds began to appear online and players began to log in.
We constantly updated the Darkmeyer newspost, the Jagex support pages and our social media channels throughout this time to keep you all in the loop.
What was the issue?
The game update corrupted item IDs. Put simply, it meant that a number of items from before the release now had incorrect IDs. Players with Twisted Ancestral Robes (any piece), the Twisted Ancestral Colour Kit, Flying Vespina or Enraged Tektiny found that those items had changed to new items which arrived alongside Darkmeyer.
Why did we choose to rollback the game?
There were 1,141 items affected in total, owned by 846 unique accounts. This means that up to 846 players could have been adversely affected by the bug. It's unlikely that all of these players logged in during the 13 minute period that the game was online, but at the time we weren't able to discern the precise number of accounts affected. We discussed simply deploying an update to convert the item IDs back to what they should have been, but we had no way of knowing whether the accounts kept the items, or had discarded or used items they didn't expect to have. We felt the safer option was to rollback the game to a point where no-one's items would be affected.
Is it fixed?
We think so! It's the first rollback performed remotely, but at the time of writing we've got around 100,000 players logged in and many of them are working their way through Sins of the Father. A very small number of players will be presented with the following error message when attempting to log in: Error loading your profile. This error is related to the rollback. Rest assured, we're trying to fix this issue as soon as possible. If you're facing this problem, you don't need to do anything - we'll identify the error with your account and work to fix it. Unfortunately we don't have an estimate of how long it will take to resolve these errors. We appreciate your patience and we apologise for the inconvenience.
What's next?
Now the real fun begins. You get to play Sins of the Father and explore the bloody delights of Darkmeyer. That usually means finding small (hopefully) things for us to fix and tweak. We'll continue to monitor the game update performance and engage as necessary. We want to avoid similar issues in the future. We're tightening our processes for how we build release versions of the game, and the QA team are adjusting their approach to merge testing to assist in preventing this from occurring again.
The Old School Team