Roblox Goes Down For Over 60 Hours During The Weekend

By now, you probably already know that Roblox was the down for nearly the entire Halloween weekend. If the flood of memes hadn’t tipped you off, then your little brother’s cries of frustration probably did. His and the tens of millions of other players like him who were treated to a nasty surprise when the beloved sandbox game unexpectedly went offline for more than 60 hours without an explanation from the devs.

Most players thought that it had something to do with Mexican restaurant Chipotle’s announcement that they were giving away $1 million worth of burritos through the game. The team later assured the community that it had nothing to do with the promotion and that the outage was caused by a combination of several server issues as CEO David Baszucki explained in yesterday’s dev blog:

“On Thursday afternoon October 28th, users began having trouble connecting with our platform. This immediately became our highest priority. Teams began working around the clock to identify the source of the problem and get things back to normal.

This was an especially difficult outage in that it involved a combination of several factors. A core system in our infrastructure became overwhelmed, prompted by a subtle bug in our backend service communications while under heavy load. This was not due to any peak in external traffic or any particular experience. Rather the failure was caused by the growth in the number of servers in our datacenters. The result was that most services at Roblox were unable to effectively communicate and deploy.

Due to the difficulty in diagnosing the actual bug, recovery took longer than any of us would have liked. Upon successfully identifying this root cause, we were able to resolve the issue through performance tuning, re-configuration, and scaling back of some load. We were able to fully restore service as of this afternoon.”

Roblox live operations resumed Sunday afternoon. Details can be found on the Roblox official site.