So I woke up Thursday morning to a strange surprise. I had a few tweets saying "good luck with 1.8" and suddenly many more "when is 1.8 gonna be updated lol" than I usually had. What a curious thing!
Unfortunately, sleepy me didn't realise what this meant at this point in time, and was unaware that he was better off just staying in bed all day, so he decided to go investigate. Some time later, it hit me. Like a wall full of irc highlights. Mojang released 1.8 much earlier than we thought they would. Uhoh!
It appeared that I was a couple of hours late, so I thought, maybe it's ok and we've already started to work on it. But checking our repos and irc channels... it seems I was the only one who was available to handle it. It was just a bad time for all of us. The best part about our team is also the worst part; we're each in a separate timezone so there's always one of us awake for the major hours, but times like this it's just not possible for anyone else.
What's a bone supposed to do, but update the server by himself? First things first, though, is to tell everyone DON'T PANIC and that I'm working on it. Done! That wasn't so hard. Things might be easier than I expected!
So, to the actual updating. We have a top-secret-ultra-sensitive-really-hackish 3 step process of handling updates. Four, if you count the pizza time afterwards. The first two steps are essentially deobfuscation, and the third is almost rewriting the entirety of Craftbukkits net.minecraft.server section. That's a whole lotta code. We have some very fancy tools written by ourselves for assisting during this, but it's mostly manual hard work all around.
This time, though, something odd happened. 1.8 broke our tools. Fortunately we anticipated this before and the brilliant Grum had some fixes out, but there were still some errors cropping up that we had to ignore for now. It took me a couple of hours, which is much longer than I expected, but I had finished both stages and I was happy with the result. We had a decent deobfuscated copy to work with. Now for the fun part; rewriting Craftbukkit!
I got about 10 files in, when I realised "this is going to be difficult with nobody else to keep people updated at the same time", so I decided to take a 10 minute break and throw up a google spreadsheet with a fancy progress indicator and stuff. Filled in the list of things I had to do, tweeted the link, and that was my entire break over. Back to coding!
There were a big bunch of changes and refactoring which meant that we had to (and still have to, at time of writing) review a lot of our old code to make sure we can still use it in the new version, and it introduced a big opportunity for bugs to sneak in.
4 hours passed, I hadn't taken a single break other than to post updates on twitter every half hour or so, and my hands were killing me. The list had just hit 50%, and I was so very tired already. Grum had become available to help speed it up, and did a handful of files before he unfortunately had to run off again, but that shaved a tonne of time off for me. Yay Grum! :D
7 hours later. Over 13 hours from when I started this whole thing. Tahg was available to assist with some things too, and we finally hit 100%. It was like spending all of Christmas dinner as a kid, gorging yourself on food that you just can't eat anymore but you know that you'll get to open presents soon. Well, that time came, and I decided we should build it and let everyone see for themselves just how badly it'll break.
Uh. Yeah. One problem with that. Jenkins, a robot which builds Craftbukkit for us, decided now would be a good time to not function as we like. Humbug! I compiled it myself and tweeted the link, and my server went down within seconds. Nothing bad though, seems there was a leak of some kind in nginx and a restart fixed it easily. Jenkins realised he would be out of the job soon and decided to build properly, and then I could put pressure away from my server. Brilliant!
So just how bad did it break? Not that bad, actually! There were some obvious bugs to start with, which we fixed pretty easily, but there were also some nasty bugs which we couldn't figure out yet. I stayed to test and fix all the bugs I could find, and help locate the other ones, and before I realised it the team had already gone to sleep and woke up again for the next day. Oh crap, I stayed up all night working on this. Oh well!
I fixed a few more bugs and decided to head to sleep, because at this point I had been working for over 24 solid hours with no breaks, my hands were raw and my eyes had melted. I needed rest. I had promised people that I wouldn't sleep until it works, and it works so I think I earned the rest :D
Next day, we had some more progress squashing the small bugs but there were still 2 killer bugs that we absolutely could not find; you can't walk through doors that are placed together (double doors), and entities keep being saved into the wrong chunks for no apparent reason. We managed to squish the first bug yesterday, apparently there was an issue with overloaded methods (yay obfuscation!), but the second one still alludes us and we cannot find it at all.
We also fixed a pretty nasty accidental DOS within the server; every tick, it would send playercount^2 packets. For a server with 100 people, that's 200,000 packets being send out a second plus 2,000 being received. Ouch! We fixed this with some cool mathy code that Grum devised which distributes the load equally every tick, and added an option to disable the whole player list feature entirely if people so wish, so it should be much better now.
Yet again, as I type this I notice that I'm slowly fading into the next day, so I'm going to get some more rest and hopefully find the entity bug tomorrow. Wish me luck :D