Elune Instance Server Down

#0 - Aug. 18, 2007, 12:26 a.m.
Blizzard Post
I was in Karazhan about 30 minutes ago when the server crashed. Now I cannot log on, it says that all instance servers are down. About how long until this is fixed? Thanks!
#1 - Aug. 18, 2007, 1:09 a.m.
Blizzard Post
Are you still experiencing this issue, Sweetcakes? If so, can you offer the precise error message you receive when trying to log in? :)
#9 - Aug. 18, 2007, 2 a.m.
Blizzard Post
I shall see that our server technicians are informed of potential concerns with Elune's stability, folks, and keep you apprised of any updates. :)
#37 - Aug. 18, 2007, 2:50 a.m.
Blizzard Post
As an update, in order that hardware concerns may be further troubleshooted, Elune is being restarted momentarily. I'll keep you apprised. :)

http://forums.worldofwarcraft.com/thread.html?topicId=1133963591&sid=1
#69 - Aug. 18, 2007, 5:21 a.m.
Blizzard Post
Our server technicians and database administrators continue in their investigation of potential underlying issues that are contributing to the instability Elune has recently experienced, folks; you have my utmost assurance that your concerns -- which are more than valid -- aren't being disregarded, but I'm afraid that such research and detailed analysis does take time. We are much obliged to have your continued patience. :)
#97 - Aug. 18, 2007, 6:49 a.m.
Blizzard Post
Q u o t e:
YES another shutdown....


Indeed, due to recent hardware changes, a full hardware reboot is being performed in our continued efforts to resolve the matter. :)
#101 - Aug. 18, 2007, 7:11 a.m.
Blizzard Post
Q u o t e:
I hope this fixes it. One crash occurred as we were looting a boss in heroic BF. No one got any loot, but we were saved to the instance. ><


Once Elune is brought back online, we shall monitor its performance to ensure that its stability remains within acceptable boundaries. :)
#116 - Aug. 18, 2007, 7:47 a.m.
Blizzard Post
Q u o t e:
i'd settle for a few fewer condescending blue responses and a few more concrete posts regarding why i'm paying for inferior customer service, but of course the specific error message would help ;) =P xD


Pray tell, Wau, what condescension have you found in my posts regarding this matter? I am ever eager to work toward self-improvement. :)
#118 - Aug. 18, 2007, 8 p.m.
Blizzard Post
Q u o t e:
my sincerest apologies and regrets.


No need for apologies, friend. I sympathise with your frustration. :)

As an update, our server technicians are aware that Elune's lapses in connectivity have persisted beyond the recent hardware reboot; their diagnostic efforts are currently underway, and our system administrators have become involved for further troubleshooting and consultation.

Should any new information come to light regarding our progress, I shall be certain to let you folks know; for now, our work continues. :)
#122 - Aug. 18, 2007, 9:38 a.m.
Blizzard Post
Q u o t e:
All instance servers down, as of now.

Good times.


Which instances in particular have you observed experiencing issues? :)

Edit: To clarify, I'm able to zone into instances on all three continents; at what point are complications arising for you?
#124 - Aug. 18, 2007, 9:52 a.m.
Blizzard Post
Q u o t e:
Well, at this moment it's Karazhan.


I'm also able to zone into Karazhan normally. What occurs when you try to do so? :)
#126 - Aug. 18, 2007, 9:57 a.m.
Blizzard Post
Q u o t e:
Our whole raid is stuck at the loading screen. We got d/c and now we are all getting "no instance servers are available" when we try to get back on our toons.


Understood; I've already passed this information onto our server technicians for further perusal, Macfeagle, and shall let you know if I hear anything. In my experience, however, these concerns can also resolve themselves after a short time -- this may occur for your group. :)
#175 - Aug. 18, 2007, 9:29 p.m.
Blizzard Post
Our server technicians are currently investigating the source of these repeated realm crashes and hope to implement a more permanent resolution soon; in the meantime, however, another restart may become necessary as the troubleshooting progresses.

I am presently awaiting word from my sources and will be happy to relay further updates as they arrive.

Thank you so much for your continued patience, folks. It truly is quite appreciated. :)
#284 - Aug. 18, 2007, 11:33 p.m.
Blizzard Post
I apologize profusely for the consistent crashes many of you have experienced. Our server technicians are presently conducting a series of tests to gauge the likely source of Elune's current instability; the unfortunate result of this testing has been, as you have witnessed, multiple drops in all applicable and associated servers.

We are restarting the realm at this time. Please stay tuned for more information.
#371 - Aug. 19, 2007, 2:06 a.m.
Blizzard Post
Folks, I am very sorry for the frustration and inconvenience this has caused you, our technicians are working as quickly as possible to resolve the issue and what updates I can provide will be brought to you as soon I get them.


That said, please keep any comments civil and within the Code of Conduct.
#375 - Aug. 19, 2007, 2:12 a.m.
Blizzard Post
Rantuk, my comment was directed to those people who feel it necessary to use inappropriate language and subject matter on these forums; it was not to you.

#457 - Aug. 19, 2007, 5:36 a.m.
Blizzard Post
Thank you all for your patience while our Technicians work on this issue, the realms will be coming down shortly to make a few adjustments. Hopefully that will improve the stability on the realm.

http://forums.worldofwarcraft.com/thread.html?topicId=1133904630&sid=1

The realm is scheduled to be down 15-20 minutes.
#481 - Aug. 19, 2007, 6:01 a.m.
Blizzard Post
Q u o t e:
Sorry but people do not deserve vacations for fixing screw ups that they made.


And all people everywhere should instinctively know the cause of an error when it happens instead of having to investigate the cause, or better yet predict that an error is going to occur with a procedure and fix it before it breaks.

They should also be able to give me the lotto numbers and winning horses when I ask and hand out fresh minty gum when I ask for it.

Q u o t e:
You edited my post but didn't change anything. o_O

I removed the inappropriate language, Blootie. :)
#521 - Aug. 19, 2007, 6:23 a.m.
Blizzard Post
Kimixi, I'm not a technicians I don't know what steps they are taking to investigate and resolve this but from everything I understand restarting and testing is part of the procedure.

For example the realm is up and running but crashes a few times; you investigate and figure out that lever B might need to be at 90% instead of 75%. The realm comes back up and all seems fine.

The realm crashes again and in looking at what was happening you find that some strange message which may indicate a problem registered after that last crash so you go into it and make some adjustment to allow for 100 of aspect E instead of the previous 10.

These systems hold massive amounts of data with thousands of people logged into them at any one time. There are programs as well as hardware issues to consider and although some things may appear obvious it is not always so. The issue could with one system or five.

What if aspect E conflicts with aspect 5A but does not show up until the realm has 2001 people on it instead of just 2000, what if you never see it until that one last person enters the game?

I understand the frustration but shutting down the realm completely doesn't do any good if you can't see if what you think should have fixed the problem did or not. If it is a hardware issue they need to find which part is not working right and fix or replace it.

Equipment rarely comes with a read out and a red blinky light that says "oh it was me, this is what is wrong and what you need to do to fix it".

I'll provide what information I can but we don't receive technical updates, we are notified when a restart is needed.
#532 - Aug. 19, 2007, 6:44 a.m.
Blizzard Post
I can't believe you all understood my completely non-technical ramblings. *laughs*

Let us hope that the adjustments have resolved the issue. Our technicians will monitor the realm and I'll keep an eye on it as well but please let me know if the realm experiences any trouble.
#549 - Aug. 19, 2007, 3:10 p.m.
Blizzard Post
HAY GUYZ WATS GOING ON IN DIS THREAD?

I'm currently showing that Elune has been running at near-optimum levels for approximately 7 hours; all zones (Kalimdor, Eastern Kingdoms, Outlands) and instance servers should be available at this time. Can I have this confirmed?
#581 - Aug. 19, 2007, 4:58 p.m.
Blizzard Post
I'm currently speaking with our active server technicians. I'll relay whatever information is provided me shortly.
#586 - Aug. 19, 2007, 7:03 p.m.
Blizzard Post
You're welcome to assume whatever you like, Taf; assumptions, however, do not immediately dictate reality.

This said, our server technicians have been actively monitoring Elune's progression throughout the evening and early morning hours. Stability peaked around 12am PDT, and was maintained until roughly 7:30am PDT; since then, levels seem to have normalized.

We will continue to review Elune and its associated instance servers as the day progresses. At this time, we are still investigating the source of the most recent crash, though remain hopeful that it was simply an isolated incidence.

Any information you can provide--regarding latency or overall stability--is greatly appreciated. :)
#800 - Aug. 20, 2007, 1:31 a.m.
Blizzard Post
Colourful, Slopayne, but no.

Our technicians are 24/7 and are working on the issue. We have been unable to provide an update because we haven't received one yet.

Everyone knows that we are aware of the issue and are working on it so I didn't think it would be wise to post that over and over again as it would just add to the frustration.

As soon as can provide an update with new information I will.
#816 - Aug. 20, 2007, 1:51 a.m.
Blizzard Post
Q u o t e:
thank you for the canned response. And I realize in the end that you are not the one that should be taking the brunt of the flak. however, this has been an issue for 5 days. I work for a tech company, and I know this. If I functioned this way in my job allowed this problem to persist for 5 days I would be fired. I have a feeling it is the same in you're company...that leads me to believe nothing is being done about this, and probably wont be until tommorrow morning at the earliest.


Canned? Interesting.

You may believe whatever you'd like, Varthu, I'm not going to argue with you. You work for a tech company and that's great, but until you work for this company and know our systems you really can't compare it the two since you have no idea about the inner workings of our systems.

Q u o t e:
This is new... I'm in TK, all the mobs have stopped, and so have all of the PCs... no one is moving but me... BUT we can still talk in chat.


Thank you, I'll pass that along.
#859 - Aug. 20, 2007, 3:20 a.m.
Blizzard Post
Folks, please. Throwing insults at one another, me or our technicians will not get this issue resolved any faster. Our technicians are not reading this thread, only I am.

Let's keep this thread open and available for updates when they become available.
#902 - Aug. 20, 2007, 5:34 a.m.
Blizzard Post
It looks like the last efforts by our technicians may have had a positive outcome and the realm has thus far held steady for over 2 hours.

We will continue to monitor the realm and respond if any more issues arise but hopefully the instability has been resolved.

Thank you all for your patience.
#909 - Aug. 20, 2007, 6:11 a.m.
Blizzard Post
Sarik, the Game Master Department would not make such decisions and no inquiry by me will result in an answer. If it happens it will be announced in due time.
Q u o t e:

Stable for two hours is great. But it's like, you know, after peak time. It was "stable" for the majority of yesterday until many people started logging on. Gratz on people going to bed and making you guys look good!


Perhaps that is why I said "We will continue to monitor the realm and respond if any more issues arise".

#913 - Aug. 20, 2007, 8:26 p.m.
Blizzard Post
Q u o t e:
We'll see what tommorow is like, but for the time being I wanted to thank the Blizzard crew for giving me the rest of my night crash free. It was nice to get my quests done and hit 40.

Hope it works tommorow.


Congrats, Laelae, we'll be watching the realms eagerly as more and more people log on.
#922 - Aug. 20, 2007, 3:26 p.m.
Blizzard Post
Elune's progress has been continually monitored throughout the late evening and early morning hours; and, at this time, we are noting 11 hours and 30 minutes of stability and counting.

As concurrency increases, we will likely redouble our efforts and maintain a firm watch on this particular realm's performance. We remain hopeful, however, that last night's corrective measures to have successfully resolved all associated concerns.

Only time will tell, though, so please do keep me updated with any information you may feel pertinent. Thank you once again. :)
#926 - Aug. 20, 2007, 5:46 p.m.
Blizzard Post
Nearly 14 hours, ladies and gents; so far, so good. :)
#930 - Aug. 20, 2007, 6:06 p.m.
Blizzard Post
Q u o t e:
Shall we?


Let's!


That aside, for those of you who are experiencing connectivity issues, could you do me a fairly large favor and attempt to connect to Khaz'goroth at this time? Please report any results back here. Thank you!
#937 - Aug. 20, 2007, 6:26 p.m.
Blizzard Post
Awesome. Thank you for your feedback. :)
#943 - Aug. 20, 2007, 8:28 p.m.
Blizzard Post
Elune has been operating within optimum levels for approximately 16.5 hours with an average concurrency of 1700 players. :)
#945 - Aug. 20, 2007, 9:01 p.m.
Blizzard Post
Q u o t e:
Syndri. 16.5 hours during off peak hours isn't very comforting. The real problems keep showing up when players start loggin in in the evening.


This is why we are continually monitoring Elune's activity and intend to continue to review all associated statistics as the day progresses.
#949 - Aug. 20, 2007, 9:55 p.m.
Blizzard Post
Q u o t e:
Were you monitoring during peak hours?


Yes.

Q u o t e:
Was something changed/fixed?


Yes.

Q u o t e:
Was that monitored? Was there a comparison made?


Yes.

At present concurrency yesterday, Elune could not maintain its active population for more than 3 hours; as noted, both realm and instance servers were experiencing regular drops in stability, kicking all logged-in players offline for a brief period of time. These statistics were monitored and eventually cataloged within our records.

Based upon this recorded data, our server technicians believe to have discovered the source of the consistent crashes and implemented a solution accordingly late last evening. Regrettably, this occurred during off-peak hours, so population effect has not yet been fully tested.

We continue to keep close watch on Elune and its inhabitants, particularly as peak hours draw near. Again, any assistance or information you can provide is greatly appreciated.

#965 - Aug. 21, 2007, 5:02 a.m.
Blizzard Post
I'm very glad to hear all their efforts were not in vain. We'll stop poking them with sticks for a little while. :D
#967 - Aug. 21, 2007, 6:43 a.m.
Blizzard Post
Our technicians do not monitor this forum, Megamitsuki, and we wouldn't be able to provide you with that information, I'm sorry.
#971 - Aug. 21, 2007, 4:06 p.m.
Blizzard Post
Concurrency peaked last evening around 3,500; an impressive population statistic, both alone and in comparison to active weekend numbers.

This said, Eune has remained stable for over 24 hours and continues to operate within normal ranges. All instance servers are active; mobs, doodads, and NPCs are functioning appropriately; and players are able to connect and stay connected without unwarranted interruption--performance is optimal. At this time, we are considering the issue resolved. :)

Thank you for your profound patience, ladies and gentlemen. It's been a pleasure.