NETWORK STATUS
Sorry if you see this post twice... We are investigating reports of telephone company recordings ("all circuits busy") from users in Chapel Hill who are calling 425-7500. Please call 929-2160 until further notice -- but if you are having problems with X2 and 425-7500 works better, use 425-7500. -SC
Our Long haul vendor will be replacing a defective IMUX unit at our Los Angeles POP. Customers in Southern CA will experience failing authentication and loss of IP connectivity during this maintenence window. -rick
Our Long haul vendor will be replacing a defective IMUX unit at our Los Angeles POP. Customers in Southern CA will experience failing authentication and loss of IP connectivity during this maintenence window. -rick
Although we did get a new circuit in the hunt group tonight, lessening the impact of the busy signals, another circuit we intended on releasing didn't get turned up as expected. We will be working to have this additional circuit in place this morning.... -rick
We did manage to increase capcity here at mid-peak hours to satisfy demand on this pop tonight. However, there is still a circuit that isn't working properly. We'll be working with the telco to resolve this prior to peak tomorrow evening.....thanks for your patience. -rick
Gridnet is currently experiencing routing/throughput problems at the MAE-East network access point. Users on Gridnet POPs may see sluggish response or odd routing to sites on other networks. This should not affect Gridnet POPs' users' access to MindSpring services (mail, news, etc.), which don't pass through MAE-East. -SC
Just as a reminder to everyone who's having trouble connecting... Be sure you do talk to tech support first. You're moving from Ascend Maxen to USRobotics chassis, and I know for a fact that, if you've had your modem 'tweaked and plucked' to work the best with the Maxen, then they'll probably have trouble when they talk to USR modems. Support can re-tweak them to make them work with our equipment, and that might serve to improve the situation. I've been at that end before, and it's amazing what one little entry in the init string can do. Cheers, Brian
As some of you have experienced, we're having strange disconnection problems in the Orlando POP. We're still at a loss as to what is causing this, but we are investigating as much as we can. Please, if you experience these problems, fill out one of our poor connections reports at http://nmc.mindspring.com/poorconns/connect.shtml Please include all requested information. Also, we might need a few people to test with in the coming week(s). Please let us know if you'd not mind being one of the people who help us test. Cheers, Brian
Our telco anticipates re-releasing these numbers to us sometime this afternoon. We will post again when we see actual resolution. We appreciate your patience. These are the kinds of problems we run into with a new PoP. -Amy
Customers transitioned from PSI to GridNet should have their DNS set to 207.69.188.185 and 207.69.188.186 NOT anything starting with a 38. The 38 ip addresses you might have been using before will no longer work now that you are no longer using PSI PoPs. -Amy
The Mindspring POPs in Charlotte, NC and Knoxville,TN will undergo maintenance Thursday morning from 2am-6am EDT. Users dialed in during the window may see their connections drop once, and there's a slight chance we could give busies. We will be upgrading the software in our modems and terminal servers.
We are investigating reports of telephone company recordings ("all circuits busy") from users in Chapel Hill who are calling 425-7500. Please call 929-2160 until further notice -- but if you are having problems with X2 and 425-7500 works better, use 425-7500. -SC
We found a hung modem causing rna, and had to knock off three users to fix the problem. We apologize for the inconvenience.
We accidentally dropped power to some equipment in this POP when doing other maintenance. Some users may have been dropped, others may have noticed slow connections or pauses in file transfer. We're making sure this won't happen again. -SC
I had to reboot a piece of our equipment in Orlando, knocking some folks offline. My apologies for having to take such a drastic step. Thanks for your understanding. Cheers, Brian
The PSI POP is back up. BUT... We have decided that this would be a wonderful occasion to move our Fort Myers customers to GridNet. Please change your access number to (941) 334-8877 and your DNS to 207.69.188.185 and 207.69.188.186. -SC
We found a few leftover problems in Greenville and Norfolk/Va. Beach that caused fast busies and drops -- they're fixed now. Sorry for the inconvenience... -SC
This POP is operating normally now. -SC
A UPS failed at this POP, taking some of our equipment's power with it. The equipment has been placed on an alternate power source and is back up. Downtime will be scheduled for replacement of the UPS shortly. Sorry for the inconvenience... -SC
We finished the maintenance succesfully. Thank you for your patience. -idzam
Looks like most of our users have gone to bed now since our utilization has decreased significantly. Also, we managed to restore our hardware that was giving problem earlier. We thank you for your patience. -idzam
PSI believes they've fixed the problem...I can get in as well. Please let us know if you have any more troubles. Cheers, Brian
We received the following announcement concerning emergency maintenance by BBN Planet: > On Monday October 13, 1997 between 3:00 AM and 6:00 AM EDT, Field Service > will repair the circuit on one of the T3 links between the Nyc1-br1 & > Washdc1-br2 backbone routers. This work is being done to correct a carrier > error detected by Network Engineering. Customers may experience minor > degraded service as traffic which may cross this area will route to alternate > paths.
This maintenance was completed.
We received the following announcement for maintenance in UUNet's network. This maintenance will take place on Monday Oct 13th at 0400. We will be upgrading routers in Raleigh Durham, NC, Los Angeles, CA. This upgrade will result in additional backbone capacity in order to meet expected customer demands. Customers in these cities may experience up to a 30 minute interruption of service as the work is performed.
Some outgoing routes advertised through UUNet recently experienced instability. (UUnet routers are generally in the alter.net domain, which I think may have been a separate network that UUnet eventually ate. The accuracy of this trivia is not guaranteed.) UUnet found that some of their routers had been drinking heavily and have since corrected the problem.
The offending equipment is back in service and nobody wasted any time getting back online. Thanks for your patience and understanding. Cheers, Brian
This problem is taken care of. Please let us know if you have any more difficulties. Cheers, Brian
The problem with Opelika has been found and resolved. Please let us know if you have any troubles with this site. Cheers, Brian
This problem has been rectified, they believe the POP should be stable now. Please let us know of any further problems. Cheers, Brian
Wow, Idzam and I posted an alert on the same exact outage at the same exact time... :-/ Norfolk is back. The telco was performing a major switch upgrade. It's still ongoing but they say it shouldn't affect us again. (OK, I have faith in that, I suppose...) Customers should be able to dial in normally. Thanks for your patience.
Our circuits in Norfolk, Va Beach are being hit and are emitting error alarms. Due to this, customers will have problems connecting to this PoP. Earlier, our circuits were down and were causing busies. We have contacted our circuit provider to fix the situation. We highly appreciate your patience. We will keep you posted. Thank you. -idzam
One of our 0600 circuits bounced earlier and left some fallout that briefly gave busy signals. We've found and neutralized the fallout. 0600 users should be able to connect normally now. We apologize for the inconvenience.
We have confirmed with PSI that the PoP is accessible again. We thank you for your patience. If you having problem, please contact our support. In the meantime, we will keep an eye on this pop, in case if it goes down, in which we hope won't happen again. -idzam
I apologize for the latency in our updates about this matter. New code was released by USR and put on our ISDN netservers in Nashville and elsewhere several weeks ago. I have heard of no problems affecting ISDN directly related to this new version of code since its deployment. The bugs we were seeing in earlier versions appear to have all been corrected. -Amy
The particular terminal server in question was not found, and the problem has not persisted. This is probably something we will see in the future until the code is upgraded again. The problems we will see from time to time are possible dropped connections followed by a brief period of busy signals as the terminal server reboots itself of its own accord. As users are moved over to our MSPG NY PoP this will cease to be a concern. Thank you for your patience. -Amy
Maintenance was completed. -Amy
This looks to have been a problem with our router. Our Network Engineering group is doing a post mortem on the router problem.
We completed this maintenance at 5:15. Sorry for the delay in posting. All mail services should be working.
We had to reset some of our equipment in our 0600 rotery that caused a few dropped connections. Everything is working normally now.
PSI will be doing maintenance on several routers in their network between midnight and 2 AM tonight. This maintenance will result in periods of routing instability, and will also result in MindSpring services (mail, news, irc, ftp, www.mindspring.com) being unreachable from PSI POPs for a period of 15 or so minutes sometime between 1:15 and 2 AM. Users of MindSpring and Gridnet POPs may see odd routing to sites on PSINet during the same time frame. -SC
This was effecting 1b connections only....and is now fixed. --rick
this problem is resolved, let us know if you see any further problems...... -rick
We have worked with our circuit provider and found a cure for the packet loss on this network. We will be watching this closely for a while, but please let us know if you are seeing any problems.
We had power problems in Raleigh which caused some conections to get dropped. When the power came back, a couple modems did not revive. These modems caused busies for a while but I have removed them from service now. Things look to be back to normal now.. Please report any further anomolies to support.
This problem was taken completely out of customer view. Please let us know if you see any further troubles here. -Amy
Maintenance was finished this morning. Thank you for your patience. -idzam
PSI had this PoP power restored. The dispathed technician replaced the broken UPS. We thank you for your patience. -idzam
Our telco updated us with this trouble report. They found a translation problem which made our lead number being dialed by our customers pointing to the wrong circuit. This was corrected and we thank you for your patience. -idzam
Due to some intense troubleshooting here, we've been giving busies here and there in Atlanta, and have knocked people offline in the process of trying to fix those busies. We now believe that we have conquered the busy signals, and will continue to monitor the site throughout the evening. Please let support know if you encounter problems in the Atlanta dialin related to busy signals and failures to connect. Cheers, Brian
This POP is back up. Thanks for your patience. Cheers, Brian
I was performing some maintenance here which shoulkd have been completely customer invisible. Another piece of equipment in the chassis I was working with had an unexpected response to my actions. Everything is now functioning properly and I apologize for the busies I caused. -Amy
We haven't seen any problems since this maintenance.. Everything should be back to normal. We apologize for any inconvenience this maintenance may have caused. Cheers, Brian
Upgrade were successful. Sorry if it inconvenienced anyone.
We found a problem with our authentication database and it has been corrected.
We found that this router had lost its dynamic routing ability. We have made a configuration change to this router that we believe will prevent this from happening again.
We are having a problem with our router that handle traffic from our New York, South California, North California, and Harrisburg. This will be seen as authentication failures if you are calling into the POPs or as an inability to get to any sites if you are already on line. We are investigating the problem and will post more info as we can.
upgraded, acting normally.....
The telco found a problem in the switch programming. The long and short of it is all of our circuits fell out of our call group. I have verified that all of our circuits are once again part of the 423-523-2586 call group. I apologize for the trouble.
PSI reported that a circuit at this POP was bouncing but has now stabalized. If the circuit bounces, it causes drops an retrains. If these are happening to you, please let us know.
With assistance from the telco, we resolved this problem about 30 mins ago. We'll continue to monitor throughout the day. We appologize for the length of time this took to get fixed. Thank you for your patience. --rick
We found a card in alarm here that was giving customers occasional busy signals. The modem card has been reconfigured, and is now accepting calls. These calls are rolling normally now. If you experience further busy signals, please let us know. -Rick
We found a pri card whose b channels were in an Out of Service state. Restoring them seems to have solved the ptoblem. We'll continue to monitor.... -rick
we managed to get this done without dropping any connections.... -rick
We replaced a problematic management card earlier today; this didn't affect users at all. Hopefully we will see no more odd behavior at this POP. -SC
PSI found a misoptioned terminal server here that was causing some ISDN calls to fail authentication. The problem has been corrected and the remainder of the POP is also being checked.
This maintenance was completed successfully by UUNET. Customers shouldn't have noticed much more than some slow connectivity. -SC
This maintenance was completed successfully... -SC
We briefly gave busy signals a bit earlier. We have the busy problem under control now, and will watch this throughout the night. -SC