| *** joevano has quit (Quit: leaving) | 13:27 | |
| *** joevano (~joevano@bzflag/developer/JoeVano) has joined #wikid | 13:28 | |
| *** nowen (~nowen@99-174-92-191.lightspeed.tukrga.sbcglobal.net) has joined #wikid | 14:00 | |
| *** Troy (329b98a8@gateway/web/freenode/ip.50.155.152.168) has joined #wikid | 14:59 | |
| Troy | Good morning Nick | 14:59 |
|---|---|---|
| nowen | morning | 15:00 |
| Troy | We had a issue with our primary server yesterday evening.. the primary hit 100% CPU usage.. | 15:00 |
| nowen | ohh | 15:00 |
| nowen | what was doing it? java? | 15:01 |
| Troy | good news is that the wikid service failed over successfully to the secondary | 15:01 |
| Troy | Feb 26 17:24:41 hsvwikidp1 postgres[1144]: [1-1] LOG: could not receive data from client: Connection reset by peer | 15:01 |
| Troy | Feb 26 17:24:41 hsvwikidp1 postgres[1144]: [2-1] LOG: unexpected EOF on client connection | 15:02 |
| Troy | those are the last entries in the /var/log/messages before it hit 100% cpu | 15:02 |
| nowen | I'm guessing you haven't upgrade to the latest rpm? | 15:03 |
| Troy | we couldn't SSH or console to the VM once we hit 100%.. we had to shutdown the vm | 15:03 |
| Troy | no.. we have only updated our lab system.. i was planning the upgrade next week | 15:04 |
| Troy | this is the first time we've encountered this 100% cpu usage problem. i was surprised | 15:04 |
| nowen | yeah, me too | 15:04 |
| nowen | it could be a VM issue. | 15:05 |
| Troy | is there any other logs I can look into to find what happened? | 15:05 |
| nowen | I was seeing that on an older version of VirtualBox. | 15:05 |
| Troy | ok.. I think this is on VMWare | 15:05 |
| nowen | take a look at /opt/WiKID/tomcat/logs/catalina.err | 15:06 |
| Troy | ok | 15:06 |
| nowen | yeah, I was pretty sure you weren't using it, but there may be similarities | 15:06 |
| nowen | the latest RPM should fix any postgres issues | 15:06 |
| nowen | it doubles the number of connections | 15:07 |
| nowen | also, was someone using the WiKIDAdmin UI at the time? | 15:07 |
| Troy | I don't see the catalina.err.. just catalina.out | 15:08 |
| nowen | ok - you can check that one. | 15:08 |
| nowen | can you move up the upgrade? | 15:12 |
| Troy | i'm going to try .. we have a quiet period over the end of the month so it's difficult to make any changes to production system without special permission | 15:14 |
| nowen | gotcha | 15:14 |
| nowen | seems likely that it was a postgresql connection issue and that it would be fixed in the update | 15:15 |
| nowen | also, the UI is better for the user page and log page, returning fewer results per page, so hitting them doesn't slam the box | 15:16 |
| *** joevano_ (~joevano@bzflag/developer/JoeVano) has joined #wikid | 15:17 | |
| *** rudy7 (~rudy6@213.132.115.194) has joined #wikid | 15:18 | |
| nowen | morning joevano | 15:21 |
| *** joevano has quit (*.net *.split) | 15:21 | |
| Troy | wikid-server-enterprise-3.5.0.b1542-1.noarch.rpm is the latest build correct? | 15:22 |
| nowen | yes | 15:24 |
| nowen | have you got it? need the link? | 15:26 |
| nowen | nevermind - you do ;-) | 15:26 |
| Troy | yea.. i have it.. i'm going to install this latest build on the lab VM.. | 15:33 |
| *** rudy7 has quit (Quit: Nettalk6 - www.ntalk.de) | 15:58 | |
| *** nowen has quit (Quit: Leaving.) | 19:34 | |
| *** nowen (~nowen@99-174-92-191.lightspeed.tukrga.sbcglobal.net) has joined #wikid | 21:02 | |
| joevano_ | morning nowen ;-) | 22:59 |
| nowen | lol | 22:59 |
| joevano_ | found this window under everything as I was leaving for the day | 22:59 |
| joevano_ | guess it was a busy day | 23:00 |
| nowen | good on you. | 23:01 |
| nowen | nice to be busy | 23:01 |
| *** nowen has quit (Remote host closed the connection) | 23:04 | |
| *** nowen (~nowen@99-174-92-191.lightspeed.tukrga.sbcglobal.net) has joined #wikid | 23:04 | |
| *** nowen has quit (Client Quit) | 23:07 | |
| *** nowen (~nowen@99-174-92-191.lightspeed.tukrga.sbcglobal.net) has joined #wikid | 23:17 | |
| *** nowen has quit (Client Quit) | 23:21 | |
| *** Troy has quit (Quit: Page closed) | 23:44 | |
| *** joevano (~joevano@bzflag/developer/JoeVano) has joined #wikid | 23:44 | |
| *** joevano_ has quit (Ping timeout: 272 seconds) | 23:46 | |
Generated by irclog2html.py 2.11.0 by Marius Gedminas - find it at mg.pov.lt!