Wednesday, 2014-04-23

*** nowen (~nowen@99-174-92-191.lightspeed.tukrga.sbcglobal.net) has joined #wikid12:09
*** salik (45f6d450@gateway/web/freenode/ip.69.246.212.80) has joined #wikid15:20
salikhi nick.  you there?15:20
nowenyes15:20
salikok we made a bit more progress.  still not able to login but now seeing radius traffic on gateway/wikid server15:20
nowensweet15:20
salik10:54:41.856791 IP atldc02.tcprod.local.61814 > gtwl01.gain.tcprod.local.radius: RADIUS, Access Request (1), id: 0x08 length: 101 10:54:41.859380 IP gtwl01.gain.tcprod.local.radius > atldc02.tcprod.local.61814: RADIUS, Access Reject (3), id: 0x08 length: 3015:21
nowenis the user still enabled?15:21
saliklet me check15:21
salikand this is from the logs:  Access denied for ssiddiqi, domain code: 010030136114 client: /10.30.137.20215:22
salikuser is enabled.  staus is 115:22
nowenok15:23
nowenare the loggers still on debug?15:23
nowenthey get reset on a restart unless you specify otherwise15:24
salikyeah filter is set to debug15:24
nowenunder configure loggers too?15:24
salikso what do i need to do under Configure Loggers15:25
saliki never set that before15:25
nowencom.wikidsystems , com.wikidsystems.client.wClient, com.wikidsystems.server.wAuth and the radius logger to debug15:26
saliki dont see a radius logger.  I see com.wikidsystems.radius.access.WikidAccess4.  is that what u are talking about?15:28
nowenyeah do that one, but also under New logger, Select a current logger, add com.wikidsystems.radius.log.DBSvrLogImpl as debug15:29
salikok.  any changes in the startup logging configuration section?15:30
nowenno15:31
nowenjust try to login and it will be there15:31
nowenremember to reset these by restarting before going into production15:31
nowenor you will swamp the system with logs ;)15:31
salikok.  made changes.  will try to login again15:32
nowenok15:32
salik2014-04-23 11:34:53.721INFOcom.wikidsystems.radius.log.DBSvrLogImpl<12> Access-Request(1) LEN=101 10.30.137.202:61814 Access-Request by ssiddiqi Failed: AccessRejectException: Access Denied  2014-04-23 11:34:53.720INFOcom.wikidsystems.radius.access.WikidAccess4Access denied for ssiddiqi, domain code: 010030136114 client: /10.30.137.20215:41
nowencan you paste the whole thing after the passcode into pastebin.com?15:42
salikok15:42
salikhttp://pastebin.com/Qn624pHr15:45
nowencheck that you're still enabled.  Can you also double-check the shared secret b/w wikid and nps15:46
nowenI don't see any change in the logs after changing the loggers15:46
salikok.  status 1 means enabled?15:46
nowenyes15:46
salikok account is configured.15:46
salikenabled15:47
salikwhich screen do i go to check shared secret in wikid side?15:49
nowenNetwork client15:49
nowenyou have to modify the one for nps15:53
salikyeah i just entered the nps one again and will try logging in again.15:53
saliki think it was ok, but we shall see15:53
salikstill getting access denied15:55
salikwill get u the logs15:55
nowenand are  com.wikidsystems , com.wikidsystems.client.wClient, com.wikidsystems.server.wAuth all set to debug?15:56
nowennot sure why you don't see more in the logs15:56
salikok let me double check15:57
salikyeah all set to debug15:57
salikhttp://pastebin.com/rg5gVgdf15:58
nowenand that's the correct domain?15:59
nowenyou only have the one network client?15:59
salikyes16:00
salikwell actually, what is the definition of network client in wikid talk?16:01
nowenit would be the nps.16:01
salikyeah only 116:01
nowenit's like your other loggers aren't reporting at all16:01
nowenSource is None and substring is empty?16:02
salikhow do i check that16:03
nowenit's just on the logs page16:04
salikoh ok16:04
nowenthe default source is None, which means all ;-)16:04
salikoh ok16:04
salikon the network client section where I set shared secret, I have Assign Return Attribute set to user-name and a blank in the box after.  not sure if that is ok?16:08
nowendid you set it up on the NPS too?16:08
nowenI say delete it16:09
saliki dont see how i can delete it16:09
nowenyou may need to restart the server.  if so, you might want to save the loggers16:09
salikit was already selected by default16:09
nowenoh16:10
nowenif you add it, then you will see a table below the dropdown16:10
salikyeah it wasnt something i added16:11
nowenok16:11
salikso to save loggers, i select save current configuration as startup configuration"?16:12
nowenyes16:12
salikok restarting server and trying again16:12
nowenok16:13
nowenyou know you only need to run 'wikidctl restart', right?16:26
salikyeah.  thats what i did.  it looks like it didnt retain the logging changes I made even though I saved them16:28
nowenhmm16:28
salikill set them again and try again16:29
nowenok16:29
salikhttp://pastebin.com/L94RL2C916:34
nowennot much difference, is there?16:35
nowen:(16:35
saliknot really :(16:35
nowenis there anything in /opt/WiKID/log/radius.log?16:35
saliklet me check.  one sec16:36
salikno.  empty16:36
nowenhmm16:38
nowenwhat version of wikid is this?16:38
nowenthe latest, right?16:38
salikyes16:38
nowencan you edit a file in linux?16:41
saliksure16:41
nowenok16:42
nowenok - open /etc/WiKID/log4j.properties16:44
salikok i have it open16:44
nowencomment out the line "log4j.rootLogger=DEBUG, socketLogger"16:44
nowenand uncomment "#log4j.rootLogger=DEBUG, socketLogger,  A1"16:45
salikok done16:46
nowenok - restart wikid16:46
nowenand now the logs should go into /opt/WiKID/log16:46
salikok16:46
nowenin particular radius.log16:46
salikok16:46
saliki can share my desktop with you.  it might make it easier to troubleshoot.16:47
nowen maybe.  I run linux so not everything works16:48
saliki was going to use join.me.  it works fine in linux16:48
nowenlet's see the radius.log first, then16:48
salikok.  wikid has restarted.  i am trying to login now16:49
salikhttp://pastebin.com/eWfMHvuL16:52
nowenugh16:52
nowen10.30.137.202 is the NPS?16:52
salikyes16:53
nowenrun 'netstat -anp | grep 1812' on the server16:55
salikudp        0      0 0.0.0.0:1812                0.0.0.0:*                               26122/java16:57
nowenrun tcpdump port radius on the server and try to login16:58
salikhttp://pastebin.com/qBnCCP3917:00
nowentry 'tcpdump -vv port radius'17:01
salikand login again?17:02
nowenyes17:02
nowensend me a join.me invite17:05
saliki just emailed it to u17:07
nowenok - I'm in17:17
salikok17:17
nowenshow me the WiKIDAdmin - configure loggers17:18
salikit seems to reset it everytime i reboot17:18
nowendid you "Save currrent configuration as startup configuration"17:19
nowen?17:19
salikyeah17:19
salikand i clicked change config button17:19
nowenhmm17:19
nowenok - well set them to debug again17:19
saliklook ok?17:20
nowenyep17:20
nowenclick apply chabnges17:20
nowenand try to login17:21
salikok logging in from another system now17:21
nowenhmm Bad udp checksum17:23
salikok17:23
nowenshow me the wikidadmin logs17:23
nowenwait, why are there so many17:25
saliki tried 6 times until it kicked me out17:25
nowenoh17:25
nowenis the user enabled?17:25
nowenhit reload on that page17:26
saliki did17:26
saliku want to take a quick look at how my NPS is setup?17:29
nowenshow me the network client first17:31
nowenok17:31
nowenI wonder if we should try a simple shared secret17:32
salikok17:32
saliki can change it17:32
*** ddreggors (b85a8505@gateway/web/freenode/ip.184.90.133.5) has joined #wikid17:34
salikok shared secret is changed17:35
nowenyou need to restart wikid17:35
saliku want to look at anything else at NPS?  i kept pretty much the standard policies.  and just configured them to froward requests to wikid server17:36
salikshould i set these to debug again?17:38
nowenyes17:38
salikok trying to login now17:40
nowenok17:40
nowenjust once is all we need17:40
salikok done17:41
nowentesting something over here17:42
salikok17:42
nowenI'm at a loss17:49
nowenshow me your nps again17:50
nowenwait17:51
salikok17:51
nowengtwl01.gain.tcprod.local is what machine?17:51
salikthats the gateway/wikid server17:51
nowenso, I think this is not going to work17:52
nowenlooking at tcpdump: 12:59:05.377200 IP gtwl01.gain.tcprod.local.27459 > atldc02.tcprod.local.radius: RADIUS, Access Request (1), id: 0x68 length: 9117:52
nowenwhat is atldc02.tcprod.local.radius17:52
nowen?17:52
salikyes17:52
nowenis that AD?17:52
salikyes17:52
salikits a domain controller17:52
nowenok - show me the radius server on NPS17:53
salikwell the FQDN is atldc02.tcprod.local17:53
saliknot .local.radius17:53
nowenok17:53
nowenedit that17:54
nowenshow me the Authentication table17:54
nowenchec the Authenticator attribute17:55
nowenand turn off accounting17:55
ddreggorsadd shared secret now?17:56
salikyeah our shared secret is blank here17:56
nowen?17:56
nowenisn't that where you entered it?17:56
saliki entered it on Radius client window17:56
nowenoh17:56
nowenyeah, you need it here too17:56
salikok17:57
nowenI bet that was it17:57
salikok let me save and try again17:57
nowenok17:57
salikstill access denied17:58
saliklets go through the rest of the NPS settings to make sure it is all ok17:59
salikso i am forwarded requests to the wikid server here18:00
salikforwarding18:01
nowendid you use the new shared secret?18:01
salikyes18:01
saliki updated it at radius client and radius server18:02
nowenand wikid?18:02
salikand it is updated on wikid too18:02
nowenshow me the auth methods again18:02
salikin wikid?18:03
nowenno on nps18:03
nowenradius server18:03
nowenok - let me see the network policy settings18:04
nowenok show settings18:05
nowengo back to encrpytion18:06
nowenok18:07
nowenlogin again and show me the tcpdump18:09
salikok18:09
ddreggorsbad udp checksum18:10
nowenpastebin that for me18:10
ddreggorsctrl+c Salik18:11
salikok18:11
ddreggorsand copy all output18:11
salikhttp://pastebin.com/DWM9dwqA18:12
nowenI think the checksum error is an artifact of using tcpdump18:16
salikok18:16
nowenshow me the radius.log18:16
nowendid you have this working with just pam and wikid?18:20
saliknick,  do we need to update the passwords in the raddb/server file too?18:22
salikthe shared secret18:22
ddreggorsno I do not believe so, this is a new install and we have never authenticated with it18:22
noweneach pair is separate but must be the same18:22
salikok18:22
nowendoes the one in nps client match the one in /etc/raddb?18:23
salikwhat about pam_radius.conf18:23
salikno18:23
nowenso you have two pairs of shared secrets18:24
salikall we updated on wikid was through the web interface18:24
ddreggorsOEL 6 ships with pam_radius module that provides a pam_radius.conf18:24
nowenpam/nps and nps server/wikid18:24
noweneach one must be the same18:24
nowenjust set all four to the same thing ri18:24
nowennow18:24
nowenshow me /etc/raddb/server18:25
salikok everything is same pw now18:26
nowenok try logging in18:26
ddreggorstcpdump too18:26
ddreggorsok18:27
ddreggorsI was able to capture dump lol18:27
ddreggorslooks like it succeeded18:27
saliki think it worked18:27
nowenit did18:27
ddreggorsnice18:27
ddreggorsgreat job guys18:27
nowenyes18:28
ddreggorsSalik, go for NoMachine now18:28
saliknick,  i am trying with nomachine now18:28
nowenok18:28
ddreggorsshould be the same, it uses ssh18:28
ddreggorsit worked18:29
salikyeah that worked18:30
ddreggorsdo we always have to give a pass at Gnome?18:30
ddreggorsor was that because you did not log out and that was screen saver pass check?18:30
saliki will log out completely and see18:31
ddreggorsplease log completely out and reconnect witn NM18:31
ddreggorsok18:31
ddreggorsok18:32
ddreggorslooks good18:32
ddreggorsnow try Domain credentials without wikid in ssh18:32
ddreggorsor NM18:32
ddreggorsthat should not happen right?18:33
ddreggorswith just domain credentials and no wikid token18:34
salikso nick, we can still login with just active directory credentials18:34
salikbypassign wikid18:34
nowendepends on your pam config18:34
ddreggorsok we have sufficient and not required18:34
salikthis was the issue we were having before on the other server18:34
nowenshow me /etc/pam.d/sshd18:34
salikbut never resolved18:34
salikok18:34
ddreggorswe added top line only, the rest is default18:35
nowenchange it to required and try again18:35
nowenkeep a session open18:35
salikok18:36
salikok fails with AD creds18:36
salikwill try with wikid nw18:36
saliknow18:36
salikwikid fails now too18:37
nowentry include18:37
nowenoh18:37
nowenwhy did it fail on wikid?18:38
saliksays access denied18:38
nowentry again18:38
ddreggorsSalik, start process over with new token18:38
salikstill access denied18:40
nowenuser enabled?18:40
salikyes.  status 118:40
nowentry include rather than required18:41
nowenoh18:41
nowenprobably an NPS setting. you don't have a user account on the gw server right?18:41
salikno18:42
salikwe use winbind18:42
salikand use AD accounts to authenticate18:42
nowenfor auth? or acct?18:42
nowenI don't see winbind there18:42
ddreggorsfor auth18:43
nowendid you turn it off?18:43
ddreggors--> /etc/pam.d/password-auth:auth        sufficient    pam_winbind.so use_first_pass18:43
nowenis that for local or ssh or both?18:44
ddreggorsboth18:44
nowenwell18:44
ddreggorsthat is just one line18:44
ddreggorsSalik... run this "grep winbind /etc/pam.d/*18:45
nowenyeah, but if it is allowing AD auth...18:45
ddreggorsshouldn't it succeed if sshd/auth required passes though?18:46
ddreggorsand then only use acct after that18:46
nowenpam is odd18:47
ddreggorsok it is because of this:18:48
salikaccess still denied after changing that line to "include" btw18:48
ddreggorsauth       include      password-auth18:48
ddreggors3rd line18:48
ddreggorsin sshd18:48
ddreggorsSalik try requisite18:49
ddreggorsrather than required18:49
ddreggorssorry out now18:50
salikso u are saying to change include to requisite?18:51
ddreggorsyes18:51
ddreggorsit is still going to include though, but a fail should not process the include18:52
ddreggorsbasically, if pass... include, if fail stop here and send fail18:52
salikwikid fails18:52
ddreggorsyeah18:52
salikAD fails18:53
ddreggorsof course, you never reach winbind now18:53
ddreggorsrequisite will not include password-auth which does winbind18:53
ddreggorsonly if pass with it do include18:54
ddreggorsput back to sufficient18:54
salikok wikid works now18:55
ddreggorslook at wikid logs18:56
nowen comment out password auth as an option and I bet AD will no longer work18:56
ddreggorsso questions is why does wikid succeed when sufficient?18:56
ddreggorsand not when required or requisite?18:57
ddreggorsSalik comment password auth include line18:57
ddreggorstry wikid now18:58
salikok AD doesnt work.  trying wikid now18:58
salikwikid works18:59
ddreggorsyeah...18:59
ddreggorsnot sure that is what we want, but it works as expected18:59
nowennot sure why sufficient is required, but it's between pam and pam-radius19:00
nowenmost likely it is a pam-radius issue.  that module is old19:03
salikok.  i think we are on the right track now.  we will do some more testing and get back to you if we need help19:03
nowenok19:03
salikwe will have some more team members test it for functionality and then we will probably roll this out to our other users next week19:04
nowenok19:04
salikwe will go back and change all 4 shared secret as well19:05
salikthanks again for the help19:05
nowennp19:05
nowenwill you give Talal the update?19:05
salikyeah19:05
nowencool19:05
saliknick, you there?19:57
nowenyes19:57
saliku had me uncomment some stuff in log4j.properties19:57
salikcan i change that back to how it was?19:57
salikor is it ok to leave as is19:58
nowenchange it back19:58
nowenalso reset your loggers to factory defualt19:58
salikok19:59
*** ddreggors has quit (Ping timeout: 240 seconds)20:31
*** nowen has quit (Quit: Leaving.)22:13

Generated by irclog2html.py 2.11.0 by Marius Gedminas - find it at mg.pov.lt!