Thursday, March 3, 2011

grep, od, and dos2unix - Need to fix some crappy files that won't grep?

Recently I had to write a program to extract about 2500 passwords from a Novell eDirectory. That was all well and good, until I sent said output of file to a user who mangled the format on a Windows Machine. The user opened it in Excel or Notepad (or something...who knows...) and then worked with it, and send it back to me to pull the users they'd left in the file out of the Novell eDirectory.

Well that file wasn't in a format grep liked. How long did it take me to figure that out...quite a while, because there's nothing that looks different about the file to the naked eye (or naked editor.)

After googling and feeling very greptarded, I came upon the solution. I identified that the file I previously had and the file I receieved back had different octal values (man od) - I then used dos2unix to reconvert the file to a format grep would work with... grep really didn't like grepping on the improperly formatted file. Very strange... Simple magic... it worked, and the day was saved.

Aside from that, Novell, Netware, and eDirectory are crap. Unfortunately the place I work is still utilizing several servers that run it. Everything else is switched to Microsoft Windows and Active Directory (and my division which runs UNIX (HP-UX and RHEL))