Monday, July 28, 2008
select random thoughts...
shane: who cares if she likes jason mraz, shes a girl shes supposed to like shitty music
me: yeah, but she said she didn't know what techno was ;(
shane: girls who like good music are usually ugly or whores
me: lol!!
me: i just told her that..and she agreed. thats a nice line of wisdom!
-- later
shane: if i ever get married i want to have a few best men, but at least you know you're the one that makes it ...
me: ..i think those are called Groomsmen ;p
me: if i ever get married, i wont have a best man, and my bitch of a wife wont have any bridesmaids either. im going to have a judge marry me in a court room with the scales of justice behind me.
shane: lol
OSX Server Woes - and I have to agree.
--
As someone that's cursed to administer an OS X Server machine, I have nothing good to say about Apple in general and OS X Server in particular. Apple's history of patching---or, in this case, not patching---stuff has been lukewarm at best and downright abysmal at worst. The Server 10.5.3 update introduced something that causes ClamAV to crash/reboot a Server machine when mail is turned on (since ClamAV is on by default. Nice one. They've had other stellar examples of their extreme lack of QA for their Server software, such as updating their included PHP to a version that was known to break Squirrelmail (the default webmail that comes with OS X Server), even though a fix had been available for months from the PHP maintainers.
I'm a huge fan of FreeBSD. I have been doing this OS X Server thing for more than two years now. I went in to it with an open mind, hoping that Apple wouldn't screw things up too badly. I was disappointed. The only things I've learned is that their Server QA is awful, they don't actually use their own Server software internally, their customer service is horrible when it comes to their Server stuff and their Server documentation is awful. I could rant about that for several pages. All of this leads me to believe that Apple really doesn't want to do well in the "server" segment of the market...Which is really too bad, cause they've finally got the hardware side of it to the point where there's not much separating them from most other low-end server vendors.
Now, that I've got that all that off my chest, Apple's dropped the ball on the BIND update. This is not surprising. Anyone that's administered OS X Server for any length of time probably feels the same way. It's so bad that I will suppress my OS X experience next time I am in the job market again; I hope to never work with OS X (particularly as a server) again and will do everything in my power to avoid doing so. I'm batting a thousand on persuading people interested in using OS X Server to use anything else...Apple really has to get things together or get out of the "server" market.
---
DVD/CD-Rom Drive problems ; Choppy Audio / Video
First off, go to:
Start -> Settings -> Control Panel -> System
This will open up a display. From here, on the top of this new window click on
"HARDWARE"
From under this tab, click on DEVICE MANAGER.
Another window should open up with several listings. Look for the one called
"IDE ATA/ATAPI controllers" or something very VERY similier to this.
Click on the + next to the name, a list of other things should drop down under it.
From in here, we ONLY want to look at
SECONDARY IDE CHANNEL
This is because, such things like your CD-ROM, DVD-ROM etc are connected through this.
Now, this is what you need to look for in the Secondary IDE Channel
Right click Secondary and select Properties
Another window will open up, look for the Tab named Advanced Settings and click on it.
Under this tab you should see two box's.
DEVICE 0 and DEVICE 1
NOTE: DEVICE 1 under CURRENT TRANSFER MODE might state it does not apply. If so, ignore it, if it does not state that it is not in use, then do the same for DEVICE 1 as you do for DEVICE 0
In Device 0 it should read out as following, there should not be anything stating PIO unless your computer is fairly old (like... 1999 or older.. it varies):
-----------------------------------------
DEVICE TYPE: Auto Detection
Transfer Mode: DMA if available
Current Transfer Mode: Ultra DMA Mode 5
-----------------------------------------
Now, Current Transfer Mode can say something different.
EX: Mode 1 is a slower form, and Mode 5 is very fast as in Ultra Quick. So it may display something a bit different, as long as it states it is in DMA mode it should be alright. Also, it may say something like.. "Multi-Word DMA Mode 2"
IF
TRANSFER MODE: "PIO Only"
switch that over to "DMA if available" and restart your computer.
Once restarted, come back in and continue the checks from this point
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
IF
-----------------------------
TRANSFER MODE: "DMA if available"
Current Transfer Mode: PIO Mode
-----------------------------
Then we have a problem, if this is TRUE, then click on the Driver Tab in this window at the top.
From here, Uninstall Drivers.
It will give you a warning, no worries, continue on. Once you restart your computer it will automatically detect the devices and reinstall everything for you.
Once done, try playing your DVD and see if it works. If not, go back in and make sure everything is the way it should be.
If not, sorry, you could try this again if you'd wish.
Causes for this changing over to PIO mode:
Disk Errors on the cd/dvd disk itself can cause this. For an example of what caused mine:
I've been playing Final Fantasy IX on my laptop using an emulator. Though, alot of the copies of DISK 2 for the game were bad copies and have corrupted files on it. I kept trying to play past these corrupted parts.
Sometimes, this repeated use to read the disk and it having to skip back and fourth trying to kicked it into another mode.. this usually happens after five or so of these. A dirty lense causing it to have plenty of trouble or dirty disk could also do the same thing.
Wednesday, July 23, 2008
when I see things like this it upsets me:
Hey man! God has blessed us both with a great job! How awesome is He! He is even allowing us to work together! God is good! Thanks for your help man!
wooooo yay 4 god...hes letting us work together!!!
I'm sure he held a gun to the hiring manager's head, too.
Monday, July 14, 2008
grep is a beautiful tool
July 13th, 2008 | Category: Productivity, Tools
Global Regular Expression Print is a staple of every command-line user’s toolbox. As with find, it derives a lot of power from being combined with other tools and can increase your productivity significantly.
Following is a simple tutorial that will help you realize the power of this simple and most useful command. If you are on Windows and haven’t already, download and install Cygwin. If you are also new to regular expressions (regex), here is a great regular expressions reference to get you started.
Tutorial
Suppose we want to search for duplicate functions in all of our JavaScript files. Let’s start basic and work up to it. This technique can be used to search for a TON of duplicate items like:
* Duplicate HTML IDs
* Check how many times a CSS class is used
* Duplicate java classes
* many, many more…
1.
# Search JS files in this directory for "function"
2.
grep function *.js
# Search JS files in this directory for "function"
grep function *.js
The above command will print the lines containing "function" in all JavaScript files in the current directory (NOT subdirectories). Printing out line contents would be much more helpful if we knew what files they come from and their line numbers:
1.
# Print filenames, line #s, and lines that start with "(white space)function"
2.
grep -EHn "^\s*(function \w+|\w+ \= function)" *.js
# Print filenames, line #s, and lines that start with "(white space)function"
grep -EHn "^\s*(function \w+|\w+ \= function)" *.js
Depending on how you format your JavaScript files, something like this will omit comments, anonymous functions, and also words like "functionality" giving you better results.
1.
# Print a list of: function
2.
grep -Eho "^\s*function \w+" *.js | sort
# Print a list of: function
grep -Eho "^\s*function \w+" *.js | sort
-o prints only the part that matches the regular expression. -E options gives me extended regex and -h suppresses printing of the file name. I am then piping to sort which just sorts the output so it a list of function
1.
# Print only duplicate function names
2.
grep -hEo "^\s*function \w+" *.js | sort | uniq -d
# Print only duplicate function names
grep -hEo "^\s*function \w+" *.js | sort | uniq -d
There we go! That will list only the duplcated functions. I know that we can expand this with awk or other stuff and get the file names and line numbers of the duplicates, but I don’t want to explaining the details of awk ;). I actually had it in this article and then removed it so leave a comment or contact me if you want the code for that.
Other Examples
1.
# Count the number of functions in all JS files
2.
grep -c function *.js
3.
4.
# Print lines that DO NOT have "function"
5.
grep -v function *.js
6.
7.
# List processes that match "pidgin" (non-Windows)
8.
ps -ef | grep pidgin
# Count the number of functions in all JS files
grep -c function *.js
# Print lines that DO NOT have "function"
grep -v function *.js
# List processes that match "pidgin" (non-Windows)
ps -ef | grep pidgin
Conclusion
grep is one of the most used command-line tools, often piped to for filtering output. Understanding it is essential to increasing productivity on the command-line. There is so much more to grep than what I’ve shown here, and it would be cool to see your best uses in the comments!
Saturday, July 12, 2008
"if you shop best buy you paid 2 much"
'Not only is this inaccurate but it is such a broad statement that it just sounds idiotic. Like any retailer their non sales price is usually higher than I would like but Best Buy has some very slick deals sometimes.'
---
Thats a load of tripe. Best Buy is never actually the best buy...but often it is like a 'pretty good buy that I could get somewhere else most likely for a bit less.'
Tuesday, July 8, 2008
Archive files on Linux...
Level: Intermediate Carlos Justiniano, Software Architect, Ecuity Inc. 08 Jul 2004 The loss of critical data can prove devastating. Still, millions of professionals ignore backing up their data. While individual reasons vary, one of the most common explanations is that performing routine backups can be a real chore. Because machines excel at mundane and repetitive tasks, the key to reducing the inherent drudgery and the natural human tendency for procrastination, is to automate the backup process. If you use Linux, you already have access to extremely powerful tools for creating custom backup solutions. The solutions in this article can help you perform simple to more advanced and secure network backups using open source tools that are part of nearly every Linux distribution. This article follows a step-by-step approach that is quite straightforward once you follow the basic steps. Let's begin with a simple, yet powerful archive mechanism on our way to a more advanced distributed backup solution. Let's examine a handy script called arc, which will allow us to create backup snapshots from a Linux shell prompt. Listing 1. The arc shell script
The arc script accepts a single file or directory name as a parameter and creates a compressed archive file with the current date embedded into the resulting archive file's name. For example, if you have a directory called beoserver, you can invoke the arc script, passing it the beoserver directory name to create a compressed archive such as: beoserver.20040321-014844.tgz The use of the Listing 2. Archiving the beoserver directory
This simple backup example is useful; however, it still includes a manual backup process. The industry's best practices recommend backing up often, onto multiple media, and to separate geographic locations. The central idea is to avoid relying entirely on any single storage media or single location. We'll tackle this challenge in our next example, where we'll examine a fictitious distributed network, illustrated in Figure 1, which shows a system administrator with access to two remote servers and an offsite data storage server. Figure 1. Distributed network The backup files on Server #1 and #2 will be securely transmitted to the offsite storage server, and the entire distributed backup process will occur on a regular basis without human intervention. We'll use a set of standard tools that are part of the Open Secure Shell tool suite (OpenSSH), as well as the tape archiver (tar), and the cron task scheduling service. Our overall plan will be to use cron for scheduling, shell programming and the tar application during the backup process, OpenSSH secure shell (ssh) encryption for remote access, and authentication, and secure shell copy (scp) to automate file transfers. Be sure to review each tool's man page for additional information. Secure remote access using public/private keys In the context of digital security, a key is a piece of data which is used to encrypt or decrypt other pieces of data. The public and private key scheme is interesting because data encrypted with a public key can only be decrypted with the associated private key. You may freely distribute a public key so that others can encrypt the messages they send you. One of the reasons that public/private key schemes have revolutionized digital security is because the sender and receiver don't have to share a common password. Among other things, public/private key cryptography has made e-commerce and other secure transactions possible. In this article, we'll create and use public and private keys to create a highly secure distributed backup solution. Each machine involved in the backup process must be running the OpenSSH secure shell service (sshd) with port 22 accessible through any intermediate firewall. If you access remote servers, then there is a good chance you're already using secure shell. Our goal will be to provide machines with secure access without requiring the need to manually provide passwords. Some people think that the easiest way to do this is to set up password-less access: do not do this. It is not secure. Instead, the approach we'll use in this article will take perhaps an hour of your time, set up a system which gives all the convenience of "passphraseless" accounts -- but is recognized as being highly secure. Let's begin by ensuring that OpenSSH is installed and proceed to check its version number. At the time this article was written, the latest OpenSSH release was version 3.8, released on February 24, 2004. You should consider using a recent and stable release, and at the very least use a release which is newer than version 2.x. Visit the OpenSSH Security page for details regarding older version-specific vulnerabilities (see the link in Resources later in this article). At this point in time, OpenSSH is quite stable and has proven to be immune to many of the vulnerabilities which have been reported for other SSH tools. At a shell prompt, type If ssh returns a version number greater than 2.x, the machine is in relatively good shape. However, it is recommended that you use the latest stable releases of all software, and this is especially important for security-related software. Our first step is to log in to the offsite storage server machine using the account, which will have the privilege of being able to access servers 1 and 2 (see Figure 1). Once logged on to the offsite storage machine, use the ssh-keygen program to create a public/private key pair using the -t dsa option. The -t option is required, and is used to specify the type of encryption key we're interested in generating. We'll use the Digital Signature Algorithm (DSA), which will enable us to use the newer SSH2 protocol. See the ssh-keygen man page for more details. During the execution of ssh-keygen, you'll be prompted for the location where the ssh keys will be stored before you're asked for a passphrase. Simply press enter when asked where to save the key and the ssh-keygen program will create a hidden directory called .ssh (if one doesn't already exist) along with two files, a public and private key file. An interesting feature of ssh-keygen is that it will allow you to simply press enter when prompted for a passphrase. If you don't supply a passphrase, then ssh-keygen will generate keys which are not encrypted! As you can imagine, this isn't a good idea. When asked for a passphrase, make sure to enter a reasonably long string message which contains alphanumeric characters rather than a simple password string. Listing 3. Always choose a good passphrase
Because the .ssh directory which ssh-keygen creates is a hidden "dot" directory, pass the -a option to the ls command to view the newly created directory: Enter the hidden .ssh directory and list the contents: We now have a private key (id_dsa) and a public key (id_dsa.pub) in the hidden .ssh directory. You can examine the contents of each key file using a text editor such as vi or emacs, or simply by using the less or cat commands. You'll notice that the contents consist of alphanumeric characters encoded in base64. Next, we need to copy and install the public key on servers 1 and 2. Do not use ftp. Rather, use the secure copy program to transmit the public keys onto each of the remote machines: Listing 4. Installing the public keys on the remote servers
After we install the new public keys, we'll be able to sign on to each machine using the passphrase we specified when creating the private and public keys. For now, log in to each machine and append the contents of the offsite.pub file to a file called authorized_keys, which is stored in each remote machine's .ssh directory. We can use a text editor or simply use the cat command to append the offsite.pub file's contents onto the authorized_keys file: Listing 5. Add offsite.pub to your list of authorized keys
The next step involves employing a bit of extra security. First, we change the access rights for the .ssh directory so that only the owner has read, write, and execute privileges. Next, we'll make sure that the authorized_keys file can only be accessed by the owner. And finally, we'll remove the previously uploaded offsite.pub key file, since it's no longer required. It's important to ensure that access permissions are properly set because the OpenSSH server may refuse to use keys which have non-secure access rights. Listing 6. Changing permissions with chmod
After completing the same process on server2, we are ready to return to the offsite storage machine to test the new passphrase type access. >From the offsite server you could type the following: Use the Automating machine access using ssh-agent The ssh-agent program acts like a gatekeeper, securely providing access to security keys as needed. Once ssh-agent is started, it sits in the background and makes itself available to other OpenSSH applications such as ssh and scp programs. This allows the ssh program to request an already decrypted key, rather than asking you for the private key's secret passphrase each time it's required. Let's take a closer look at ssh-agent. When ssh-agent runs it outputs shell commands: Listing 7. ssh-agent in action
We can instruct the shell to execute the output commands which ssh-agent displays using the shell's eval command: The The ssh-agent has now become a background process which is visible using the Now we're ready to share our passphrase with ssh-agent. To do so, we must use a program called ssh-add, which adds (sends) our passphrase to the running ssh-agent program. Listing 8. ssh-add for hassle-free login
Now when we access server1, we're not prompted for a passphrase: If you're not convinced, try removing ( Simplifying key access using keychain So far, we've learned about several OpenSSH programs (ssh, scp, ssh-agent and ssh-add), and we've created and installed private and public keys to enable a secure and automated login process. You may have realized that most of our setup work only has to be done once. For example, the process of creating the keys, installing them, and getting ssh-agent to execute via a .bash_profile only has to be done once per machine. That's the really good news. The less than ideal news is that ssh-add must be invoked each time we sign on to the offsite machine and ssh-agent isn't immediately compatible with the cron scheduling process which we'll need to automate our backups. The reason that cron processes can't communicate with ssh-agent is that cron jobs are executed as child processes by cron and thus do not inherit the Fortunately, there is a solution which not only eliminates limitations associated with ssh-agent and ssh-add, but also allows us to use cron to automate all sorts of processes requiring secure passwordless access to other machines. In his 2001 three-part developerWorks series, OpenSSH key management (see Resources for a link), Daniel Robbins presented a shell script called keychain, which is a front-end to ssh-add and ssh-agent and which simplifies the entire passwordless process. Over time, the keychain script has undergone a number of improvements and is now maintained by Aron Griffis, with a recent 2.3.2-1 release posted on June 17, 2004. The keychain shell script is a bit too large to list in this article because the well-written script includes lots of error checking, ample documentation, and a generous serving of cross-platform code. However, keychain can be quickly downloaded from the project's Web site (see Resources for a link). Once you download and install keychain, using it is remarkably easy. Simply log in to each machine and add the following two lines to each .bash_profile: The first time you log back in to each machine, keychain will prompt you for the passphrase. However, keychain won't ask you to reenter the passphrase on subsequent login attempts unless the machine has been restarted. Best of all, cron tasks are now able to use OpenSSH commands to securely access remote machines without requiring the interactive use of passphrases. Now we have the best of both worlds, added security and ease of use. Listing 9. Initializing keychain on each machine
Our next task is to create the shell scripts, which will perform the necessary backup operations. The goal is to perform a complete database backup of servers 1 and 2. In our example, each server is running the MySQL database server and we'll use the mysqldump command-line utility to export a few database tables to an SQL import file. Listing 10. The dbbackup.sh shell script for server 1
On server 2, we'll place a similar script which backs up the unique tables present in the site's database. Each script is flagged as executable using: With a dbbackup.sh file on servers 1 and 2, we return to the offsite data server, where we'll create a shell script to invoke each remote dbbackup.sh script prior to initiating a transfer of the compressed (.tgz) data files. Listing 11. backup_remote_servers.sh shell script for use on the offsite data server
The backup_remote_servers.sh shell script uses the ssh command to execute a script on the remote servers. Because we've set up passwordless access, the ssh command is able to execute commands on servers 1 and 2 remotely from the offsite server. The entire authentication process is now handled automatically, thanks to keychain. Our next and final task involves scheduling the execution of the backup_remote_servers.sh shell script on the offsite data storage server. We'll add two entries to the cron scheduling server to request execution of the backup script twice per day, at 3:34 am and again at 8:34 pm. On the offsite server invoke the crontab program with the edit ( The crontab invokes the default editor, as specified using the Listing 12. Crontab entries on the offsite server
A crontab line contains two main sections, a time schedule section followed by a command section. The time schedule is divided into fields for specifying when a command should be executed: Listing 13. Crontab format
You should routinely check your backups to ensure that the process is working correctly. Automating processes can remove unnecessary drudgery, but should never be a way of escaping due diligence. If your data is worth backing up, then it's also worth spot checking from time to time. Consider adding a cron job to remind yourself to check your backups at least once per month. In addition, it's a good idea to change security keys every once in a while, and you can schedule a cron job to remind you of that as well. Additional security precautions For added security, consider installing and configuring an Intrusion Detection System (IDS), such as Snort, on each machine. Presumably, an IDS will notify you when an intrusion is underway or has recently occurred. With an IDS in place, you'll be able to add other levels of security such as digitally signing and encrypting your backups. Popular open source tools such as GNU Privacy Guard (GnuPG), OpenSSL and ncrypt enable securing archive files via shell scripts, but doing so without the extra level of shielding that an IDS provides isn't recommended (see Resources for more information on Snort). This article has shown you how to allow your scripts to execute on remote servers and how to perform secure and automated file transfers. I hope you'll feel inspired to start thinking about protecting your own valuable data and building new solutions using open source tools like OpenSSH and Snort.
|
Monday, July 7, 2008
zenhabit: patience
from http://zenhabits.net/2008/07/15-tips-for-becoming-as-patient-as-job/
15 Tips for Becoming as Patient as Job
“Patience and fortitude conquer all things.” - Ralph Waldo Emerson
In the Old Testament, the story of Job showed a very faithful man whose faith is put to test, and shows an extreme example of perseverance through suffering … but in my mind, whenever I read Job’s story, I am struck by the man’s supreme patience.
While living a very faithful and righteous life, he nevertheless endured one infliction after another without ever cursing God’s name. I think most of us would have lost our patience and become frustrated and angry much earlier in the story.
While Job’s patience is legendary, I believe that even the most impatient of us can learn to be more patient with practice.
Personally, patience is something I’ve been cultivating for a long time. And while I often fail, I believe I’ve progressed over the years, and things that used to get me hot and bothered now just float past me. I still get upset, of course, but not nearly as much as I used to.
Here are some tips that might help you become more patient, with practice:
- Tally marks. This is the first strategy, if you have real problems with patience: start by simply keeping tally marks on a little sheet of paper every time you lose your patience. This is one of the most effective and important methods for controlling an impulse — by learning to become more aware of it. Once you become aware of your impulses, you can work out an alternative reaction.
- Figure out your triggers. As you become more aware of losing your patience, pay close attention to the things that trigger you to lose that patience. Is it when your co-worker does something particularly irritating? When your spouse leaves dirty dishes in the sink? When your child doesn’t clean up her mess? Certain triggers will recur more frequently than others — these are the things you should focus on the most.
- Deep breaths. When you first start to lose your patience, take a deep breath, and breathe out slowly. Then take another. And another. These three breaths will often do the trick, as your frustration will slowly melt away.
- Count to 10. This one really works. When you feel yourself getting frustrated or angry, stop. Count slowly to 10 (you can do this in your head). When you’re done, most of the initial impulse to yell or do something out of frustation will go away. Combine this with the breathing tip for even more effectiveness.
- Start small. Don’t try to become as patient as Job overnight. It won’t happen. Start with something small and manageable. Look for a trigger that only induces a mild impatience within you — not something that gets your blood boiling. Then focus on this, and forget the other triggers for now. Work on controlling your temper for that one trigger. If you can get this one under control, use what you learned to focus on the next small trigger. One at a time, and with practice, you’ll get there.
- Take a time out. Often it’s best just to walk away for a few minutes. Take a break from the situation, just for 5-10 minutes, let yourself calm down, plan out your words and actions and solution, and then come back calm as a monk.
- Remember what’s important. Sometimes we tend to get upset over little things. In the long run, these things tend not to matter, but in the heat of the moment, we might forget this. Stop yourself, and try to get things in perspective.
- Keep practicing. Every time a situation stretches your patience to dangerous thinness, just think of it as an opportunity to practice your patience. Because that’s what it take to become patient — practice, practice, more practice, and even more practice. And then some more. And the more you practice, the better you’ll get. So cherish these wonderful opportunities to practice.
- Visualize. This works best if you do it before the frustrating situation comes up. When you’re alone and in a quiet place. Visualize how you want to react the next time your trigger happens. How do you handle the situation? How do you look? What do you say? How does the other person react? How does it help your relationship, your life? Think about all these things, visualize the perfect situation, and then try to actually make that happen when the situation actually comes up.
- Remember that things can take time. Nothing good happens right away. If you expect things to happen at the snap of your fingers, you’ll get impatient every time. Instead, realize that things will take time, and this realization can help your patience tremendously.
- Teach. This is something that helps me a lot. I remember that no one is perfect, and that everyone has a lot to learn. Be patient, and teach others how to do things — even if you’ve tried before, it might be the 11th time when things click. And remember, none of us learn things on the first try. Find new ways to teach something, and you’re more likely to be successful.
- Find healthy ways to relieve frustration. Frustration can build up like steam in a pressure cooker, and if you don’t relieve that steam, you’ll explode. So find ways to relieve that frustration in a healthy way. Punching a pillow, going outside to a place where you’re all alone and yelling, exercise, kickboxing … these are just a few examples. Once you get that frustration out of your system, you usually feel better.
- Try meditation. You can’t meditate in the middle of a frustrating situation, usually, but often meditation can help you to learn to find a center of calm within yourself. Once you learn how to go to this calm place, you can go there when you begin to get angry. Meditation can also help you to be in the moment, instead of always wanting to get to the future, or instead of dwelling on the past and getting angry about it.
- Just laugh. Sometimes we need to remind ourselves that no one is perfect, that we should be enjoying this time with our loved ones, and that life should be fun — and funny. Smile, laugh, be happy. Doesn’t always work, but it’s good to remind yourself of this now and then.
- Just love. Instead of reacting with anger, teach yourself to react with love. Your child spills something or has a messy room or breaks your family heirloom? Your spouse yells at you or is cranky after work? React with love. It’s the best solution.
Friday, July 4, 2008
Bad Programmer
Bad Programmers
Solving your skillset problems
Signs that you are a bad programmer
1. Inability to reason about code
Reasoning about code means being able to follow the execution path ("running the program in your head") while knowing what the goal of the code is.
Symptoms
- The presence of "voodoo code", or code that has no effect on the goal of the program but is diligently maintained anyway (such as initializing variables that are never used, calling functions that are irrelevant to the goal, producing output that is not used, etc.)
- Executing idempotent functions multiple times (eg: calling the save() function multiple times "just to be sure")
- Fixing bugs by writing redundant code that overwrites the result of the faulty code
- "YoYo code" that converts a value into a different representation, then converts it back to where it started (eg: converting a decimal into a string and then back into a decimal, or padding a string and then trimming it)
- "Bulldozer code" that gives the appearance of refactoring by breaking out chunks into subroutines, but that are impossible to reuse in another context (very high cohesion)
Remedies
To get over this deficiency a programmer can practice by using the IDE's own debugger as an aide if it has the ability to step through the code one line at a time. In Visual Studio, for example, this means setting a breakpoint at the beginning of the problem area and stepping through with the 'F11' key, inspecting the value of variables--before and after they change--until you understand what the code is doing. If the target environment doesn't have such a feature, then practice in one that does.
The goal is to reach a point where you no longer need the debugger to be able to follow the flow of code in your head, and where you are patient enough to think about what the code is doing to the state of the program. The reward is the ability to identify redundant and unnecessary code, as well as how to find bugs in existing code without having to re-implement the whole algorithm from scratch.
2. Poor understanding of the language's programming model
Object Oriented Programming is an example of a language model, as is Functional or Declarative programming. They're each significantly different from procedural or imperative programming, just as procedural programming is significantly different from assembly or GOTO-based programming. Then there are languages which follow a major programming model (such as OOP) but introduce their own improvements such as list comprehensions, generics, duck-typing, etc.
Symptoms
- Using whatever syntax is necessary to break out of the model, then writing the remainder of the program in imperative/procedural style
- (OOP) Attempting to call non-static functions or variables in uninstantiated classes, and having difficulty understanding why it won't compile
- (OOP) Writing lots of "xxxxxManager" classes that contain all of the methods for manipulating objects that have little or no methods of their own
- (Relational) Treating the database as an object store by giving each table an identity column (or GUID) for the primary key, and possibly going as far as serializing the state of the object to a binary column
- (Functional) Creating multiple versions of the same algorithm to handle different types or operators, rather than passing high-level functions to a generic implementation
- (Functional) Manually caching the results of a deterministic function
- (Pure Functional) Using cut-n-paste code from someone else's program to deal with I/O and Monads
- (Declarative) Setting individual values in imperative code rather than using data-binding
Remedies
If your skills deficiency is a product of ineffective teaching or studying, then an alternative teacher is the compiler itself. There is no more effective way of learning a new programming model than starting a new project and committing yourself to use whatever the new constructs are, intelligently or not. You also need to practice explaining the model's features in crude terms of whatever you are familiar with, then recursively building on your new vocabulary until you understand the subtleties as well. For example:
Phase 1: "OOP is just records with methods"
Phase 2: "OOP methods are just functions running in a mini-program with its own global variables"
Phase 3: "The global variables are called fields, some of which are private and invisible from outside the mini-program"
Phase 4: "The idea of having private and public elements is to hide implementation details and expose a clean interface, and this is called Encapsulation"
Phase 5: "Encapsulation means my business logic doesn't need to be polluted with implementation details"
Phase 5 looks the same for all languages, since they are all really trying to get the programmer to the point where he can express the intent of the program without burying it in the specifics of how. Take functional programming as another example:
Phase 1: "Functional programming is just doing everything by chaining deterministic functions together"
Phase 2: "When the functions are deterministic, they don't need to be executed until the output is called for, and only for as much as needed. This is called Lazy Evaluation and Partial Evaluation"
Phase 3: "In order to support Lazy and Partial Evaluation, the compiler requires that I write functions in terms of how to transform a single parameter, sometimes into another function. This is called Currying"
Phase 4: "When all functions are curried, the compiler can choose the best execution plan by using a constraint solver"
Phase 5: "By letting a constraint solver figure out the mundane details, I can write programs by describing what I want, rather than how to give it to me"
3. Deficient research skills / Chronically poor knowledge of the platform's features
Modern languages and frameworks now come with an awesome breadth and depth of built-in commands and features, with some leading frameworks (Java, .Net, Cocoa) being too large to expect any programmer, even a good one, to learn in anything less than a few years. But a good programmer will search for a built-in function that does what they need before they begin to roll their own, and excellent programmers have the skill to break-down and identify the abstract problems in their task, then search for existing frameworks, patterns, models and languages that can be adapted before they even begin to design the program.
Symptoms
These are only indicative of the problem if they continue to appear in the programmer's work long after he should have mastered the new platform.
- Re-inventing or laboring without basic mechanisms that are built-into the language, such as events-and-handlers or regular expressions
- Re-inventing classes and functions that are built-into the framework (eg: timers, collections, sorting and searching algorithms)
- "Email me teh code, plz" messages posted to help forums
- "Roundabout code" that accomplishes in many instructions what could be done with far fewer (eg: rounding a number by converting a decimal into a formatted string, then converting the string back into a decimal)
- Persistently using old-fashioned techniques even when new techniques are better in those situations (eg: still writes named delegate functions instead of using lambda expressions for one-offs)
- Having a stark "comfort zone", and going to extreme lengths to solve complex problems with primitives
Remedies
A programmer can't acquire this kind of knowledge without slowing down, and it's likely that he's been in a rush to get each function working by whatever means necessary. He needs to have the platform's technical reference handy and be able to look through it with minimal effort, which can mean either having a hard copy of it on the desk right next to the keyboard, or having a second monitor dedicated to a browser. To get into the habit initially, he should refactor his old code with the aim of reducing its instruction count by 10:1 or more.
4. Inability to comprehend pointers
If you don't understand pointers then there is a very shallow ceiling on the types of programs you can write, as the concept of pointers enables the creation of complex data structures and efficient APIs. Managed languages use references instead of pointers, which are similar but add automatic dereferencing and prohibit pointer arithmetic to eliminate entire classes of bugs. They are still similar enough, however, that a failure to grasp the concept will be reflected in poor data-structure design and bugs that trace back to the difference between pass-by-value and pass-by-reference in method calls.
Symptoms
- Failure to implement a linked list, or write code that inserts/deletes nodes from linked list without losing data
- Allocating arbitrarily big arrays for variable-length collections and maintaining a separate collection-size counter, rather than using a linked list or other dynamic data structure
- Inability to find or fix bugs caused by performing arithmetic on pointers
- Modifying the dereferenced values from pointers passed as the parameters to a function, and not expecting it to change the values in the scope outside the function
- Making a copy of a pointer, changing the dereferenced value via the copy, then assuming the original pointer still points to the old value
- Serializing a pointer to the disk or network when it should have been the dereferenced value
- Sorting an array of pointers by performing the comparison on the pointers themselves
Remedies
A friend of mine named Joe was staying somewhere else in the hotel, but I didn't know which room number. I did, however, know which room his acquaintance, Frank, was staying in. So I went up there and knocked on his door and asked him, "Where's Joe staying?" Frank didn't know, but he did know which room Joe's co-worker, Theodore, was staying in, and gave me that room number instead. So I went to Theodore's room and asked him where Joe was staying, and Theodore told me that Joe was in Room 414. And that, in fact, is where Joe was.
Pointers can be described with many different metaphors, and the data structures you can build translated into many analogies. The above is a simple analogy for a linked list, and anybody can invent their own, even if they aren't programmers. The comprehension failure doesn't occur when pointers are described, so you can't describe them any more thoroughly than they already have been. It fails when the programmer then tries to visualize what's going on in the computer's memory and it gets conflated with their understanding of regular variables, which are very similar. It may help to translate the code into a simple story to help reason about what's going on, until the distinction clicks and the programmer can visualize pointers and the data structures they enable as intuitively as scalar values and arrays.
5. Difficulty seeing through recursion
The idea of recursion is easy enough to understand, but programmers often have problems imagining the result of a recursive operation in their minds, or how a complex result can be computed with a simple function. This makes it harder to design a recursive function because you have trouble picturing "where you are" when you come to writing the test for the base condition or the parameters for the recursive call.
Symptoms
- Hideously complex iterative algorithms for problems that can be solved recursively (eg: traversing a filesystem tree), especially where memory and performance is not a premium
- Recursive functions that check the same base condition both before and after the recursive call
- Recursive functions that don't test for a base condition
- Recursive subroutines that concatenate/sum to a global variable or a carry-along output variable, and aren't implementing tail recursion
- Apparent confusion about what to pass as the parameter in the recursive call, or recursive calls that pass the parameter unmodified
Remedies
Get your feet wet and be prepared for some stack overflows. Begin by writing code with only one base-condition check and one recursive call that uses the same, unmodified parameter that was passed. Stop coding even if you have the feeling that it's not enough, and run it anyway. It throws a stack-overflow exception, so now go back and pass a modified copy of the parameter in the recursive call. More stack overflows? Excessive output? Then do more code-and-run iterations, switching from tweaking your base-condition test to tweaking your recursive call until you start to intuit how the function is transforming its input. Resist the urge to use more than one base-condition test or recursive call unless you really know what you're doing.
Your goal is to have the confidence to jump in, even if you don't have a complete sense of "where you are" in the imaginary recursive path. Then when you now need to write a function for a real project you'd begin by writing a unit test first, and proceeding with the same technique above.
Signs that you are a mediocre programmer
1. Inability to think in sets
Transitioning from imperative programming to functional and declarative programming will immediately require you to think about operating on sets of data as your primitive, not scalar values. The transition is required whenever you use SQL with a relational database (and not as an object store), whenever you design programs that will scale linearly with multiple processors, and whenever you write code that has to execute on a SIMD-capable chip (such as modern graphics cards and video game consoles).
Symptoms
The following count only when they're seen on a platform with Declarative or Functional programming features that the programmer should be aware of.
- Performing atomic operations on the elements of a collection within a for or foreach loop
- Writing Map or Reduce functions that contain their own loop for iterating through the dataset
- Fetching large datasets from the server and computing sums on the client, instead of using aggregate functions in the query
- Functions acting on elements in a collection that begin by performing a new database query to fetch a related record
- Writing business-logic functions with tragically compromising side-effects, such as updating a user interface or performing file I/O
- Classes that open their own database connections or file handles and keep them open for their lifespan
Remedies
Funny enough, visualizing a card dealer cutting a deck of cards and interleaving the two stacks together by flipping through them with his thumbs can jolt the mind into thinking about sets and how you can operate on them in bulk. Other stimulating visualizations are:
- freeway traffic passing through an array of toll booths (parallel processing)
- springs joining to form streams joining to form creeks joining to form rivers (parallel reduce/aggregate functions)
- a newspaper printing press (coroutines, pipelines)
- the zipper tag on a jacket pulling the zipper teeth together (simple joins)
- transfer RNA picking up amino acids and joining messenger RNA within a ribosome to become a protein (multi-stage function-driven joins, see animation)
- the above happening simultaneously in billions of cells in an orange tree to convert soil, water and sunlight into orange juice (Map/Reduce on large distributed clusters)
If you are writing a program that works with collections, think about all the supplemental data and records that your functions need to work on each element and use Map functions to join them together in pairs before you have your Reduce function applied to each pair.
2. Lack of critical thinking
Unless you criticize your own ideas and look for flaws in your own thinking, you will miss problems that can be fixed before you even start coding. If you also fail to criticize your own code once written, you will only learn at the vastly slower pace of trial and error. This is the root of lazy thinking and egocentric thinking, so its symptoms seem to come from two different directions.
Symptoms
- "Business Rule Engines"
- Fat static utility classes, or multi-disciplinary libraries with only one namespace
- Conglomerate applications, or attaching unrelated features to an existing application to avoid the overhead of starting a new project
- Architectures that have begun to require epicycles
- Adding columns to tables for tangential data
- Inconsistent naming conventions
- "Man with a hammer" mentality, or changing the definitions of problems so they can all be solved with one particular technology
- Programs that dwarf the complexity of the problem they solve
- Pathologically and redundantly defensive programming ("Enterprisey code")
Remedies
Start with a book like Critical Thinking by Paul and Elder, work on controlling your ego, and practice resisting the urge to defend yourself as you submit your ideas to friends and colleagues for criticism.
Once you get used to other people examining your ideas, start examining your own ideas yourself and practice imagining the consequences of them. In addition, you also need to develop a sense of proportion (to have a feel for how much design is appropriate for the size of the problem), a habit of double-checking assumptions (so you don't overestimate the size of the problem), and a healthy attitude towards failure (even Isaac Newton was wrong, but we needed him to try anyway).
Finally, you must have discipline. Being aware of flaws in your plan will not make you more productive unless you can muster the willpower to correct and rebuild what you're working on.
3. Pinball Programming
When you tilt the board just right, pull back the pin to just the right distance, and hit the flipper buttons in the right sequence, then the program runs flawlessly with the flow of execution bouncing off conditionals and careening unchecked toward the next state transition.
Symptoms
- One Try-Catch block wrapping the entire body of Main() and resetting the program in the Catch clause (the pinball gutter)
- Using strings/integers for values that have (or could be given) more appropriate wrapper types in a strongly-typed language
- Packing complex data into delimited strings and parsing it out in every function that uses it
- Failing to use assertions or method contracts on functions that make assumptions about their arguments
- The use of Sleep() to wait for another thread to finish its task
- Switch statements, on non-enumerated values, that don't have an "Otherwise" clause
- Using Automethods or Reflection to invoke methods that are named in unqualified user input
- Setting global variables in functions as a way to return multiple values
- Classes with one method and a couple of fields, where you have to set the fields as the way of passing parameters to the method
- Multi-row database updates without a transaction
- Hail-Mary passes (eg: trying to restore the state of a database without a transaction and ROLLBACK)
Remedies
Imagine your program's input is water. It's going to fall through every crack and fill every pocket, so you need to think about what the consequences are when it flows somewhere other than where you've explicitly built something to catch it.
You will need to make yourself familiar with the mechanisms on your platform that help make programs robust and ductile. There are three basic kinds:
- those which stop the program before any damage is done when something unexpected happens, then helps you identify what went wrong (type systems, assertions, exceptions, etc.),
- those which direct program flow to whatever code best handles the contingency (try-catch blocks, multiple dispatch, event driven programming, etc.),
- those which pause the thread until all your ducks are in a row (WaitUntil commands, mutexes and semaphores, SyncLocks, etc.)
There is also a fourth, Unit Testing, which you use at design time.
Using these ought to become second nature to you, like putting commas and periods in sentences. To get there, go through the above mechanisms (the ones in parenthesis) one at a time and refactor an old program to use them wherever you can cram them, even if it doesn't turn out to be appropriate (especially when they don't seem appropriate, so you also begin to understand why).
Signs that you shouldn't be a programmer
The following may not have any remedies if you still suffer from them after taking a programming course in school, so you will stand a better chance of advancing your career by choosing another profession.
1. Inability to determine the order of program execution
Symptoms
a = 5
b = 10
a = b
print a
- You look at the code above and aren't sure what number gets printed out at the end
Alternative careers
- Electrician
- Plumber
- Architect
- Civil engineer
2. Insufficient ability to think abstractly
Symptoms
- Difficulty comprehending the difference between objects and classes
- Difficulty implementing design patterns for your program
- Difficulty writing functions with low cohesion
- Incompetence with Regular Expressions
- Lisp is opaque to you
- Cannot fathom the Church-Turing Thesis
Alternative careers
- Contract negotiator
- Method actor
3. Collyer Brothers syndrome
Symptoms
- Unwilling to throw away anything, including garbage
- Unwilling to delete anything, be it code or comments
- The urge to build booby-traps for defense against trespassers
- Unwilling to communicate with other people
- Poor organization skills
Alternative careers
- Antique dealer
- Bag lady
4. Dysfunctional sense of causality
Symptoms
- You seriously consider malice to be a reason why the compiler rejects your program
- When called on to fix a bug in a deployed program, you try prayer
- You take hidden variables for granted and don't think twice about blaming them for a program's misbehavior
- You think the presence of code in a program will affect its runtime behavior, even if it is never invoked
- Your debugging repertoire includes rituals like shining your lucky golf ball, twisting your wedding ring, and tapping the nodding-dog toy on your monitor. And when the debugging doesn't work, you think it might be because you missed one or didn't do them in the right order
Alternative careers
- Playing the slot machines in Vegas
