This article was planned before the creation of this blog. I tried to understand what programming is as an approach to problem-solving, rather than just as a tool. And I found out that the approach of programming reminds what happens when you mine gold (but the analogy is not the most obvious one you probably thought about). The article describes the gist of this point of view to programming and depicts the connection to mining.

The difference between programmers and common herd

Unlike the IT guys, like me and you, people aren't that excited with programming. Connecting and arranging components of an upcoming system on a blackboard is not an inspiring activity for them. They don't cheer when they manage to write a function in a curiously small number of lines. They have no possibility to settle back, satisfied, and smile, when the program works outright after the first compilation without need to debug it.

But the world is somehow arranged in such a way that these insensitive people nevertheless are those who pays us money! What programming is valued for by them is that it's a good way to solve problems. These problems are usually beyond those a human can handle, and you usually need a horde of zombies to tackle them.

However, problem solving is approached differently by those who can code and those who can't. The management of a non-software company usually needs to do something concrete, like being able to fill forms M-186 and P-337b through the web browser. After they find out that such an automated system would help to run the business, they assign their IT guys to the task. They actually don't care how beautiful the code will be, how much satisfaction will the IT crowd have. They have a problem, and they need a solution.

Sorting ten cards

Assume you are given the task to sort 10 cards... You could write code like this...

But I am sure that there's no one who calls himself a programmer and does it this way. The code's too inefficient, impossible to maintain and can't be written without help of an external program.

So you end up writing something like this (that's a Bubble sort):

But let's reckon the original task. It was, let me remind, to sort ten cards. But it happens, that to sort ten cards you literally have to write a somewhat complex program, which sorts any number of cards. You can't write 10-element sorting without writing any-number-of-element sorting!

In fact, the fact stated in previous paragraph explains the crucial difference that prevents others from understanding that programming is hard. They think like "what is so difficult in sorting ten cards?" "You could just ... what's the word?.. Um, ah! Hardcode the form M-186 onto the webpage and collect results into a database!" They can't understand that you need to create a program, which is close to processing any form, in order to do it only for two forms.

This is the evidence of the idea that programming a solution to a particular problem is always nearly as hard as programming the solution for a general one.

Boon or bane?

The downside of a tendency of "programming" to solve problems more general is that people don't understand how sorting ten items may be so hard. Books are written about it (reckon Don Knuth's "Sorting and Searching" TAoCP book?). Research is held about it (new sorting methods are still developed to solve ever increasing number of edge cases and to handle specifically ordered arrays effectively). But, hell, any kid can arrange ten cards in the order of increasing value! He can't devise an algorithm of doing it for N cards, but we didn't ask for N, did we? Thinking like this, most people just can't understand what's so hard, and seem to value the art of sorting less than it really deserves. Let alone the other crafts of programming.

It's of no wonder that scaling makes a program more complex, and thus changes the algorithm in a way more than just increasing an array's boundary. But scaling and generalizing are nevertheless different problems: while the former is about looking for effective solutions for greater inputs, the latter is just about a theoretical capability of a developed algorithm to solve a larger problem.

On the bright side is the possibility to recycle the waste of programming. Once you've written a sorting function for array of 10 cards, it takes you just to increase the array size to make your program sort a lot of cards. Of course, if you wrote an Ω(n²) sort, like I did in the example above, it won't scale that well, but we assume you picked a better solution to the initial problem. So now you have a program that can sort any array.

Creating such residue is inevitable. We have a good analogy in the other fields: the one who extracts oil should also extract the accompanying gas, build facilities to handle it and make the site protected from accidents that involve gas. In the very same way, a programmer is doomed to produce programs that are more general, and he should be aware that it's just more hard.

By the way, printing invitations in an automated way is a good sign that you're dealing with a programmer. Even if a programmer never encountered a task of such sort, he will always use a programmable way to print invitations.

When I was at school, I was assigned a task to print invitations to the prom. I never did anything like that before, but, being a programmer I managed to find a way to do this. I scanned and recognized a list of persons into Microsoft Excel, imported it to Microsoft Access database, and printed invitations as a series of nicely formatted database records with external background image.

Completely lame, but what else could I do back then?

This property has even been a source of jokes. Alex Exler in his "Notes of a programmer's bride" (oh, I re-read that and fixed the description that follows) describes a situation that a programmer spent a day to write a program that prints invitations to his wedding. The bride bragged that her husband has such a good and a very useful profession. The man she bragged to questioned if it was easier to just fill them manually in a text editor, and after bride's reply about "generalization", he astonished her with, "Yeah? So how many times are you going to get married?"

Conclusion

Back to a programmer's residue. Brooks in his famous "Mythical man-month" claims, however, that producing a reusable component takes three times greater effort than producing the non-reusable one tied to a specific task. Most programs that don't have such a clear abstraction as array sorting, are far more tricky to be scaled to greater inputs without effort.

But that does not contradict the main idea that you can't evade producing residue. The raw "general" residue programming leaves after solving a particular problem is useless, just like in gold mining. To concentrate the ore, you must put effort into doing it, and to get some product from the residue, you should work more. And that's what happens when we write programs. So, isn't programming a gold mining of sorts?