Creativity, Art, Software and Process

September 7, 2009 — Posted by Al Shalloway

I was recently surprised when someone said I had said software is not creative.  I was surprised because I can’t imagine myself ever saying that.  On reflection, I realized I had said software is not like art.  But I meant that it is not like art because art does not have a particular customer nor does it need to follow particular rules.  Art speaks to whoever it speaks to.  It comes from the artist and its message will hit various people differently.  Software has to satisfy the needs of the customer.  In other words, great artists write from their heart and soul and they provide value to whomever the art speaks to.  Great developers direct their work to specific people.

The other difference mentioned relates to form.  Art speaks to our senses.  Except for entertainment software, virtually all other software interacts with real world data.  Certain rules must be followed.

So, I do not consider software development art in that it has a particular purpose and needs to follow particular rules. However, these differences do not mean that software is not creative.  I used to have a close friendship with a well-known artist.  We used to spend hours talking about our creative process.  I think we both delighted in hearing how our creative process was very similar even though it manifested itself in different forms.  Both of us had experiences where our work just seemed to flow through us.  The ideas came from outside our minds, so to speak, and we were merely the vessel s in which they manifested themselves.  Both of us had different methods to set up this flow – but the freeing of the mind, so to speak, was the same.  I knew a sculpture who said the same thing – about how he had already seen his sculpture and now all he needed was to make it so others could see it.

There is no question that product development in general and the software development aspect of it is a very creative process. I believe that creativity, however, does not mean freedom from guidelines. Just the opposite.  In fact, I suspect this is true in art as well.  For example, you don’t see a Picasso half painted in his normal style and half painted in a real world style.  Pollock doesn’t change forms in the middle of the painting either.  A da Vinci isn’t half impressionist half realist.  Artists work within a certain self-imposed structure.  The art comes through and realizes itself in that structure.

In the software world, we all too often get bogged down by the realities of what has already been done. It is often easy to get some functionality in one’s product, but an altogether different level of difficulty in changing it (maintenance).  This is one reason we often see one brilliant programmer write a huge amount of functionality only to have an entire team have difficulty maintaining it later. They can never quite get it in their head what the original developer did.  Of course, the original developer himself may not be able to do this.  Many developers have had the experience of writing something that was very clear at the time only to have difficulty with changing it later.  The clarity that was there at one point diminishes over time.

Perhaps this is another difference between art and software.  With art, one can see it in its entirety.  With software, the interactions between the components are not in one’s senses but in one’s thoughts.  While art follows certain structures if the artist so chooses, I am of the opinion developers need to follow certain structures to allow their code to be maintainable.  

This does not mean there is any less creativity.  The creativity should not be in the form of the code but in the manifestation of the value of the code.  As Brian Kernighan remarked years ago – “Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. “  Seeing the result of your painting is much easier for an artist – although changing it may represent a different challenge.

I am of the opinion that using proper rules for programming and design lowers the noise of writing and maintaining code and therefore allows developers to be more creative. Knowing good rules of design enables me to avoid writing traps for myself and others and frees my mind to be creative.  I also don’t have to worry so much about the form to use.  That’s not the creative part of the software anyway.  The structure of the software is only a means, the creative part lies in its purpose.  Rules on the structure enable me to be more creative in the purpose.

There are also rules to follow in the workflow of building the software. I have found that being disciplined about creating test specifications before I actually write any code to be a useful workflow rule.  There are many others as well.  These include not doing too many things at once, removing delays between when I do something and when I get feedback on if I did the right thing.  So I would say that both construction and process rules are useful to me as a creator.

In other words, I do not view rules as a constraint I must overcome in order to be creative. I view the proper set of rules combined with the proper process in which to manifest my work actually can provide the stage for my creativity to come out.  This attitude provides good guidance for selecting these rules of structure and workflow.  If they assist me, they are good. If they don't, they are not good.  Unfortunately, there are many in this field that feel that  

One challenge we have in this industry is that rules and processes have so long been imposed on developers that they are sometimes considered synonymous with constraints and restraints.  I suggest they can be just the opposite.

Alan Shalloway
CEO, Net Objectives
Achieving Enterprise and Team Agility

Subscribe to our blog Net Objectives Thoughts Blog

Share this:

About the author | Al Shalloway

Al Shalloway is the founder and CEO of Net Objectives. With 45 years of experience, Al is an industry thought leader in Lean, Kanban, product portfolio management, Scrum and agile design. He helps companies transition to Lean and Agile methods enterprise-wide as well teaches courses in these areas.


Comments

There is art which is done for a given customer. The people who commissioned such art were often called "patrons" and the work of the artist was done under some form of "contract." This did not stop the art from being outstanding in many cases. However, I am sure there were many instances where the work created did not last into our time. Perhaps Open Source software is closest to the non-patron image of art that you described? Certainly, many people feel Open Source software is far superior to commercial software.
As to rules, most artists I know work within certain constraints once they chose their subject, media, style, etc. And not all art, as I alluded to just above, can be termed "great." Some is technically excellent but, otherwise, uninspired. You can buy this on street corners, often with black backgrounds. I did "modern"/"abstract" art as a kid in high school. Even won a prize once. It was all forgettable. Not bad, just not enduring in any way.
As to seeing art in its entirety, one could likely pick the results of civil engineering, as well. Indeed, almost any physical creation could be seen in its entirety compared to software. Though we can see the results of software in the displays and outputs it produce. We can certainly see all of the code. What we cannot see is what software is more than anything else and that is an instantiation of process. At heart of any system, no matter what language the code may be written in, is some representation of a process. At certain points in that process, things can be observed and outputs (some in permanent physical form) can be produced.
You use the word "purpose" and that's an important term. Connecting back to what I just said, I believe the purpose of most software is to instantiate some sort of process. Software's complexity comes from trying to do that since the process itself may not be understood well enough to implement it flawlessly. Most people cannot describe a process of any consequence in sufficient detail that another could, rotely, follow that description and produce an identical result. Certainly, the success of many processes that rely on people performing them often succeed when people need to, ad hoc, adapt to a situation they have not anticipated or encountered before (at least not in the same form as they are at that time).
The whole idea of some parts of AI research has been to design a system that could be as flexible as people in responding to unanticipated situations, much of which depends on both common and strong domain sense. The CYC project started at MCC many years ago by Doug Lenart and now continued at Cycorp (http://www.cyc.com/) has been developed to capture massive common knowledge and then reason with it. In very narrow domains, AI-based systems have been used commercially. (I worked on one in the early 80s to transform natural English sentences/phrases into commands to a database's report writer and it worked nearly perfectly. But the domain was well-bounded and any doubtful interpretation of the input request became an "I'm sorry but that request was not understood. Please try again."
I definitely agree that rules are not constraints merely to be overcome. Almost any artist/writer/musician/etc. I have ever spoken to or read about discusses how understanding known forms and studying the work of others is important in developing one's own skill and style as well as to avoid known problems in the artistic domain chosen. On the other hand, with this knowledge, these same people say that "breaking the rules" then moves you into new. creative forms. However, this "creativity" comes not from random, ad hoc expression, but from understanding when the boundaries are being pushed or crossed. Some people seem to do this almost from the start; others study for years. There are successes and failures in both.
I think our largest failure in software in this regard is the limited or no studying of other's work that occurs. Often people write software from scratch, having only their own prior school/work experience as a guide. Or, if they do maintain the work of others, focus intently on just enough to make the changes "work." Belady and Lehman's classic work on Software Evolution (based on the IBM System 360 experience, also the basis of Fred Brooks' The Mythical Man-Month) discusses the problem of the deterioration of design integrity if effort is not expended to maintain it. That requires (trying to at least) understanding what others have done in order to make changes/additions which improve, or at the very least do not deteriorate, that design. Today, agile development refers to this sort of activity as "refactoring."
So while the total result of creating software is not "art" in the physical sense that most people could appreciate, there are certainly aspects of what artists and good craftspeople do involved in creating software. I believe, though, that it is the process instantiation aspect of software that makes in uniquely different from most other acts of creation since, in the end, what software actually is, as opposed to the outputs it creates, is a process.

Scott:Thanks so much for this great comment. Agree totally.

 Alan Shalloway, CEO Net Objectives

Alan, you note that "both construction and process rules are useful to me as a creator", one of the rules you cite being "removing delays between when I do something and when I get feedback on if I did the right thing."

I want to call that rule out as applicable to art as well as programming.

I'm a programmer and now a manager of programmers, but also a photographer (once a working photojournalist, though much more amateur than artist). Nonetheless, I would class photography in the "art" category.

I grew up in the era of film, was never fond of darkroom work and was frugal, so from my earliest photos, I didn't see results for days or weeks after shooting. Shortening the feedback loop in learning to take great photographs is as important as it is in programming. At the point I became a working photographer and had a darkroom assistant who gave me negatives within hours of shooting and prints soon after, I improved much more quickly.

Thanks for the thoughts comparing the two spheres.

Creativity affects everything and it does influences everyone. This contribute a lot, in all major aspects. The level of creativity reflects on the one who created such thing. In all fields, clothing, IT especially programming, foods, entertainment, etc, creativity contributes a certain thing. Let us just take for example the current logo redesign of GAP. With nary an announcement, Gap revealed a brand new logo. Published on the website, this new emblem came with no fanfare. The reaction of fans was swift and severe. Now, Gap is standing by their brand new emblem however asking supporters to come up with a brand new one. Will this all-news-is-good-news approach help the Gap bump up sales?

Blog Authors

Al Shalloway
Business, Operations, Process, Sales, Agile Design and Patterns, Personal Development, Agile, Lean, SAFe, Kanban, Kanban Method, Scrum, Scrumban, XP
Cory Foy
Change Management, Innovation Games, Team Agility, Transitioning to Agile
Guy Beaver
Business and Strategy Development, Executive Management, Management, Operations, DevOps, Planning/Estimation, Change Management, Lean Implementation, Transitioning to Agile, Lean-Agile, Lean, SAFe, Kanban, Scrum
Israel Gat
Business and Strategy Development, DevOps, Lean Implementation, Agile, Lean, Kanban, Scrum
Jim Trott
Business and Strategy Development, Analysis and Design Methods, Change Management, Knowledge Management, Lean Implementation, Team Agility, Transitioning to Agile, Workflow, Technical Writing, Certifications, Coaching, Mentoring, Online Training, Professional Development, Agile, Lean-Agile, SAFe, Kanban
Ken Pugh
Agile Design and Patterns, Software Design, Design Patterns, C++, C#, Java, Technical Writing, TDD, ATDD, Certifications, Coaching, Mentoring, Professional Development, Agile, Lean-Agile, Lean, SAFe, Kanban, Kanban Method, Scrum, Scrumban, XP
Marc Danziger
Business and Strategy Development, Change Management, Team Agility, Online Communities, Promotional Initiatives, Sales and Marketing Collateral
Max Guernsey
Analysis and Design Methods, Planning/Estimation, Database Agility, Design Patterns, TDD, TDD Databases, ATDD, Lean-Agile, Scrum
Scott Bain
Analysis and Design Methods, Agile Design and Patterns, Software Design, Design Patterns, Technical Writing, TDD, Coaching, Mentoring, Online Training, Professional Development, Agile
Steve Thomas
Business and Strategy Development, Change Management, Lean Implementation, Team Agility, Transitioning to Agile
Tom Grant
Business and Strategy Development, Executive Management, Management, DevOps, Analyst, Analysis and Design Methods, Planning/Estimation, Innovation Games, Lean Implementation, Agile, Lean-Agile, Lean, Kanban