blog dds

UML, Everywhere

flowchart, n.: The innumerate misleading the illiterate.

— Stan Kelly-Bootle, “The Devil’s DP Dictionary”

A mechanical engineer who sees the symbol ⊥ in a diagram will immediately realize that a feature is specified to be perpendicular to another. In contrast, a software engineer looking at a diagram’s line ending with the symbol ◊ will, at best, wonder whether it denotes aggregation (as in UML), or a “zero or one” cardinality (as in IDEF1X), or something else invented by a creative academic. Worse, many developers will simply scratch their head in bewilderment.

Continue reading "UML, Everywhere"

Code Documentation

Technical prose is almost immortal.

— Frederick P. Brooks, Jr.

Although programming is a form of literary expression, the relationship between code and its documentation is uneasy at best. As Figure 1 shows, among the thousands of projects that FreeBSD maintainers have considered important enough to port to the platform, the number of comments per 100 lines varies substantially. Clearly, as developers our views on how we should document our code are anything but consistent. Yet, there are universal principles, nifty tools, and useful practices that can benefit us all.

Continue reading "Code Documentation"

Ron Heifetz on Crisis Leadership

Earlier today I had the privilege to attend a lecture on crisis management by the Harvard Senior Lecturer Ron Heifetz. Here is a list of points that struck me (in the form of slightly edited tweets), and my view of their relevance to software development.

Continue reading "Ron Heifetz on Crisis Leadership"

Software Tracks

A generous car reviewer might praise a vehicle’s handling by writing that it turns as if it’s running on railroad tracks. Indeed, tracks offer guidance and support. When you run on tracks you can carry more weight, you can run faster, and you can’t get lost. That’s why engineers, from early childhood to old age, get hooked on trains. Can we get our software to run on tracks?

Continue reading "Software Tracks"

Basic Etiquette of Technical Communication

Parents spend years trying to teach their children to be polite, and some of us had to learn at school how to properly address an archbishop. Yet, it seems that advice on courteousness and politeness in technical communication is in short supply; most of us learn these skills through what is euphemistically called “on the job training.” With enough bruises on my back to demonstrate the amount and variety of my experience in this area (though not my skill), here are some of the things I’ve learned.

Continue reading "Basic Etiquette of Technical Communication"

Job Security

My colleague, who works for a major equipment vendor, was discussing how his employer was planning to lay off hundreds of developers over the coming months. “But I’m safe,” he said, “as I’m one of the two people in our group who really understand the code.” It seems that writing code that nobody else can comprehend can be a significant job security booster. Here’s some advice.

Continue reading "Job Security"

Software Architecture Challenges in the 21st Century

A wonderful workshop, titled Software Architecture Challenges in the 21st Century, took place at the University of Southern California on June 8th. The workshop was co-sponsored by IEEE Software, USC's Center for Systems and Software Engineering, and UC Irvine's Institute for Software Research (ISR). Here is my personal summary of the talks; the presentation slides are available online.

Continue reading "Software Architecture Challenges in the 21st Century"

Drawing Tools

1 Word = 1 Millipicture

— /usr/games/fortune

It’s no accident that in all engineering branches, our colleagues often communicate using drawings and diagrams. Given many artifacts’ scale and complexity, a drawing is often the best way to describe them. Uniquely, in software development we can easily derive pictures from code, and sometimes even code from pictures.

Continue reading "Drawing Tools"

Start With the Most Difficult Part

There’s not a lot you can change in the process of constructing a building. You must lay the foundation before you erect the upper floors, and you can’t paint without having the walls in place. In software, we’re blessed with more freedom.

Continue reading "Start With the Most Difficult Part"

Visualizing Revision Logs with a UML Sequence Diagram

How can you visualize the developers' contributions in a collaborative project? One way involves drawing timelines adorned with marks indicating each developer's contribution. This is a simple UML sequence diagram, a diagram that allows you to see the interactions of objects, but in this case the objects are the actual developers and the interactions are their contributions.

Continue reading "Visualizing Revision Logs with a UML Sequence Diagram"

Beautiful Architecture

What are the ingredients of robust, elegant, flexible, and maintainable software architecture? Over the past couple of years, my colleague Georgios Gousios and I worked on answering this question through a collection of intriguing essays from more than a dozen of today's leading software designers and architects.

Continue reading "Beautiful Architecture"

The World's Smallest Domain-Specific Language

Domain-specific languages, also known as little languages, allow us to express knowledge in a form close to the problem at hand. In contrast to general-purpose languages, like Java or C++, they are specialized for a narrow domain. Earlier today I wanted to initialize a rectangular array of Boolean values to represent the stick figure of a human. For that I devised a tiny domain-specific language (DSL) consisting of two symbols (representing an on and an off pixel) and wrote its commensurably simple interpreter.

Continue reading "The World's Smallest Domain-Specific Language"

The Value of Computing Paradigm Diversity

Today I wrote a combinatorial optimization algorithm to match members of pair programming teams according to the psychological traits of each pair's members. The program appeared to rearrange the initial random allocation of pairs in a way that might match my specifications. However, as I'll use this allocation for an experiment that I'll be able to perform only once, I realized that I wanted to carefully verify the results. How does one verify the operation of such a program?

Continue reading "The Value of Computing Paradigm Diversity"

A Look at Zero-Defect Code

The US National Security Agency has released a case study showing how to develop zero-defect code in a cost-effective manner. The researchers of the project conclude that, if adopted widely, the practices advocated in the case study could help make commercial software programs more reliable and less vulnerable. I examined a small part of the case study's code, and was not impressed.

Continue reading "A Look at Zero-Defect Code"

Saving the Editor's History

I recently spent a few days writing some tricky bit-twiddling code to implement a radix tree. I found myself making many programming mistakes, and I thought it would be interesting to study them, examine their contributing factors, and think how each of them could be prevented.

Continue reading "Saving the Editor's History"

The Way We Program

If the code and the comments disagree, then both are probably wrong.

— Norm Schryer

I can still remember the first time I laid my eyes on production-quality source code. This was in the early 1980s and the code was the BIOS listing of the original IBM PC. The 5940 lines of code spanned 80 neatly typeset pages in a three-ring slip-covered binder. Two things made a lasting impression on me. The first was the elation of being able to read, understand and learn from the code that made a real machine tick. This may have sparked my current practical and research interests in open source software. The second, was the way the code was commented. The BIOS was written in 8086 assembly language, and almost every line had a comment on its right hand side. By poring over the code and its comments I learned 8086 assembly, programming style, and understood the PC’s hardware architecture.

Continue reading "The Way We Program"

Open and Closed Source Kernels Go Head to Head

Earlier today I presented at the 30th International Conference on Software Engineering a research paper comparing the code quality of Linux, Windows (its research kernel distribution), OpenSolaris, and FreeBSD. For the comparison I parsed multiple configurations of these systems (more than ten million lines), and stored the results in four databases, where I could run SQL queries on them. This amounted to 8GB of data, 160 million records. (I’ve made the databases and the SQL queries available online.) The areas I examined were file organization, code structure, code style, preprocessing, and data organization. To my surprise there was no clear winner or looser, but there were interesting differences in specific areas.

Continue reading "Open and Closed Source Kernels Go Head to Head"

Software Builders

The tools and processes we use to transform our system’s source code into an application we can deploy or ship were always important, but nowadays they can mean the difference between success and failure. The reasons are simple: larger code bodies, teams that are bigger, more fluid, and wider distributed, richer interactions with other code, and sophisticated tool chains. All these mean that a slapdash software build process will be an endless drain on productivity and an embarrassing source of bugs, while a high-quality one will give us developers more time and traction to build better software.

Continue reading "Software Builders"

Assigning Responsibility

Over the past few days I worked over a large code body correcting various accumulated errors and style digressions. When I finished I wanted to see who wrote the original lines. (It turned out I was not entirely innocent.)

Continue reading "Assigning Responsibility"

Losing the War

The following gem comes from the user manual of the spanky-new Sony Ericsson K770i (3G) mobile phone. It demonstrates amply the state of the art in software engineering as of 2007.

Continue reading "Losing the War"

An Update on UMLGraph

Today I released version 5.0 of UMLGraph. This release contains a couple of important changes: six new shapes (components, nodes, collaborations, use cases, notes, and active classes), a facility for adding notes, and the hosting and packaging under its own domain name umlgraph.org.

Continue reading "An Update on UMLGraph"

Cooperative Development at the Speed of Light

The agility and responsiveness of major open-source endeavors never ceases to amaze me.

Continue reading "Cooperative Development at the Speed of Light"

Abstraction and Variation

“Master, a friend told me today that I should never use the editor’s copy-paste functions when programming,” said the young apprentice. “I thought the whole point of programming tools was to make our lives easier,” he continued.

The Master stroked his long grey beard and pressed the busy button on his phone. This was going to be one of those long, important discussions.

Continue reading "Abstraction and Variation"

The Tools we Use

It is impossible to sharpen a pencil with a blunt ax. It is equally vain to try to do it with ten blunt axes instead.

— Edsger W. Dijkstra

What’s the state of the art in the tools we use to build software? To answer this question I let over a period of a month a powerful server build from source code about seven thousand open-source packages. The packages I built form a subset of the FreeBSD ports collection, comprising a wide spectrum of application domains: from desktop utilities and biology applications to databases and development tools. The collection is representative of modern software, because, unlike say a random sample of sourceforge.net projects, these are programs that developers have found useful enough to spend effort to port to FreeBSD. The build process involves fetching each application’s source code bundle from the internet, patching it for FreeBSD, and compiling the source code into executable programs or libraries. Over the monthly period I also setup the operating system to write an accounting record for each one of the commands it executed. I then tallied up the CPU times of the 144 million records corresponding to the work in order to get a picture of how our software builds exploit the power of modern GHz processors.

Continue reading "The Tools we Use"

Of Wine and Software

Today, following the yearly IEEE Software board meeting, we visited the Casa Valduga winery near Bento Gonçalves at Rio Grande do Sul. The guide told us that the winery nowadays made all its wine in stainless steel vats, which have largely replaced the traditional oak barrels.

Continue reading "Of Wine and Software"

Silver Bullets and Other Mysteries

It seemed like a good idea at the time.

—Ken Thomson, on naming the Unix system call to create a file "creat"

When conference participants interrupt a speaker with applause, you know the speaker has struck a chord. This happened when Alan Davis, past editor in chief of IEEE Software, gave a talk on improving the requirements engineering process at the NASSCOM (Indian National Association of Software and Services Companies) Quality Summit in Bangalore in September 2006. He was explaining why a marketing team will often agree with developers on additional features and a compressed delivery schedule that both sides know to be unrealistic. The truth is that this places the two parties in a Machiavellian win-win situation. When the product's delivery is inevitably delayed, the developers will claim that they said from the beginning that they couldn't meet the schedule but that marketing insisted on it. The marketing people also end up with a convenient scapegoat. If the product launch is a flop, they can say they missed a critical marketing time window owing to the product's delay. Where else are we playing such games?

Continue reading "Silver Bullets and Other Mysteries"

Modularity and Troubleshooting

A residual current device trips leaving the house in the dark. How do I fix the problem?

Continue reading "Modularity and Troubleshooting"

Make vs Ant: Observability

I've long felt uncomfortable with ant as a build management tool. I thought that my uneasiness stemmed from the verbose XML used for describing tasks, and the lack of default dependency resolution. Today, email from a UMLGraph user struggling with a complex ant task made me realize another problem: lack of observability.

Continue reading "Make vs Ant: Observability"

Software Rejuvenation is Counterproductive

In the February issue of the Computer magazine Grottke and Trivedi propose four strategies for fighting bugs that are difficult to detect and reproduce. Retrying an operation and replicating software are indeed time-honored and practical solutions. When coupled with appropriate logging, they may allow an application to continue functioning, while also alerting its maintainers that something is amiss. On the other hand, the proposal to restart applications at regular intervals (rejuvenation as the authors call it), doesn't allow us to find latent bugs, sweeping them instead under the carpet. This lowers the bar on the quality we expect from software, and will doubtless result in a higher density of bugs and increasingly complicated failure modes.

Continue reading "Software Rejuvenation is Counterproductive"

A Peek at Beautiful Code

An exciting new book is about to hit the shelves, and I consider myself very lucky to be among its contributors. Beautiful Code, subtitled "leading programmers explain how they think", contains 33 chapters where contributors describe some code they consider noteworthy. Although I don't consider myself worthy of the book's subtitle, I love coding, and I'm extremely happy that code is taking the leading role among such an illustrious cast. Here is the complete table of the book's contents.

Continue reading "A Peek at Beautiful Code"

Open source as a paradigm for evolving complex systems

Scientists in the 1980s hotly debated the feasibility of US's proposed Strategic Defence Initiative, commonly known as Star Wars. One argument concerned the amount of software needed to control the missile detectors and weapons.

Continue reading "Open source as a paradigm for evolving complex systems"

Cracking Software Reuse

[Newton] said, "If I have seen further than others, it is because I've stood on the shoulders of giants." These days we stand on each other's feet!

— Richard Hamming

Sometimes we encounter ideas that inspire us for life. For me, this was a Unix command pipeline I came across in the '80s:

Continue reading "Cracking Software Reuse"

SQO-OSS Launched

Yesterday my research group and our partners officially launched the SQO-OSS research project: a Software Quality Observatory for Open Source Software.

Continue reading "SQO-OSS Launched"

Choosing a Collection: A Discussion with Kent Beck

Recently I reviewed the mansucript of Kent Beck's upcoming book Implementation Patterns. I will certainly put it in the list of books any professional programmer should read. When discussing collections (containers in C++ STL parlance), Kent mentions that his overall strategy for performance coding with collections is to use the simplest possible implementation at first and pick a more specialized collection class when it becomes necessary. My view is that we should choose the most efficient implementation from the start. With prepackaged collections this doesn't have any cost associated with it, and it avoids nasty surprises when a dataset increases beyond the size the programmer envisaged. I added a comment to that effect in my review, and later I sent him an email with a supporting citation, which kindled an interesting exchange. I reproduce our email exchange here, with his permission.

Continue reading "Choosing a Collection: A Discussion with Kent Beck"

The Verbosity of Object-Oriented Code

As I refactored a piece of code from an imperative to an object-oriented style I increased its clarity and reusability, but I also trippled its size. This worries me.

Continue reading "The Verbosity of Object-Oriented Code"

UML Class Diagrams from C++ Code

I needed a UML class diagram of the classes I use in the implementation of CScout refactoring browser. I drew the last such diagram on paper about four years ago, so it was definitely out of date. I always say that whenever possible documentation should be automatically generated from the code, so I decided to automate the task.

Continue reading "UML Class Diagrams from C++ Code"

NASSCOM Quality Summit 2006

Last week I attended NASSCOM's 2006 Quality Summit in Bangalore, India. There I gave a tutorial on tooling with open source software, and delivered a talk on Global Software Development in the FreeBSD Project. It was an edifying trip.

Continue reading "NASSCOM Quality Summit 2006"

SeaMonkey vs Internet Explorer Revisited

In an older blog entry I compared the complexity of Mozilla with that of Internet Explorer by looking at the components each linked to. Recently, comments to a Slashdot posting I made, noted that I was comparing the two products on different platforms and source with binary dependencies. I therefore set out to compare the binary dependencies of the two under Windows.

Continue reading "SeaMonkey vs Internet Explorer Revisited"

Choosing a Programming Language

A language that doesn't have everything is actually easier to program in than some that do.

— Dennis M. Ritchie

Computer languages fascinate me. Like a living person, each one has its own history, personality, interests, and quirks. Once you've learned one, you can use it again after years of neglect, and it's like reconnecting with an old friend: you can continue discussions from the point they were broken off years before. For a task I recently faced I adopted a language I hadn't used for 15 years, and felt enlightened.

Continue reading "Choosing a Programming Language"

Surprising Findings on Software Reuse

Kevin DeSouza and his colleagues in a recent article in the Communications of the ACM published some surprising findings regarding software reuse: reuse happens more by novices rather than by experts, within projects rather than across them, and in transient teams rather than permanent ones. The statement regarding the higher propensity of rookies to reuse compared to older professionals rang particularly true to my ears.

Continue reading "Surprising Findings on Software Reuse"

Code Quality: The Open Source Perspective

My new book Code Quality: The Open Source Perspective got published, three years after I started writing it. The book owes more to open source software than any of the books dealing with Linux, PHP, Apache, Perl or any other book covering a specific technology.

Continue reading "Code Quality: The Open Source Perspective"

Project Asset Portability

It's said that real computer scientists don't program in assembler; they don't write in anything less portable than a number two pencil. Joking aside, at the end of the 1970s, the number of nonstandard languages and APIs left most programs tied to a very specific and narrow combination of software and hardware. Entire organizations were locked in for life to a specific vendor, unable to freely choose the hardware and software where their code and data would reside. Portability and vendor independence appeared to be a faraway, elusive goal.

Continue reading "Project Asset Portability"

Version Control Talk Demystified

One indication of the importance an endeavor has in our lives is the vocabulary associated with it. If developers employ a tool or a method, inevitably they will come up with words to describe their corresponding work in an accurate and concise way. I recently heard a colleague describe version control systems (also formally known as configuration management tools) as boring. I hope that this dictionary will dispel this myth by documenting a rich technical and social vocabulary. If you don’t work with a VCS I believe this list will give you plenty of reasons to look at what these systems can do for you and your projects. On the other hand, if you already use a VCS I hope you will find ideas on how to use it more productively and how to improve your configuration management process. And, no matter to which group you belong to, I am sure you’ll find here some new words worth knowing.

Continue reading "Version Control Talk Demystified"

Version Control Systems

A source code control system [is] a giant UNDO key—a project wide time machine.

— A. Hunt and D. Thomas

Sane programmers don't write production code without the help of an editor and an interpreter or a compiler, yet I've seen many software projects limping along without using a version control system. We can explain this contrast if we think in terms of the increased startup costs and the delayed gratification associated with the adoption of a VCS. We humans typically discount the future, and therefore implementing version control in a project appears to be a fight against the human nature. It is true that you can't beat the productivity boost that compilers and editors have provided us, but four decades after punched card programming in assembly language has gone out of fashion we must look elsewhere to reap our next gains in efficiency. And if you or your project is not using a VCS, adopting one may well be the single most important improvement you can undertake.

Continue reading "Version Control Systems"

Tool Writing: A Forgotten Art?

Merely adding features does not make it easier for users to do things—it just makes the manual thicker. The right solution in the right place is always more effective than haphazard hacking.

— Brian W. Kernighan and Rob Pike

In 1994 Chidamber and Kemerer defined a set of six simple metrics for object-oriented programs. Although the number of object-oriented metrics swelled to above 300 in the years that followed, I had a case where I preferred to use the original classic metric set for clarity, consistency, and simplicity. Surprisingly, none of the six open-source tools I found and tried to use fitted the bill. Most tools calculated only a subset of the six metrics, some required tweaking to make them compile, others had very specific dependencies on other projects (for example Eclipse), while others were horrendously inefficient. Although none of the tools I surveyed managed to calculate correctly the six classic Chidamber and Kemerer metrics in a straightforward way, most of them included numerous bells and whistles, such as graphical interfaces, XML output, and bindings to tools like ant and Eclipse.

Continue reading "Tool Writing: A Forgotten Art?"

UMLGraph Update

I have updated the UMLGraph program to run under Java 1.5.

Continue reading "UMLGraph Update"

An Open Source Java Metrics Toolset Is Hard to Find

I spent many hours looking for a stand-alone open source metrics toolset for Java programs. Unfortunately, I was not able to find a single tool that would fit my (relatively modest) requiremets.

Continue reading "An Open Source Java Metrics Toolset Is Hard to Find"

Maintainability of the FreeBSD System

Last November Ioannis Samoladas and his colleagues published an article in the Communications of the ACM [1] that compared the maintainability of open-source versus-closed source projects. I applied the maintainability index [2] they used on the FreeBSD source repository following the code's maintainability over time, and comparing the maintainability of different modules. Here are the results.

Continue reading "Maintainability of the FreeBSD System"

The Tools at Hand

The tools we use have a profound (and devious!) influence on our thinking habits, and, therefore, on our thinking abilities.

— Edsger W. Dijkstra

With a shovel excavator a single operator can effortlessly move 720 tons of earth with a single movement; a VLSI fabrication plant allows a designer to create elaborate sub-micron structures. Without tools the thousands employed in a car factory are nothing, with tools they can assemble a car in 18 effort hours. Sometimes, tools can even subsume the importance of their operators. The violinist Ivry Gitlis, considered one of the most talented musicians of his generation, said of his Stradivarius: "I have a violin that was born in 1713. I don't consider it my violin. Rather, I am its violinist; I am passing though its life." Tools are clearly an important and defining element of any profession and activity: tools help us move boulders and atoms, tools help us reach the Moon and our soul.

Continue reading "The Tools at Hand"

Software Complexity: Open Source vs Microsoft

In a readable and interesting paper titled CyberInsecurity: the cost of a monopoly seven notable security experts argue that the Microsoft's near monopoly in the desktop operating system and office productivity markets is creating a dangerous monoculture that exacerbates the effect of security vulnerabilities.

Continue reading "Software Complexity: Open Source vs Microsoft"

Code Reading: The Open Source Perspective

In July 2000, while working on a paper on the use of slicing for choosing parts of an application to develop in a scripting language (don't ask), I found myself searching open-source programs for motivating examples, and experimenting with a tool for annotating the corresponding source code. At some point, a loud click sound in my mind brought to my attention the fact that although most books and courses teach us how to program, we actually spend most of our time reading code others have written. I reasoned that by applying my annotation tool on open source software I could write a book to present the ideas, techniques, and tools that go behind code reading.

Continue reading "Code Reading: The Open Source Perspective"