Shell scripting for software developers
In an open online edX course on Unix tools I was running over the spring with more than a thousand registered learners, I got asked for ideas on how shell scripts can be useful. This is an intriguing question, because the course focuses mainly on performing one-off tasks in the areas of software development, data engineering, and system administration, rather than automation through shell scripts. In response, I posted how shell scripting improves my personal productivity. Here’s my take on how shell scripts are employed in diverse software development tasks. I plan to post further installments on system administration and data analytics.
Continue reading "Shell scripting for software developers"
Shell scripting for personal productivity
In an edX course on Unix tools I am running these weeks, I got asked for ideas on how shell scripts can be useful. This is a very interesting question, because the course focuses mainly on performing one-off tasks in the areas of software development, data engineering, and system administration, rather than automation through shell scripts. Here’s how I’m using shell scripting to enhance my personal productivity. I’ll post further installments regarding software development and system administration.
Continue reading "Shell scripting for personal productivity"
So, you’re chairing a conference session?
Earlier today a first-time session chair asked me for tips on chairing his session. With conference presentations becoming shorter and more packed, I realized that the task of the session chair has become more demanding. Here’s some advice.
Continue reading "So, you’re chairing a conference session?"
How to avoid redoing manual corrections
Say you have an automated process to create a report, which you then have to polish by hand, because there are adjustments that require human judgment. After three hours of polishing, you realize that the report is full of errors due to a bug in the initial reporting process. Is there a way to salvage the three hours of work you put into it?
Continue reading "How to avoid redoing manual corrections"
How to Create Your Own Git Server
Although I'm a happy (also paying) user of GitHub's offerings,
there are times when I prefer to host a private repository
on a server I control.
Setting up your own Git server can be useful
if you're isolated from the public internet,
if you're subject to inflexible regulations,
or if you simply want features different from those offered by GitHub
(and other similar providers).
Setting up a Git server on a Unix (Linux, Mac OS X, *BSD, Solaris, AIX)
machine isn't difficult,
but there are many details to observe.
Here is a complete guide.
Continue reading "How to Create Your Own Git Server"
How to Calculate an Operation's Memory Consumption
How can you determine how much memory is consumed by a specific
operation of a Unix program?
Valgrind's Massif subsystem could help you in this regard,
but it can be difficult to isolate a specific operation from
Here is another, simpler way.
Continue reading "How to Calculate an Operation's Memory Consumption"
Importing an RCS Project into Git
RCS stands for Revision Control System.
You may have never heard it, because it's more than a quarter-century old;
describing it was published in 1985.
Although its commands are still available in most Unix distributions
and it's one of the easiest systems to use in a single-user scenario,
it is clearly showing its age when compared to more modern systems.
Here is how to move an existing project managed with RCS to the
21st century and Git,
while preserving all its history.
Continue reading "Importing an RCS Project into Git"
In a recent
NPR interview the journalist
described how I used a mind map to organize my work while I
served as Secretary General for Information Systems
at the Greek Ministry of Finance.
A number of people asked me for more details;
if you're interested read on.
Continue reading "Mind Mapping"
Choosing between people you want to invite to a function and people you
have to invite is sometimes difficult.
Say Alice wants to invite Tom, Dick, and Harry to a party, but she'd actually
prefer if Dick didn't show up.
Here's how Alice can send invitations by email from an email-capable
Unix system to achieve the desired result,
while covering her scheming with plausible deniability.
Continue reading "Pretend Invitations"
Synchronizing Thunderbird's Replied Flag
My main email client is Thunderbird, but I also use BlackBerry's and Apple's
all through the same IMAP account.
Disconcertingly, email messages to which I have
replied through the other clients don't show up in Thunderbird with the
"replied" () icon.
This means that when I browse my email using Thunderbird,
I waste time trying to remember whether I have responded
to a particular message.
Here is how I solved the problem.
Continue reading "Synchronizing Thunderbird's Replied Flag"
Sane vim Editing of Unicode Files
Being able to use plain alphabeitc keys as editing commands
is for many of us a great strength of the vi editor.
It allows us to edit without hunting for the placement of
the various movement keys on each particular keyboard,
and, most of the time,
without having to juggle in order to combine particular keys with
Continue reading "Sane vim Editing of Unicode Files"
However, this advantage can turn into a curse when editing files
using a non-ASCII keyboard layout.
When the keyboard input method is switched to another script
(Greek in my case, or, say, Cyrillic for others)
vi will stop responding to its normal commands, because it will
encounter unknown characters.
Here is how I've dealt with this problem.
Batch Files as Shell Scripts Revisited
Four years ago I wrote
about a method that could be used to have the Unix Bourne shell interpret
Windows batch files.
I'm using this trick a lot, because programming using the Windows/DOS
batch files facilities is decidedly painful, whereas the Bourne
shell remains a classy programming environment.
There are still many cases where the style of Unix shell programming
outshines and outperforms even modern scripting languages.
Continue reading "Batch Files as Shell Scripts Revisited"
Useful Polyglot Code
Four years ago I blogged about an
incantation that would allow the Windows command interpreter (cmd) to execute
Unix shell scripts written inside plain batch files.
Time for an update.
Continue reading "Useful Polyglot Code"
How to Get a Glowing Recommendation Letter
Students who do well in my courses often come to me asking for a
recommendation letter for graduate or postgraduate study.
I only write letters for students I know well
and I can honestly recommend, so some end up with a glowing
recommendation while others leave empty handed.
While I was drafting a few letters today,
it occurred to me that obtaining a good recommendation letter
is a lot easier if you've planned for it well in advance.
Continue reading "How to Get a Glowing Recommendation Letter"
Basic Etiquette of Technical Communication
Parents spend years trying to teach their children to be polite, and some of us had to learn at school how to properly address an archbishop. Yet, it seems that advice on courteousness and politeness in technical communication is in short supply; most of us learn these skills through what is euphemistically called “on the job training.” With enough bruises on my back to demonstrate the amount and variety of my experience in this area (though not my skill), here are some of the things I’ve learned.
Continue reading "Basic Etiquette of Technical Communication"
Tags for Bibliography References
I love writing my papers in LaTeX.
Its declarative style allows me to concentrate on the content,
rather than the form.
I even format the text according to the content,
keeping each phrase or logical unit on a separate line.
Many publishers supply style files that format the article according
to the journal's specifications.
Even better, over the years I've created
an extensive collection
I can therefore use BibTeX to cite works with a simple command,
without having to re-enter their details.
This also allows me to use style files
to format references according to the publisher's specification.
Yet, there is still the problem of navigating from a citation to
the work's details.
Here is how I solve it.
Continue reading "Tags for Bibliography References"
Unzipping Files in Order
Over the past couple of years I've enjoyed listening to the
audio edition of the
The material is superb
(although I occasionally get the feeling of listening to the
Voice of America),
the articles are read in a clear voice,
the data's encoding is plain MP3,
unencumbered by digital rights (restrictions) management silliness,
and the audio format is convenient to listen on the metro or while jogging.
Unfortunately, the articles in the audio edition's zip file are
haphazardly ordered, which, until today, marred the enjoyment of my listening.
Continue reading "Unzipping Files in Order"
In Presentations Less is More
A couple of months ago I prepared the slides for a paper I will present
30th International Conference on Software Engineering.
After reading Garr Reynolds's book
Presentation Zen: Simple Ideas on Presentation Design on Presentation Design and Delivery
I became enlightened, and I decided to redo the presentation from scratch,
creating less cluttered, more focused, and simpler slides.
Continue reading "In Presentations Less is More"
A Minute Minute Minder
Today I delivered the opening
at the 4th Panhellenic Conference on Computer Science Education.
For a number of reasons (more on that later) I wanted to keep track of
my progress during the presentation.
For this I put together a minute minder that displayed the
time from the presentation's start and the slide I should be in.
I could thus adjust my pace to finish as planned.
Continue reading "A Minute Minute Minder"
Curing MIDlet Bluetooth Disconnects
Over the last few days I've been writing a
to collect GPS coordinates and cell identifiers.
I'm doing this
in an effort to look at what algorithms might be needed
in order to implement something similar to Google's
Here is a Google Earth example of the data I'm collecting.
I reached a point where I was collecting all the information I needed,
but the program was often plagued by random disconnections of the Bluetooth
link to the GPS.
Continue reading "Curing MIDlet Bluetooth Disconnects"
Global Web Site Redirect
I recentry moved UMLGraph to its
own dedicated web site.
After the move a friend pointed out that all the links in
Martin Fowler's web page on
UmlSketchingTools would now stop working.
Continue reading "Global Web Site Redirect"
Cover Letters for Publications in LaTeX
I admit it.
Although I compose most of my journal publications in LaTeX,
I use Microsoft Word for writing the cover letters.
The university's letterhead is provided in Word, and setting
up and compiling a LaTeX document for a single text
page is not worth the trouble.
Continue reading "Cover Letters for Publications in LaTeX"
A Programmer's Bookshelf
A first year student at a nearby university wrote to me asking for
advice on becoming a hacker
definition, he clarified).
He sent me a laundry-list of 18 programming languages he aimed to learn
by the time he graduated, and asked for other recommendations.
I've learned a lot from reading books,
so I compiled two reading lists for him.
Continue reading "A Programmer's Bookshelf"
The Treacherous Power of Extended Regular Expressions
I wanted to filter out lines containing the word "line" or a double quote
from a 1GB file.
This can be easily specified as an extended regular expression,
but it turns out that I got more than I bargained for.
Continue reading "The Treacherous Power of Extended Regular Expressions"
The Power of Reusable GUI Elements
One can manipulate any graphical element of the
Squeak environment by bringing up
a rectangular set of icons representing actions that one can perform
on any object.
At first I found it cumbersome to have to go through the halo
in order to perform any action, like recoloring an object or
changing its name.
Later I saw that this method is incredibly powerful.
Continue reading "The Power of Reusable GUI Elements"
How to Embed Citations in Diagrams
Diagrams in scientific publications occasionally link to other elements
of the publication, such as bibliographic or section references.
Maintaining consistency between the citations in the diagram
and the publication can be tricky, but a small Perl script
can automate this process.
Continue reading "How to Embed Citations in Diagrams"
The Capacitor Plague Hits a Baby Monitor
Good baby monitors are hard to come by.
Many tend to be oversensitive, others switch from squelch mode to
transmission with a loud hiss that is more irritating than a baby's
We were very satisfied with a Tomy Walkabout Digital
1998 baby monitor, until the day it started emitting a squeaking
Continue reading "The Capacitor Plague Hits a Baby Monitor"
So Long as there's a Jingle in your Head, Television isn't Free
Yesterday I switched from an ancient version of the "free" Adobe Reader to
the current version 7.0.
I spent the morning studying some fairly tricky technical documents.
Within that interval I often caught my eyes glancing to the top right of the
Adobe Reader's display window where
an advert button flashed as it changed its content.
Needless to say, this change of focus interrupted my train of thought,
and got me out of "flow mode".
Continue reading "So Long as there's a Jingle in your Head, Television isn't Free"
White Noise Calms Babies
A week ago I told my colleague
that drying the body of a newborn baby with a hair drier keeps
it consistently dry avoiding rashes, and also calms the baby down.
Today he told me the advice worked wonders.
Many parents have discovered that sources of white noise, like the sound
of a vacuum cleaner or a hair drier, seem to calm down a baby's
Continue reading "White Noise Calms Babies"
UML Class Diagrams from C++ Code
I needed a UML class diagram of the classes I use in the implementation of
CScout refactoring browser.
I drew the last such diagram on paper about four years ago, so it was
definitely out of date.
I always say that whenever possible documentation should be automatically
generated from the code, so I decided to automate the task.
Continue reading "UML Class Diagrams from C++ Code"
Efficient Human Multitasking
I sometimes hear colleagues complaining that they can't get anything done,
because they have too many tasks in their head.
I've found that in order to increase the efficiency of my work
I need a moderately large selection of pending tasks.
This allows me to match the type of work I can do at a given moment
with a task in the most optimal way.
Continue reading "Efficient Human Multitasking"
Batch Files as Shell Scripts
Although the Unix Bourne shell offers a superb environment for combining
existing commands into sophisticated programs, using a Unix shell
as an interactive command environment under Windows can be painful.
Continue reading "Batch Files as Shell Scripts"
Debuggers and Logging Frameworks
As soon as we started programming, we found to our surprise that it wasn't as easy to get programs right as we had thought. Debugging had to be discovered.
— Maurice Wilkes discovers debugging, 1949
The testing, diagnostic, and repair equipment of many professions is horrendously expensive. Think of logic analyzers, CAT scanners, and dry docks. For us the cost of debuggers and logging frameworks is minimal; some of them are even free. All we need to become productive, is to invest some time and effort to learn how to use these tools in the most efficient and effective way.
Continue reading "Debuggers and Logging Frameworks"
You're searching the internet to answer a question you have,
and after some painstaking detective work you locate the answer.
Where do you store the answer for future reference?
Continue reading "Public Bookmarking"
Although only a few may originate a policy, we are all able to judge it.
— Pericles of Athens
Popular folklore has our profession's use of the word bug originating from a real insect found in an early electromechanical computer. Indeed, on September 9th of 1947 the Harvard Mark II operators did find a moth obstructing a relay's contacts. They removed it and dutifully taped it in the machine's logbook. However, engineers were using the term "bug" many decades before that incident. For example, in a 1878 letter Edison used the term referring to the faults and difficulties he was facing while moving from an invention's intuition to a commercialisable product.
Continue reading "Bug Busters"
How to Sort Three Numbers
Quick: how do you sort three numbers in ascending order?
Continue reading "How to Sort Three Numbers"
Supporting Java's Foreach Construct
Java 1.5 supports a new
construct for iterating over collections.
The construct can be used on arrays and on all classes in Java's Collection
I searched the internet for an example on how to make my own
classes iterable with this construct, but could not find an example.
Continue reading "Supporting Java's Foreach Construct"
Hard Disk Failure
I tell everybody that the question is not whether your hard drive
will fail, but when it will fail.
My laptop's drive started emmitting a loud grinding sound last Saturday.
Continue reading "Hard Disk Failure"
Machines should work. People should think.
— Richard Hamming
Continue reading "Dear Editor"
I know that you are nowadays often taken for granted, and that many programmers consider you a relic of an older age. Yet, programmers continue to spend an inordinate amount of time with you, and often listen to your advice. As you have no doubt observed I am often mistreated; in this letter I have written my most common grievances hoping you can convince them programmers to behave better toward me in the future.
Macro-based Substitutions in Source Code
A friends asks:
"How can one easily replace a method call (which can contain
arguments with brackets in its invocation code) with a simple
Continue reading "Macro-based Substitutions in Source Code"
Who Maintains the GNU Plotutils?
The GNU plotutils
package contains a reimplementation of a pic language processor.
I find pic invaluable for drawing diagrams;
in fact the sequence diagram editor in my
UMLGraph system, depends on it.
The current version is 2.4.1, and was released in July 2000 - almost four years ago.
I have discovered two bugs, but no one seems to be maintaining the package.
This is unfortunate.
Continue reading "Who Maintains the GNU Plotutils?"
When editing documents or code, my not so agile fingers, often trigger
a movement or search command that accidentally throws me to a random
location in the text I am editing.
How can I return back?
Amazingly, I noticed I am using exactly the same trick for returning back
on both the vim editor I use for most of
my editing tasks, and Microsoft Word I use for collaborating with many
Continue reading "Continous Bookmarking"
Patching Framework III
I needed to read some old files I wrote in 1992 using the Ashton-Tate
Framework III program.
Unfortunately, trying to run the program under Windows XP resulted in a
Continue reading "Patching Framework III"
Divide overflow" error.
A bit of searching on the web revealed that the problem was related
to the system's speed (1.6GHz).
Apparently, Framework tries to calculate the speed of the machine
by dividing a fixed number with a loop counter;
on modern machines this results in the overflow.