blog dds

2005.02.04

Maintainability of the FreeBSD System

Last November Ioannis Samoladas and his colleagues published an article in the Communications of the ACM [1] that compared the maintainability of open-source versus-closed source projects. I applied the maintainability index [2] they used on the FreeBSD source repository following the code's maintainability over time, and comparing the maintainability of different modules. Here are the results.

Maintainability Changes Over Time

The following two diagrams illustrate the changes of the FreeBSD kernel maintainability and size over a period of 10 years (1995-2005). The first depicts the kernel maintainability evolution, while the second depicts the maintainability evolution of all the user-space code. Higher maintainability index (MI) figures indicate a more maintainable system. As we would expect, the maintainability decreases over time, but only slightly. This is especially important, if we consider that both the kernel and the user code more than doubled in size over that period. To provide a perspective, Coleman's article [2] describes two competing systems at HP that had a difference in the MI of over 30 points, while a CMU SEI report documents how a reeingineering a US Air Force system doubled its MI.

Maintainability of the kernel
Maintainability index over time in the FreeBSD kernel.

Maintainability of the user code
Maintainability index over time in the FreeBSD user-space code.

Maintainability of Modules

The last diagram depicts the maintainability index of different FreeBSD modules (a module is a collection of files grouped together for administrative purposes; a module typically represents a single command, or a subsystem). In our calculation we have not included modules less than 500 lines of code, to remove strange outliers. Based on the diagram one can argue that for this project an MI cutoff threshold of 40 can be established; modules under this value should be scrutinized.

Maintainability of different modules
Maintainability index of different modules in the FreeBSD user-space code.

Here are the modules with the highest and lowest MI.

Least Maintainable Modules

Module NameSize (LOC)MI
fore_dnld10284-2.77781525697723
ipfw87302.19445150187947
ancontrol35647.28187026797856
ee185847.68737418575401
ipcs13809.27186025233512
route332810.6633834274736
ping6544410.7523490117775
elfdump220413.5413559180982
ip6fw290615.9562542125594
wicontrol247817.5318159363428
moused625219.5567221067852
rpcinfo333420.0572886254166
cdcontrol252420.1761505886906
contrib_pam_modules1034421.4155931121246
fdisk_pc98251222.6477288555626
fdisk279223.0836640302608
ping322223.4929663251796
vmstat207826.7612628539371
chat308426.8113703372858
burncd139827.4184012138496

Most Maintainable Modules

Module NameSize (LOC)MI
librpcsvc1526114.229834520675
libcrypt918115.437965055286
libscrypt918115.437965055286
libypclnt1384115.790647786048
libkern9742115.934585498123
sys_libkern9742115.934585498123
libc_include1616116.122388754993
sys_ia64_include15392116.210428306165
sys_sparc64_include18160116.278661622061
sys_powerpc_include14672117.736894781652
sys_arm_include14368117.907041705555
lukemftpd966119.875420612704
contrib_openpam14902120.021600297376
vi828121.217588381786
libc_quad4830123.358498814576
kgzldr516124.143227794042
libc_sys774129.149297651959
libc_alpha2266129.78494426648
libc_string8584131.45431196217
libc_compat_431046134.126190286764
libc_i3862084135.901800173643
contrib_gperf1418136.301393878737

The complete MI module list is here.

References

[1] Ioannis Samoladas, Ioannis Stamelos, Lefteris Angelis, and Apostolos Oikonomou. Open source software development should strive for even greater code maintainability. Communications of the ACM, 47(10):83–87, 2004. (doi:10.1145/1022594.1022598)

[2] Don Coleman, Dan Ash, Bruce Lowther, and Paul Oman. Using metrics to evaluate software system maintainability. Computer, 27(8):44–49, 1994. (doi:10.1109/2.303623)

Read and post comments, or share through   


Creative Commons License Last modified: Friday, February 4, 2005 6:50 pm
Unless otherwise expressly stated, all original material on this page created by Diomidis Spinellis is licensed under a Creative Commons Attribution-Share Alike 3.0 Greece License.