|
Measurement |
O X F O R D S O F T W A R E E N G I N
E E R I N G |
|
|
|
|
|
|
|
|
|
|
Getting better information for lower cost:
Defining, refining and implementing useful, cost
effective measurement
|
|
|
|
|
Measurement is important and valuable, but can be difficult: It is integral to good management and control, the analysis of
software, and, especially software testing. Yet for something so potentially
useful, pervasive and apparently commonplace measurement itself is strangely
elusive and difficult. Defining new measures can be problematic and
contentious, and introducing and using them is notorious for its expense,
poor return on investment, lack of value, and unintended and undesirable
effects. Software measurement has enormous promise but, like software
development itself, is still developing and has a long way to go to reach its
potential. Data is often of uncertain quality and statistical or graphical
analyses can fail to deliver the clarity and decision making capability
expected of them. Those involved in measurement work, or simply wishing to use measurement
within their projects or organizations face a number of hurdles: there are
few accessible or easy to use standards or guidelines for software
measurement, and little credible data for useful comparison. The little that
is available needs careful review and filtering and is often lacking
essential contextual information. Consequently measurement practitioners often have to design
their measures and analyses for themselves with all that implies. Not
unreasonably they can fall back on their experience of measurement from other
areas, or other software contexts, or rely on ‘shrink wrapped’ solutions or
tools. Sometimes this works
(usually when measurement is kept very simple), but often not. Even worse it can appear to work but
doesn’t: the measures or analyses are flawed, misleading those trying to use
them. It is no surprise that developers and managers can be sceptical of the
value of measurement, relegating it to an administrative overhead. (However,
valueless data will often continue to be collected, analysed and to mislead
well after its is known to be useless, the rationale being that some data is
better than none, or that not collecting data is unacceptable. Once data
collection mechanisms are in place it can be difficult to stop this time consuming
and expensive activity, and even changes to improve it need to be managed
with care.) Because we are so familiar with using measurement we tend to
underestimate it and the care needed for its design. Without a clear
understanding of what measurement really is it is easy to make serious
mistakes that will derail the attempt to design and acquire good data. Developing good, new measurements, introducing them from
elsewhere, or upgrading or validating existing measurement systems requires
an understanding of what measures really are, the types of measures that can
be used and the limits of their applicability. The selection and design of
analyses that are not just correct, but also robust, cost effective and
useful is important too. An understanding of the context of measurement and
the effect measurement will have both within and beyond that context is also
essential. So are methods to verify collected data and validate the measures
and their analyses. This is a lot to get right, but get any of it wrong and
you can be in trouble. Get it right and measurement comes good. It can become a
surprisingly low cost, low profile, lightweight, but powerful tool for
investigation, analysis and control. The characteristics of good measurement
are its apparent simplicity and obviousness and the trust and confidence it
encourages. With good measurement in place sound, credible data can be
generated to improve your understanding and decision making. Get better decision making and control with software
measurement If you want: -
increased
confidence in your measurement data, -
better
decision making, -
reduced
data collection and analysis costs, -
to know
how well your existing measurement systems are performing, -
to learn
how to develop or redevelop your own software measurement tools and systems, -
to equip
your measurement practitioners to deliver the above, please do contact us. We have more than twenty years experience in the application
of measurement to software development including: -
initiating
and managing major measurement initiatives in industry, -
defining
and introducing measures and analyses for software project management,
estimation, software, software development and test reporting – including
metrics for CMM and CMMI, -
the use
and development of software measurement design and implementation methods and
tools (including GQM (Vic Basili’s ‘Goal Question Metric’ framework), ami
(the application of metrics in industry), the CMMI’s Measurement and Analysis
PA (partly GQM in another guise), and our own ‘measurement maturity model’, -
investigation
of novel software measures and methods, measurement theory and measurement
dysfunction. We cannot tell you what to measure, only you can decide
that, but we can help you make the right decisions, and show you how
to measure. CCS March 2009 |
- |
|
|
|
|
|
|
|
|
|
|
|
|
||
|
© Copyright
OSEL 2009 |
|
|