Smart IT决策分析方法.pdf

Making Smart IT Choices Understanding Value and Risk in Government IT Investments Sharon S. Dawes Theresa A. Pardo Stephanie Simon Anthony M. Cresswell Mark F. LaVigne David F. Andersen Peter A. Bloniarz Center for Technology in Government University at Albany, SUNY 187 Wolf Road Albany, NY 12205 Phone: (518) 442-3892 Fax: (518) 442-3886 E-mail: info@ctg.albany.edu www.ctg.albany.edu Second Edition April 2004 ©2001 Center for Technology in Government The Center grants permission to reprint this document provided this cover page is included. Table of Contents Part One Introduction 1 Chapter 1. 7 The risks of IT innovation in government 8 Risks of IT innovation 10 Public sector risks 12 How this guide can help 13 The analysis and evaluation process 15 Analysis is a group process 15 Techniques for acquiring needed information 17 Phase 1. Understand the problem and its context 21 Phase 2. Identify and test solutions 26 Phase 3. Evaluate and make smart choices 33 Preparing a business case 34 Essential elements of a business case 36 Problem statement 37 A mission or vision statement 38 Specific objectives 39 Preferred approach 40 Expected benefits 41 Performance and group process 42 Risks and ways to address them 43 A basic plan of work, timeline, and key milestones 44 Project management and staffing 45 Cost estimates and funding sources 46 Alternatives considered 47 Opposing arguments and responses 49 Presenting your business case 50 Identifying the audiences for your business case 52 Presenting your business case 54 Be prepared for all kinds of questions Chapter 2. Chapter 3. Chaper 4. Part Two Skills for working with groups 62 Group facilitation 65 Consensus building, collaboration, and decision making 67 Audience and presentation skills 69 Library and document research 71 Internet research 75 Surveys 77 Interviews 79 Experiments 82 Simulations Information gathering techniques Tools for Phase 1: Understanding the problem and its context 85 Visioning 87 Hopes and fears 89 Service objective 91 Strategic framework 95 Stakeholder analysis 97 Positioning charts 99 Partisan analysis 101 Process analysis 105 Self-assessment tools 107 Models of problems Tools for Phase 2: Indentifying and testing solutions 110 Best and current practice research 112 Technology awareness reviews 114 Benchmarking 116 Environmental scanning 118 Models of solutions 120 Modest, moderate, elaborate alternatives 123 Prototyping Tools for Phase 3: Evaluating and making smart choices 125 Risk analysis 128 Scenario building and forecasting 130 Cost estimation 133 Cost-benefit and cost-performance analysis 135 MAU models 137 Prioritizing methods 139 SWOT analysis CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Introduction T his guide is designed to help you and your organization make good decisions about when and how to invest in information technology (IT). Put another way, it will help you avoid becoming one of the statistics that dominate reports on information technology investments. Reports on failure rates range from 50 to 80 percent and sometimes more. Failures don’t happen because people aren’t smart or trying hard. But failures do happen every day—mostly because people fail to realize and appreciate the complexity of these decisions and the way they affect nearly every other aspect of an organization’s work. Failure is almost inevitable if decisions about IT are hasty, unrealistic, or uninformed. To help you avoid this fate, we lead you through the complex and challenging process of analyzing an information problem or need—and its context. We help you identify, evaluate, and choose possible solutions. And we guide you through the process of building a solid business case for investing in your recommendations. Everything in this guide is based on years of experience working with dozens of government and nonprofit agencies that CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES needed to improve or change the way they gather and use information. Here at the Center for Technology in Government, we have worked on projects in human services, criminal justice, financial management, and environmental protection, among others. The agencies we’ve worked with range in size from enormous federal departments to tiny towns, and everything in between. Their goals focused on a wide variety of functions including case management, direct citizen contacts, research and analysis, general administration, and regulatory affairs. Some had many years of experience in using IT; others were novices. Regardless of their differences, every project encountered similar basic challenges. Early conceptualizations of their problems were often oversimplified. The influences of their larger organizational and political environments were underestimated. The ways in which current work would have to change were not fully considered. In those cases where new ways of working were considered, estimates of the effort needed to identify how people and processes work now (and how they would have to change) were vastly insufficient. Agencies sometimes hoped that “the right” technology would solve almost any problem. 1 These challenges that confront every organization do not mean that the people working on them are not focused, skilled, or well versed in their fields. They emerge from highly complex work environments and from unexpected interdependencies among organizations and processes. They require an appreciation for the critical importance of “up front” business and risk analysis. Too often, the quest for action–purchasing, hiring, designing, and programming—pushes critical knowledge-building activities aside. “Don’t you know enough to move forward with an RFP yet?” is a question many of our agency partners have heard after just a few weeks—along with, “We don’t have time to study the business problem any more, we need a system in place in six months.” In our experience, when the pressure to act exceeds the ability to understand the consequences of action, the risks of failure soar. The Center for Technology in Government is an applied research program at the University at Albany/SUNY. Established in 1993, the Center works with government to develop information strategies that foster innovation and enhance the quality and coordination of public services. We carry out this mission through applied research and partnership projects that address the policy, management, and technology dimensions of information use in the public sector. This guide offers our best thinking about how to define an information technology project and make a solid case for needed financial and organizational investments. It will be completed in two parts. Part One has four chapters and begins by considering the special characteristics of the public sector as an environment for making management decisions and IT choices. In the second chapter, we describe an analytical 2 process that accounts for program goals, stakeholders, processes, costs, and technology alternatives. In chapters three and four, we guide you through the process of turning your analysis into a business case and presenting it to various audiences. Part Two presents a wide variety of skills, techniques, and tools that can help you through the exercises introduced here. We will also publish a series of case studies that provide practical examples of how these concepts, tools, and techniques were used in some of our projects. We know that no single formula can guarantee success in developing and implementing a new IT resource. But this guide does offer a well-tested approach to reducing the risk of failure. The first principle is to apply the familiar principle of modern architecture—that form follows function. Accordingly, the initial focus of any IT effort needs to be on the service objectives and underlying business processes of the organization(s) involved, rather than on the technology itself. The best technology will not correct outdated policies, inadequate management practices, or poorly designed workflows. The second principle is to identify all of the internal and external stakeholders and to understand clearly their different needs, resources, and expectations. Each stakeholder group needs to be taken into account in identifying and considering the costs and benefits of various options. We also describe some ways to evaluate and choose among an ever-expanding array of technologies, including how to recognize when the best solution is no new technology at all. Finally, we offer some advice about how to use performance measures to evaluate your results. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES How the material is organized This guide is organized to lead you through a careful analytical process that results in a sound business case for investing in a significant IT project. Along the way, we refer you to specific tools and techniques that can assist you. We also offer case illustrations that show the tools in action in the context of specific, real projects. The analytical process, Part One, has four chapters. The first discusses the risks of IT innovation. The second describes the kinds of problems that are worth the time and effort of careful analysis we present a three-stage analytical process that helps you understand the problem and its context, identify and test solutions, and evaluate alternatives and make smart choices. Chapter three addresses the process of turning your analysis into a business case. Chapter four suggests ways a business case can be presented to various stakeholders, including top management, budget officers, and elected officials, as well as to users and customers. Part Two describes dozens of skills, techniques, and tools that are useful at various stages of analysis and case building. Many of them are well within the skills of any competent manager. Others require the help of an expert. You may already be familiar with many of them. For each, we describe its purpose, strengths, and limitations. We also cite books and articles that go into more detail. In some cases, we also present “how tos” that you can apply on your own. Illustrations drawn from the following cases are used throughout this guide: I I The Internet Services Testbed which involved seven state and local government agencies in a process of defining, designing, and building their very first information services on the World Wide Web. I The Bureau of Shelter Services and a score of nonprofit service providers and local governments who need to share information in order to evaluate services to homeless people. I The Municipal Affairs Division and its effort to create a consistent statewide information repository to support regional staff working on the financial affairs of local governments. I The Council on Children and Families and its 13 member agencies who wanted to offer a wide variety of statistical information about children over the World Wide Web. I The Office of the State Comptroller and its effort to ground a redesign of the Central Accounting System in a rigorous analysis of stakeholder needs. We acknowledge, with thanks, the energy and collaboration of the federal, state, local, and nonprofit agency managers and technical staff who have participated in projects at CTG. We also thank the corporate partners and university faculty who contributed their expertise. Without their willingness to experiment with new approaches and their strong commitment to improving government services, the work we describe here could not have been accomplished. We hope that these lessons and experiences will help other public sector organizations make their own smart IT choices. The Adirondack Park Agency (APA) and its need to manage information and improve both customer service and records management related to land use permits. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 3 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Part One Introduction 1 Chapter 1. 7 The risks of IT innovation in government 8 Risks of IT innovation 10 Public sector risks 12 How this guide can help 13 The analysis and evaluation process 15 Analysis is a group process 15 Techniques for acquiring needed information 17 Phase 1. Understand the problem and its context 21 Phase 2. Identify and test solutions 26 Phase 3. Evaluate and make smart choices 33 Preparing a business case 34 Essential elements of a business case 36 Problem statement 37 A mission or vision statement 38 Specific objectives 39 Preferred approach 40 Expected benefits 41 Performance and group process 42 Risks and ways to address them 43 A basic plan of work, timeline, and key milestones 44 Project management and staffing 45 Cost estimates and funding sources 46 Alternatives considered 47 Opposing arguments and responses 49 Presenting your business case 50 Identifying the audiences for your business case 52 Presenting your business case 54 Be prepared for all kinds of questions Chapter 2. Chapter 3. Chaper 4. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Chapter 1. The risks of IT innovation in government In 1995, the Standish Group began to publish reports of the IT failure rates of both public and private organizations in the United States. They suggested that more than 80 percent of systems development projects fail in whole or in part. Most projects cost more, take longer than planned, and fail to achieve all of their goals. One-third are canceled before they are completed. One system development project cancellation took place at the State of California Department of Motor Vehicles (DMV) in the mid-1990s. The project to move nearly 70 million vehicle, license, and identification records from an antiquated system to a new relational database was both behind schedule and over budget. When California’s lawmakers finally decided to end the agency’s IT project, over $44 million had already been spent and no end was in sight. One of the reasons the DMV project failed, says California Assemblyman Phillip Isenberg, is “because the agency staff were over their heads with a technology they did not understand.” The project also lacked a clear link between agency operational goals and the capabilities of the selected technology. Due to procurement restrictions, the agency was committed to a specific hardware platform before all the available options could be explored. As a result of the failure, California’s technology procurement process faces even greater control and oversight. Despite these problems, California has an annual IT budget of well over $1 billion, and more big projects are on the horizon—as they are everywhere in the world. In fact, government constitutes one of the world’s largest consumers of information technology. Because of its size, complexity, and pervasive programs and services, government cannot operate effectively without CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES using advanced information technologies. However, as the California DMV failure amply demonstrates, the risks of IT innovation in government are daunting. Years of research on information system success and failure have been unable to conclusively identify the factors that cause good or bad results. Information technology success and failure seem to be in the eye of the beholder. We’ve spoken to public managers who consider a project a success if it comes in on time and on budget. Others, who evaluate functionality and usability, might call the same project a failure. Many see failure when, regardless of time and budget, a new system makes it more difficult to do routine and familiar tasks. They have the latest technology but can’t get their work done as well as they did before. We’ve heard about systems that perform beautifully, but can’t be supported by in-house staff and therefore continue to generate high costs for consultants to maintain them. Failure may be a desirable statewide system that local governments can’t use because they lack their own expertise and technical infrastructure to connect to it. Failure has also been described as an on-time, on-budget system with great user interfaces and functionality, but users will not work with it because they don’t trust the underlying data sources. How do you protect against something you can’t define? We advocate an approach that builds knowledge and understanding through careful analysis of the goals, the larger environment, the specific situation, the likely risks, and the reasonable alternatives. That kind of thinking will help you raise useful questions, engage partners, challenge old models, garner support, assess policies, 7 identify risks, consider contingencies, and result in more successful innovation. Risks of IT innovation Expert observers of IT trends say organizations waste time, money, and credibility on IT because of a few fatal mistakes. They either buy the wrong technology for the job, or they buy the right technology but do not implement it effectively. They allow technical experts to design systems without the substantive, ongoing involvement of system users. They build systems that ignore the way people and processes really work. They don’t take into account the other systems that are already in place. They start investing resources in an IT solution before they really understand their needs and options. They are overconfident that they will get it right and don’t plan or budget for the inevitable post-implementation refinements that any system needs. They don’t attend to environmental realities such as workforce limitations, election or business cycles, rapidly changing technologies, political processes, and competing priorities. Clearly, IT innovation is risky business in every organization. Repeatedly, organizations abandon IT projects because these initiatives fail to accomplish the objectives they were intended to meet. In both the public and private sectors, a well-documented set of risks attends IT innovation. Unrealistic expectations Organizational perceptions of new technology are critical to achieving success. Positive expectations help lead to success, but too often overly optimistic expectations cause serious trouble. All the people involved in an IT initiative, from sponsors to users, need to have realistic goals and must share a common understanding of potential benefits, required policy and process changes, and the financial and organizational costs. 8 The technology is only a part, and often a small part, of the story. We’ve seen projects delayed or even halted due to unrealistic expectations about how quickly a project can be completed, about the human resource requirements, about how much collaboration was necessary and how costly it can be, and about how much and how many kinds of new learning and training would be required to build and use a new system. Lack of organizational support and acceptance Adoption of a new way of doing business or of a new technology is unlikely to succeed if it does not have widespread organizational support and acceptance. Much has been written about the critical importance of top management support and this is surely necessary. But, we’ve learned that success depends on many other organizational factors as well. It also takes skilled and committed team members and support and acceptance throughout the organization, especially among the people who will use the new processes and the new technology. Often this is the most important level of support, and often the most difficult to achieve. Failure to evaluate and redesign business processes IT management expert Michael Hammer says systems may not meet performance expectations because organizations “tend to use technology to mechanize old ways of doing business. They leave the existing processes intact and use computers simply to speed them up.” Meeting the needs of customers, employees, and decision makers means carefully studying and evaluating business processes. In most organizations, new processes are added as needed, but old processes are rarely evaluated to determine if they still make sense. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES When new technology is brought into the picture, a cumbersome and inefficient process may be automated “as is.” The results are systems that do not serve business needs, systems that are too expensive for the small productivity gains they provide, and systems that are not flexible enough to meet changing demands. This leads to poor performance in individual organizations and even worse problems when the system is expected to connect multiple programs or agencies, as is often the case in government. In order to make system design work more manageable, we often ignore the ways in which one system affects related work processes. For example, an accounting system needs to factor in the ways in which accounting is related to budgeting, revenue collection, and financial management. The accounting function does not stand alone, and a system that supports accounting cannot behave as if it does. Often organizations are not willing to invest the time and money necessary to do comprehensive process mapping and analysis. Many organizations see this as an unnecessary effort; in place of it they share procedures manuals (typically outdated and not reflective of what is going on) with system designers. Unfortunately, this sets up a “pay me now or pay me later” proposition. Ineffective or incorrect processes become embedded in the new system causing need for costly revisions or manual “work-arounds” that defeat the purpose of applying technology in the first place. Lack of measurable alignment between organizational goals and project objectives Another risk factor involves the alignment (or lack of alignment) between organizational and project objectives. The goal of IT adoption should be to enhance or improve an organization’s ability to carry out its main mission or business objectives. For instance, it should improve customer service, reduce CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES manual record keeping, speed up transactions, increase revenue, prevent errors, or support good and timely decisions. All of these kinds of improvements should be represented by baseline measures and target levels of performance. Without them, the technology tends to take on its own independent course, and fails to have its intended impact on how real people use information to accomplish real work. Failure to understand the strengths and limitations of new technology Information technology is constantly changing and improving. No one is able to keep up with the details of all new developments or to understand comprehensively how each new technical tool works. Add to this the fact that most new technologies must work in tandem with others, or must be incorporated into existing older systems, and the possibility for trouble mounts rapidly. Since most organizations are not in the IT evaluation business, they may rely on word of mouth, vendor claims, and trade publications for the bulk of their knowledge about which tools may be right for which jobs. There is little opportunity to learn first-hand before making expensive, irrevocable decisions. Projects that are too specialized or ambitious to manage successfully For years, oversight organizations like the US General Accounting Office (GAO) have warned against information technology initiatives that some characterize as “grand designs.” These are the projects whose scope is so large, time horizon so long, or design so unique that they will almost certainly falter or fail. Such projects invite delays, unexpected complications, gaps in funding, management nightmares, and other problems. Instead, most experts recommend that systems be designed and deployed in modules and be built on standard technologies using well-tested methodologies. This risk is relative and applies to large and small organizations. A town or county system with a 9 price tag of $15,000, in an overall budget of $400,000, with an IT staff of one is just as ambitious as a multi-million dollar project in a large state or federal agency. Public sector risks Government seems to have even more trouble than the private sector in successfully applying new technology. The public policy choices and public management processes that are part of government make it an especially difficult environment for IT managers. Some contend that bid protests, relatively low government wages, and legislative interference lead large government information technology projects into trouble. Others recognize that the structure of government decision making, public finance, and public accountability complicate the government managers’ jobs and limit the choices available to them. In addition, some government programs are so large and pervasive, that they simply push the limits of the best technical and managerial know-how. This environment brings a layer of risks unique to the public sector. When added to the organizational, operational, and technical risks described above, they present a daunting challenge to public managers responsible for choosing, funding, and building IT innovations. Layers of complexity surrounding government IT initiatives Organizational setting Program, policy & political context Technology solution Business processes 10 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Limited authority to make decisions By design, governmental authority is divided across multiple decision makers. Executive managers do not have a clear line of authority over operations. Their decisions are circumscribed by law, limited appropriations, civil service rules, and a variety of legally mandated procedures or court decisions. These restrictions do not blend well with the complexities of managing an expensive and complex IT project in a rapidly changing technical environment. Worse, as can be seen in the California DMV experience, when IT projects fail the common legislative response is to place more restrictions and more controls over the IT management process. Multiple stakeholders and competing goals Government programs are characterized by a multiplicity of stakeholders who often have competing goals. Customers, constituents, taxpayers, service providers, elected officials, professional staff, and others all have some stake in most programs. Some want more or different services, others want lower taxes or fewer rules. Understanding how different choices may affect each stakeholder group helps to identify likely conflicts and prevent unexpected problems. One year budgets Uncertainty about the size and availability of future resources weakens the ability of government agencies to successfully adopt new IT innovations. Most government budgets are handled on an annual cycle. While many agencies have developed planning mechanisms to cover a three to five year period, annual appropriations (influenced heavily by changing government-wide priorities) tend to negate long-term planning. As a result, funds promised for a project in the first year may not be continued during the second or subsequent years. Highly regulated procurement Most decisions to adopt emerging technologies are made through the traditional competitive bidding process. While the goals of competitive CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES procurement are goals of integrity and fairness, the processes are often a source of problems and delays. Agencies write Requests for Proposals (RFPs) using the information they have been able to gain from limited research. Vendors spend large sums of money trying to develop the winning response. Time consuming, arms-length reviews and negotiations ensue. Losers often take advantage of bid protest procedures that can further delay contract awards for months—or even years. Due to insufficient project support or understanding by top management, IT procurement requests may receive low priority. The resulting delays can mean time and cost overruns which in turn yield negative publicity, decreased support from top management, and negative perceptions of the overall value of the effort. Commodity-based procurement, on the other hand, is easy for agencies to use, but assumes that they have all the information and expertise they need to design and assemble a highperformance system out of a catalog of parts. Little capability to design or operate integrated or government-wide programs Critics (including many public officials themselves) complain that different government agencies operate more or less without regard to the fact that they often serve the same people. The difficult fact of life is that government is organized mostly into separate programs that receive specific authorization and funding from Congress or a state legislature. Accountability rules and traditions demand that these programs be operated separately, and one result is the famous “stovepipe” systems of government. Because these isolated programs emerge from deliberate design, they are very difficult to coordinate, much less integrate. New technologies, especially the Internet, are making it feasible to present a unified face to the public, but the changes that need to take place behind the scenes in policies, accountability mechanisms, processes, data sharing, and records management are only beginning to be understood. 11 Extreme risk aversion Government’s business is public business. This means that most new ideas have to be implemented in full public view. An innovation “gone wrong” risks not only dollars, but also the credibility of an agency and its leadership with legislators, executive officials, and the public. It’s not surprising, then, that government tends to rely on the “tried and true.” How this guide can help Government managers need to analyze and evaluate IT choices because these choices are among the most complex and expensive decisions they are expected to make. Whether you are a local official considering a new system to support building permits or an administrator at a large federal agency considering a new network infrastructure, you are faced with a complex and relatively expensive decision-making process. The consequences of your IT decisions often have a significant and direct impact on the public. For example, the safety of the flying public rests on the ability of the Federal Aviation Administration to implement systems that control air traffic. A state child welfare agency gathers and responds to information that protects the health and well being of children in that state. A local government emergency response application ensures that emergency vehicles are routed to incidents in the fastest and safest way possible. Systems that support law enforcement make a big difference in the ability of federal, state, and local criminal justice agencies to provide public safety. These systems cost thousands, millions, even billions of dollars. They are important because the goals they serve are crucial to our quality of life. The risks of system failure are linked to the risks of service failure for hundreds, thousands, or even millions of people. This guide can’t tell you what technology to buy, or even what problem is most important to solve. It can’t tell you how much money you will 12 save using IT, or even if you will save any money at all. This book is about how to think about an IT investment. It offers a set of analytical techniques to understand the issues and opportunities and to build a business case for investing in a particular path. It shows how to use that business case to get the support you need to move forward with your project. We think of this phase of the IT investment process as the one that comes “before the beginning.” It is necessary before the first design meeting, before an RFP is written, before the budget is developed. We believe this kind of up front analysis is essential to doing all of those things well because it uncovers both risks and resources that lead to smarter IT decisions. In short, three kinds of analysis mitigate the risks of these investments: I thoroughly understand both the problem to be solved and its context I identify and test possible solutions to the problem I evaluate the results of those tests against clear service and performance goals The following chapters present a well-tested methodology that can help you to understand and carry out these three critical tasks. References Hammer, Michael. (1990.) “Reengineering Work: Don’t Automate, Obliterate,” Harvard Business Review, July-August. Isenberg, Phillip. (1994.) “Point of View: When BIG IT Projects Falter,” Government Technology, July, 7(7). Miller, Brian. (1994.) “DMV Project Hits Dead End,” Government Technology, July, 7 (7) pp. 1, 53-54.The Standish Group. (1995). The CHAOS Report. On line at www.standishgroup.com/sample_research/ chaos_1994_1.php CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Chapter 2. The analysis and evaluation process In this chapter, we present an analytical process that can be used in any project that applies IT to a service delivery or administrative goal. The process begins by defining a problem or purpose within its environmental context. It then goes on to identify and test possible solutions. The third phase focuses on the evaluation of alternatives and the selection of a preferred approach. For simplicity, we present these phases as sequential steps, but in action they are often iterative. What you learn in one phase may prompt you to return to an earlier one to refine your thinking before moving forward again. Although this process of feedback and learning takes extra time, it is critical to comprehensive understanding, and it allows you to build a business case that reflects all the essential risks and options. The final steps described in the chapters three and four are the preparation and presentation of that business case. Each phase builds on the perspective gained from earlier ones. The result is a multi-faceted analysis of a proposed project that has a high likelihood of accurately predicting real costs and outcomes. Figure 1 illustrates the entire process. Figure 1. Overview of the analytical and evaluation process Choose a "good" problem How to Make Smart IT Choices Phase 1 Understand the problem and its context Specify your program or business objective Identify and assess stakeholders Analyze the problem or process to be tackled Phase 2 Identify and test solutions Find relevant practices, tools & techniques Develop and test alternative solutions Phase 3 Evaluate alternatives and make smart choices Compare risks Compare costs and expected performance Make and explain final choices Present results in a business case CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 13 First things first: Choose a “good” problem The research, testing, and evaluation that go into this analysis are labor intensive processes. They can be applied to any problem, but not every problem deserves this level of attention. This is a process that is worthwhile for complex, mission-critical, information-intensive problems. We call these “good” problems because their solution has high positive impact. For these problems, you need to pay close attention to the political, economic, legal, and organizational environments as well as the technologies involved. Not every IT decision has these characteristics or warrants the investment of resources that this process entails. Here are some administrative problems and service delivery questions that have successfully used this process: I How effective are current service packages in helping homeless families achieve independence? I How can the complete history of relationships between a state agency and local governments be captured for ongoing use in current and future technical assistance? I How can information about the health and well being of children be made readily available to professionals and individuals concerned with the design of children’s service programs? The first question represents an information sharing and program evaluation problem. The homeless services system involves not only state agencies, but local governments, and scores of nonprofit shelter and service providers, all with different kinds of information maintained in different systems and formats. These organizations also respond to different funding streams and to a variety of legal requirements. The second problem involves the central office of a major administrative agency and its regional offices distributed around the state. 14 Each regional office is responsible for monitoring and advising local governments in its geographic area on their financial affairs. Each region collects information and does things in its own way. There is no statewide repository of information that can provide context, history, best practices, or overall performance information about this important aspect of municipal government. The third example does not focus on operational concerns, but on the availability and usability of statistical information about children. In this case, the information comes from 13 state agencies, is compiled once a year into a printed book by a central coordinating agency, and is used by hundreds of municipalities, nonprofit service agencies, and research organizations. The data is collected according to different time periods, using different definitions, and covering different geographic distributions. Despite their differences, these problems have some similarities. First, they are mission critical to the agencies that sponsor them. The homeless services project is deeply embedded in the core services and values of all involved organizations. The financial health of municipalities is one of a handful of overriding mission goals for the administrative agency. And a visible public focus on the needs of children is the entire reason for the existence of the coordinating agency. Second, these problems are informationintensive situations. The solution to each problem depends, in large part, on the quality, timeliness, and accessibility of information. In most cases, some information systems are already in place and need to be taken into account in any new approach. Third, they all exist in an environment of high complexity. Both the homelessness and children’s data projects involve many independent organizations, each with its own CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES practices, values, and rules. Similarly, the municipal affairs project covers the whole state and must deal with local diversity ranging from huge sophisticated cities to tiny rural towns and villages. All three projects must deal with public opinion, public budgeting and legislative cycles, and legal requirements. Civil service rules circumscribe staffing assignments and compensation. Organizational rules, traditions, and structures set boundaries. Many different business processes already in place will need to be understood, and may need to be changed. In projects with these characteristics, opportunities abound for wrong assumptions, premature decisions, and dangerous oversimplification. This kind of complexity seems overwhelming, and a common reaction is to try to cut through to the part of the project that is more concrete and manageable—the technology. This is almost always a mistake. Projects like these demand a careful analysis that works through and manages the complexity at every level, from the larger environment, to the organizational considerations, to the work processes and data needs, to the technology choices. Analysis is a group process For projects like those described above, the analytical process is not a solitary process. These projects involve many people in different organizations or organizational units. One person may lead an effort, or might collect, organize, and present information, but groups of people will inevitably become involved and group processes will be needed. Government information technology projects can involve dozens of people. Individuals with vastly different work styles, backgrounds, and talents are often brought together, asked to form a cohesive group, and charged with solving a problem. Often people from different organizations need to work together to plan CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES and implement a project. But their individual differences, and group dynamics, can make it difficult for the group to reach its goals. Consensus-building tools and skilled group facilitation can be very helpful in guiding a group through the steps necessary to make effective decisions. Consensus-finding and -building tools are often needed to help a group resolve different views and conflicting objectives or interests. Groups also frequently need to be introduced to models for collaboration, especially if they’ve never worked together before. Effective teamwork may also involve difficult trade-offs and other choices, so some decision-making tools and techniques can be useful. Group processes take skill and time to work effectively, but they result in well-documented and well-understood decisions that can then guide the work group to a successful outcome. Tools and techniques for all of these topics are described in Part Two. Techniques for acquiring needed information Your analysis will rely on many kinds of information that can be gathered in a variety of ways. While most of the tools and techniques we identify later are associated with a particular phase of the analytical process, data gathering techniques can be used whenever they fit the situation. You can use survey research methods such as self-administered or telephone questionnaires to capture data about some characteristic, attitude, or opinions of users and stakeholders during the initial problem definition stage, during evaluation, or at any point in between. For example, interviews can be used to assess stakeholders at the beginning of the process, to gather information about best and current practices in the middle, and to evaluate a prototype near the end. Similarly, simulations and process mapping can be used to 15 understand current processes, and to design or evaluate new ones. All of the following data-gathering techniques will be described more fully in the tools installment. I Library and document research – The purpose of these systematic searches of print and electronic sources is to identify, review, analyze, and evaluate the recorded experience and assessments of others who have dealt with a similar information problem, or who have used a technology similar to the one you are considering. I Internet research – Web searching is a specialized kind of literature review that involves using Internet search engines to locate pertinent information about a given topic. By entering various key words and phrases, you can search the Internet for Web sites, publications, listservs, and other resources about your topic. An important point to remember about the Web: contrary to much popular opinion, the Web doesn’t provide you access to every worthwhile thing ever written—it only directs you to those things people have made available. I Self-administered questionnaires – When you need information from a large number of people or organizations, self-administered questionnaires are the right method. They allow you to collect structured data fairly quickly from many people in different locations. They usually employ short, simple questions and follow a careful sampling plan. I Interviews – When you want to gather detailed information about people’s impressions, experiences, ideas, and attitudes, interviews are often the best method. Face-to-face or telephone interviews yield rich information, and are good way to understand complex topics. 16 I Experiments – Use an experiment to learn what influences the way some process or activity actually works. The data is typically a result of direct observation of behavior in a controlled situation. The natural setting involves a combination of many interacting influences that make it very difficult to sort out the independent effects of one factor or another. By contrast, an experiment is designed to control some factors so the impacts of specific ones can be assessed. Experimenting with new process models or user interfaces, for example, might provide a team with insights into their proposed changes. I Simulations – System simulations provide a structured approach to visualizing and understanding how complex social and managerial systems give rise to problem behaviors, as well as what types of solutions might be applied. I Process Analysis – Business process maps (sometimes referred to as process models) are a graphical representation of the steps followed to achieve specific purpose, like hiring a new staff member or approving a permit application. Process models help you understand and communicate about a business process. They can be used to improve a process by evaluating, reorganizing, changing, or eliminating steps. They can also be used to track the flow of resources, such as information, through a process. Some of these data gathering techniques, like Web searching and basic interviewing, are easily learned. Others, like experimental design or simulation, require considerable expertise. For these, you may want to consult with or hire an expert. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Phase 1. Understand the problem and its context Having identified a significant need or problem, the first phase of the analysis is to understand it as fully as possible in the context in which it occurs. Three kinds of work help you reach this depth of understanding: specifying objectives, identifying and assessing the influence of all stakeholders, and then analyzing the need or problem in detail. A more detailed description of the tools that can help gain a greater understanding of the problem and its context can be found beginning in Part Two. The first kind of work leads you and the others involved to a clear, unambiguous, shared understanding of the business or program Choose a "good" problem How to Make Smart IT Choices Phase 1 Understand the problem and its context Specify your program or business objective Identify and assess stakeholders Analyze the problem or process to be tackled objective you want to achieve. This sounds simple, but in practice is often very difficult. Many projects go wrong at this very first step because those responsible assume everyone sees the situation and its resolution in the same way. This is almost never the case. Even simple programs or processes can be approached from different points of view. One participant may see a service that could be more accessible to its customers, another can look at the same service and want to reduce the cost and effort to deliver it, and still another may take an evaluator’s perspective and ask what value it delivers to society. Tools for Phase 1 Specify objectives: I visioning hopes and fears exercise I strategic framework I Identify and assess stakeholders: Phase 2 Identify and test solutions Find relevant practices, tools & techniques Develop and test alternative solutions Phase 3 Evaluate alternatives and make smart choices Compare risks Compare costs and expected performance Make and explain final choices I positioning charts partisan analysis I stakeholder analysis I Analyze problem or process: I self-assessment models of problems I process analysis I Present results in a business case CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 17 Specify objectives Several tools can be used to work through this essential first step of setting objectives. One simple tool is called a hopes and fears exercise in which the members of a work group individually state what they hope the project will accomplish—and what they are afraid might happen instead or as a consequence. These individual statements can be grouped into themes that often reveal multiple, competing, and sometimes conflicting objectives. Once specified in this public way, the group can work toward the specification of a shared statement of objectives. The public statement of fears works in a similar way. It represents early indications of problems that are likely to be encountered along the way. Other tools such as service objective and a strategic framework are different approaches to the same goal—a clearly specified, unambiguous, agreed-upon objective. Sharpen your goals In the Internet Testbed project, agencies generally started out with enthusiasm for the big possibilities of the Web for reaching more people with more information. By using the strategic framework tool, especially its service objective component, they soon came to see that these goals were too vague to be the basis for design or implementation. This tool pushed them to say what exactly they would do for whom with what result. For example, the Office of Real Property Services developed an objective to use the Web to support professional training and provide reference services to local government assessors in order to develop, improve, and maintain their assessment skills without costly classroom training and cumbersome paper binders. By specifying in concrete terms how they would use the Web to provide value to a well-defined audience, they were able to sharpen their focus and move more smoothly to understand their stakeholders, find suitable practices and tools, and consider alternative approaches. 18 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Identify and assess stakeholders The next step in this phase is careful identification of all stakeholders and the ways they can influence or be affected by the project. This can be done with a stakeholder analysis exercise. Many projects limit stakeholder considerations to those who are directly involved in the development of a system. Generally, this is not enough. Those who are indirectly affected count, too. Often, this kind of analysis does a good job of identifying positive effects such as who saves time or money, but they often ignore the negative effects such as who gets lower priority or picks up the cost of making things cheaper for someone else. A positioning chart is a good first stakeholder analysis tool. The chart places stakeholders on a two-dimensional grid that shows both the degree to which they support or oppose the project and their importance to its success. Those stakeholders who are important to success, whether they are supporters or opponents, deserve careful scrutiny and a clear-cut plan of action for how to get them involved or how to deal with their concerns. A partisan analysis can be used to uncover potential conflicts and completing interests among stakeholders. Stakeholder influences are also critical. Some stakeholders are essential because of their legitimacy or power regardless of how active they are in a project. Know your stakeholders When the State Comptroller’s Office decided to plan for a successor to its 18 year-old statewide accounting system, the staff recognized how crucial this system is to the financial operations of every state agency. They embarked on an extensive stakeholder analysis that divided agencies into groups with similar characteristics and invited them to take part in 13 day-long workshops to specify their needs and ideas for a revised system. This process engaged the key users of the system in a dialog that generated strong recommendations for meeting user needs, and initiated an atmosphere of collaboration and user support for the project. One group of stakeholders was especially important to the project. These “strategic partners” represented the agencies that would eventually have to approve the design and funding of the project. They were involved continually, not only as system users, but as key players whose understanding and support could make or break the effort. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 19 Analyze the problem and the process The last work in this phase is where you begin to tackle the problem itself. Often the problem or objective is embedded in at least one work process. Process analysis is therefore a good way to delve into the details, understand the bottlenecks, see the handoffs, and identify where information is added or recorded. A process map that shows this in sufficient detail is then ready for preliminary process improvement ideas where the work team identifies ways to streamline or get more value from it. Other problems are not process oriented so they demand different tools. Self-assessment tools can help gather information about the current situation. Models of the problems can help to make your situation more explicit. Sometimes you need customer satisfaction information and may want to conduct interviews or a survey. Statistical data about some aspect of performance or cost may be needed. The important point is to analyze the problem fully, using the tools at your disposal. You will often be surprised by unexpected findings and new ways to understand cause and effect. Get some baseline data The Adirondack Park Agency faced both process and workload problems that generated low customer satisfaction. Most staff time was devoted to the processing of permit applications by landowners or developers for permission to build or otherwise develop privately owned property in the six-million acre park. Workload statistics, however, revealed that the vast majority of customer contacts (and therefore the best opportunity to give good service) were telephone inquiries about relatively simple questions. By collecting these workload statistics, the Agency was better able to focus its efforts. Staff learned that answers to telephone inquires could be shortened by 99 percent by changing the way information was organized and accessed. On the other hand, process analysis revealed that permits were subject to so many legally mandated time frames that theire processing time could be shortened by only one to two percent. 20 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Phase 2. Identify and test solutions Phase 1 arms you with a very detailed understanding of your objective and the context in which you must try to achieve it. In Phase 2, the work turns to a search for reasonable solutions that you can investigate and test. This phase makes substantial use of the experiences of others who have tackled similar goals. It also leads you to develop alternative solutions and offers some ways to try them out in low-cost, low-risk ways. You are not building a system yet—but you are building a great deal of knowledge about the system that you will eventually construct. A more detailed description of the tools that can help identify and test possible solutions are available in Part Two. Choose a "good" problem No matter how unique your problem may seem, some other organization has probably faced it. That organization may be another agency in your own state; it could be a government somewhere else in the US or the world; it could be a commercial or nonprofit organization. It might be much bigger or much smaller than your organization, or engaged in a very different kind of work. Regardless of their settings, you can learn from them. The first part of the work in Phase 2 is the search for relevant practices, tools, and techniques whose use in other places can teach you something that you don’t have to learn on your own. How to Make Smart IT Choices Phase 1 Understand the problem and its context Specify your program or business objective Identify and assess stakeholders Analyze the problem or process to be tackled Phase 2 Identify and test solutions Find relevant practices, tools & techniques Develop and test alternative solutions Tools for Phase 2 Find relevant practices, tools, and techniques: I best and current practice research I technology awareness reviews I benchmarking I environmental scanning Develop and test alternatives: Phase 3 Evaluate alternatives and make smart choices Compare risks Compare costs and expected performance Make and explain final choices Present results in a business case CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES I models of solutions I modest, moderate, and elaborate alternatives I prototyping 21 Find relevant practices, tools, and techniques You can learn from the experiences of others by using a variety of research tools. Best and current practice research is a good generalpurpose tool that any professional can use effectively. You can conduct this kind of research in the library, on the Internet, at professional meetings, in face-to-face interviews, and on the phone. The key is to specify, as clearly and narrowly as you can, the questions you want to answer, and then to evaluate critically what you read and hear. One benefit of working in the public sector is the general willingness of public managers to share their good and bad experiences with one another. You can often learn more by picking up the phone and calling a contact person than by just reading a story in a newsletter or Web site. Probe for the complete story of how a project unfolded or a technique was used. Ask about critical success factors and ask also what should have been done differently. Learn from the experiences of others When the Division of Military and Naval Affairs (DMNA) was considering how to design its first Web site, best and current practices research helped staff make basic decisions about what to focus on and what to avoid. By searching the Web sites of similar state and federal agencies, DMNA staff identified good business uses of the Web (such as recruiting, location of facilities, and employment opportunities), good design principles (such as consistency from page to page, clear contact information, and an informative home page), and features to avoid (big graphics, no feedback mechanism, agency-centric presentation). Today many of these are considered gospel, but in the early days of the Web this research was invaluable to making a good start. 22 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Know your environment An environment scan can help determine what is happening in the overall environment that may have an impact on the project. The results of this kind of review may lead you to reconsider some aspects of your own service or business objective. You may want to return to and refine your focus before moving forward to other parts of the analysis. You might also discover other stakeholders need to be considered or that some part of your problem analysis needs more attention. Take advantage of these early opportunities to improve your fundamental analysis—they are more valuable and less costly now than they will be later in the project. Benchmarking, and building technology awareness through trade shows, demos, and vendor presentations are other good ways to collect current information that you then evaluate for comparability, completeness, and relevance to your own situation. These investigations may help you understand the features and applications of technologies you are considering for your project. They may also help you understand the broader infrastructure needs you will have to take into account. Understand the implementation environment The Bureau of Shelter Services is responsible for programs that serve homeless people around the state. It carries out this mission in concert with local governments and many nonprofit organizations that actually provide shelter and services. When these agencies began to discuss ways to share information to better assess service performance, they needed to consider how information is collected and processed in these many different organizations. Some of these organizations already had sophisticated information systems; others had just the basics or none at all. As a result, part of the project to define a shared Homeless Information Management System (HIMS) included a series of technology awareness meetings in which different kinds of case management systems were introduced and discussed, and the current and future technology capabilities of the participants became much better understood. Consequently, the system design took these variations into account. By focusing on shared service values and data quality, as well as technology, the design offered benefits even to those with little technical capability. Benchmarking, and building technology awareness through trade shows, demos, and vendor presentations are other good ways to collect current information that you then evaluate for comparability, completeness, and relevance to your own situation. These investigations may help you understand the features and applications of technologies you are considering for your project. They may also help you understand the broader infrastructure needs you will have to take into account. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 23 Identify modest, moderate, and elaborate solutions Phase 2 is completed by specifying several alternative approaches or solutions that seem to fit well with the context, problem, and stakeholder analyses from Phase 1 as well as the information you gathered and evaluated from other places at the start of Phase 2. At this point, your work group may have several ideas to pursue. You should specify them all in similar ways so that you can compare them. Models of solutions can be built to help minimize risks and get all the potential development costs on paper. A description of features and functionality at modest, moderate, and elaborate levels of investment is one way to do this. First, list the key elements of a solution. These will vary according to your objective, but some common elements are the means of customer access, response time, degree of customization, level of security, extent of manual data handling, and degree of integration with other processes or systems. The next step involves describing how a minimally useful solution would address each of these elements. This is the “modest” solution or the one that accomplishes the least worth doing. Then specify how a “moderate” solution would address each element. This one offers greater functionality, more convenience, or other improvements over the modest level. Third, list how an “elaborate” solution would address each element. This is the most advanced solution that an organization might attempt. Finally, for all three alternative solutions, state the benefits and who reaps them. Benefits may be quantifiable as dollar or time savings. We think of these as the “cheaper” or “faster” benefits. Another category of benefits might be categorized as “better.” These benefits come from qualitative changes such as improvements in service quality or availability. Associate benefits with cost and complexity The idea behind the Kids Well-being Indicator Clearinghouse (KWIC) is to make statistical information about the condition of children readily available to government agencies, researchers, service organizations, and advocates over the Web. The statistics already existed in the form of an annual printed book. The challenge was to create a Web-based repository that would be easy to use and easy to update. During the project, the potential clearinghouse was specified in modest, moderate, and elaborate terms. The modest version would simply to post the tables from the book on the Web. It would be more accessible because it would not be tied to the number of copies that could be printed. The moderate version would use a database format, a query capability, meta data, and links to help users understand the data better. The elaborate version would add data manipulation tools and mapping capability. Each level obviously offered different benefits and different operational and management considerations. 24 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Prototype when possible Testing your alternatives may start by gathering and comparing performance data from existing projects that are operating elsewhere. If all the alternatives are operationally feasible, then testing the concept with appropriate stakeholders may be sufficient. If the proposals involve some real unknowns, then try to create test conditions that are as close to real life as possible, such as building one or more prototype systems tested under field conditions in a controlled experiment. Also remember that the more costly the solution (in dollars, effort, and change to the status quo) the more reason you have to conduct a life-like field test. One surprising result of these tests may be the extent to which non-technology solutions fill your needs. Often significant improvements in business processes or information flow go a long way toward meeting your objectives, even without the application of new technologies. In one agency we worked with, $3 million had been set aside for an imaging system to improve customer service transactions. After successfully building and testing a prototype that reflected completely revamped business processes, the agency decided not to spend the money on new technology. Why? The prototype effort showed that most of the benefits could be obtained by process improvements alone. Test and evaluate alternatives The operating environment of homeless services is both complex and highly variable. For this reason, it was important to build and test a prototype of the Homeless Information Management System (HIMS) to determine its feasibility and value to different types of users. To prototype the system, the project participants had to focus first on the kinds of questions they would want the system to answer and then on the nature and quality of the data that would be needed. These discussions were very lengthy and detailed, much more so than the technical system design step. Once built, the prototype was deployed in a testing environment in which shelter providers, local officials, and other users evaluated its usability and its ability to help them answer questions and perform functions relevant to their particular situations. While some organizations did not even have computers, better service and data definitions benefited everyone, even those with little or no technology. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 25 Phase 3. Evaluate alternatives and make smart choices The final analysis of alternatives is mainly an opportunity to integrate and compare information. Your work group has looked at the problem and potential solutions from many points of view using several tools. Now is the time to step back and ask what the entire body of data has to say. What are the main findings and conclusions that emerge? This final analysis helps you see what can and cannot be done to achieve your goals. Often, at this point you have several reasonable approaches to consider. This is the stage where you compare and contrast them along important dimensions such as risk, cost, and acceptability to key stakeholders. A more detailed description of the tools that can help you evaluate alternatives are available in Part Two beginning here. Choose a "good" problem How to Make Smart IT Choices Phase 1 Understand the problem and its context Specify your program or business objective Identify and assess stakeholders Analyze the problem or process to be tackled Phase 2 Identify and test solutions Find relevant practices, tools & techniques Develop and test alternative solutions Phase 3 Evaluate alternatives and make smart choices Compare risks Compare costs and expected performance Make and explain final choices Present results in a business case 26 Usually one or two of the possible solutions seem better to you than the others. The analysis you conduct at this stage needs to use the full range of data to test these preferences and justify why a particular course of action is preferable. The most important limitation of any analysis is the quality and completeness of the data on which it is based. Weak data can’t be improved by powerful analytical tools or fancy presentations. Be sure to pay attention to assumptions, estimates, and just plain guesses, and be honest with yourself and others in revealing what part they play in your analysis. Remember, too, that everyone has biases. Try to identify yours and counter balance them with solid analysis and reasonable alternative interpretations of the data. Tools for Phase 3 Compare risks: I risk analysis scenario building I forecasting I Compare costs and expected performance: I I cost estimation cost benefit and cost performance Make and explain final choices: I MAU models Prioritizing methods I SWOT analysis I CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Compare risks In this phase you will make an explicit analysis of the risks associated with various tested alternatives. Notice that you have been identifying and dealing with risks from the outset. Opposing stakeholders represent risks, new technologies bring risk, and so on. Risk analysis can be very formal and quantitative or more qualitative. Scenario building (one of the strategic planning tools in Part 2) is a technique in which you describe a hypothetical situation in a way that helps you predict the consequences of decisions and actions. It helps you see what could go wrong in different situations. You could also model these situations on paper or in simulations to understand how different actions or decisions might lead to unexpected results. This kind of analysis improves the confidence level of decision makers. These realistic projections of risk will help them understand the possible consequences of different choices. Scenarios can help identify risks The Homeless Information Management System (HIMS) project was not the first attempt by a government agency to gather information from service providers. In fact, a project to do just that had failed completely a few years earlier. That first effort did not take provider needs and capabilities into account, but was simply selected by a government agency and imposed on the nonprofit organizations. The nonprofit providers resisted strongly and formed a coalition to prevent future efforts of this kind. Through a process of scenario building the Bureau of Shelter Services could see that their idea was at great risk of rejection. To move forward, they had to build trust among the providers and engage them fully in the planning and design effort. This process took considerable time but it removed perhaps the greatest risk of failure—the likelihood that the users would reject the system. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 27 Compare costs and expected performance improvements Costs, of course, are critical to the final analysis of alternatives. You begin to identify and specify them much earlier in the process, just as you did with risks. This is the time to examine them in more detail to be certain that all cost factors are accounted for and that future costs are considered along with the costs of design and implementation. In projects that involve more than one organization or level of government, be sure to assess the costs to all players. Avoid the common mistake of costing out only the direct investments that your own organization must make. Cost estimates can be obtained from historical data such as budgets or spending records, feasibility studies, or from outside consultants or agencies that have attempted similar projects. A cost-benefit analysis can be a simple comparison of costs and projected savings, or it can be a more detailed financial model. You can also conduct a robust cost-benefit or cost-performance analysis by considering all the players and effects of the system over time. Whatever you choose, the results have to be a convincing aspect of the business case you will develop. Estimate the costs of reasonable options The Kids Well-being Indicator Clearinghouse Project (KWIC) moved from the specification of modest, moderate, and elaborate program designs to a similar cost-performance evaluation. The staff identified the benefits that could be expected from each level of system specification and who would receive those benefits. They then estimated the cost of developing and operating each version. Cost categories included project management, organizational readiness activities, user technology, technical and support staff, information content development and maintenance, and the costs of hosting the KWIC Web site. They developed cost estimates for the first year and for annual recurring costs. It turned out that the moderate version was both feasible and cost-effective. It would deliver more benefits than the modest version, as well as most of the benefits of the elaborate version, at a more affordable price to the agency. 28 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Make and explain final choices At this point you may have a clear best choice and need go no further to consider other options. If you still have at least two feasible alternatives, though, some selection tools will be useful. These are designed to compare alternative courses of action in a structured and explicit way. The multi-attribute utility model or MAU model is a versatile tool for comparing alternatives. Its academic-sounding name makes it seem forbidding, but if you have been involved in any major procurements, you have probably already used it. The MAU model rests on the specification of weighted evaluation criteria. These might be such items as total cost, convenience to users, time to completion, reliance on contractors, or a host of others. Each criterion is weighted relative to the others, often by distributing 100 points among them. Then each alternative solution is scored on all criteria. Simple math gives each alternative a total score. Sometimes the scores demonstrate a clear best choice, sometimes they narrow the field, and other decisionmaking processes are needed. Occasionally the numerical scores do not “fit” the intuitive assessments of the evaluation group or decision maker. Often this means that an CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES important criterion is not being made explicit and should be added to the model. SWOT (for “strengths, weakness, opportunities, and threats”) analysis presents another way to compare alternatives. This and other prioritizing and choice-making tools to consider will be described in the tools installment. If you have done a thorough job of working through the three phases of analysis and evaluation, you will now be in a good position to make “smart” choices. You will have a clear understanding of the project objective(s) and will know what your stakeholders think about it. Relevant business processes will have been identified and improved. You will have information about alternative ways of approaching your goal, including the experiences of others who have done similar work. These alternatives will have been tested in some way with stakeholders and users, and the costs and risks of each one will have been identified. If more than one approach makes sense, you’ve conducted a comparison process that reveals their relative strengths and weaknesses. Now you are ready to make your recommendation for action in the form of a business case. 29 The bare bones analysis Many situations present a need for thorough analysis, but neither the time nor the staff resources to conduct one. Fortunately, the process presented here is not an all-or-nothing proposition. We urge you to do as much analysis as your resources allow because each inquiry adds valuable information to your ultimate decision. But in those all too common situations where time pressures mean only a few items can be considered, what should you do? Even if you have only a few weeks, you can gather crucial information that will help your project succeed. To do this effectively, you need a team of people who represent different perspectives and skills. We strongly recommend these activities: Start with a hopes and fears exercise Gather a representative group of stakeholders together for a morning or afternoon. This activity identifies and makes explicit the variety of initial expectations that people bring to a new project. Where hopes differ, the group can work toward consensus on either a common goal or an agreed upon priority for multiple goals. Stated fears give you strong signals about the risks you face and an early start on the work it will take to address them. For each goal, create a structured service objective For each goal, reach agreement about what would be done for whom with what effect. We often use this fill-in-the-blank sentence: “Our service (or business) objective is to do something specific for a specific customer so that the customer will receive the following benefits.” This exercise forces your planning group to be specific about three things: the practical nature of each goal, the intended customer or beneficiary, and how that customer will be affected. It begins to give you a sense of how straightforward or complex your project will be. 30 If you have one objective and one customer, your job will be easier than if you have several of either. Conduct some form of analysis that actively involves system users There are many ways to do this. The crucial element is to engage the users directly. This might be done in focus groups, site visits, workshops, or process mapping sessions. Engaging users directly is the best way to test and refine your objectives and to understand the existing reality that you intend to change with your project. These activities can build trust with users, who are key stakeholders, and can prevent you from making big mistakes based on untested assumptions about work practices, skills, and organizational culture. Conduct targeted best and current practice research List the top four or five questions you have about your project and find out who has experience that can help you answer them. You can use the Web or publications to locate likely sources, but don’t rely on these exclusively. The best information will come from actually talking with these people. Most government managers are very willing to share their knowledge and lessons learned. Take full advantage of this opportunity to learn from the experience of others. Specify modest, moderate, and elaborate alternatives and estimate their costs and benefits Using the information compiled from the previous activities, specify the characteristics of modest, moderate, and elaborate versions of your project. Then describe the cheaper, faster, and better benefits of each one. Finally estimate the cost of each one for design and implementation and for ongoing operations. This abbreviated analysis is far from perfect, but it will give you a great deal of specific information to support your recommendations. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Get ready to prepare a business case Gather together the findings from each stage of your analysis and review them all. Even if you were involved in all parts of the project, you may find that new information and insights emerge only when you step back and look at all the evidence together. Look for patterns, reinforcing information, conflicts, and gaps. Try to state the main findings in a few sentences or bulleted key phrases to create a framework for the final analysis and recommendations. Get all the people on your team together to discuss the results and to decide how best to present your key findings and recommendations. Your business case will need to answer several questions in a direct and convincing way in order to garner support from decision makers. Try to state the answers to the following questions in a few powerful sentences and then organize your detailed data to back them up. I What are the characteristics and dimensions of the problem to be solved or initiative to be undertaken? (Review your service objectives.) I What are the expected benefits of solving this problem or taking this initiative? (Review the strategic framework and features and functionality analysis.) CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES I What alternatives are available and how do they compare in terms of cost, feasibility, effect on stakeholders, and other key criteria that matter to decision makers? (Draw from cost-performance and stakeholder analyses.) I What course of action do you recommend and why? (Use the results of tests and evaluations.) I What are the one-time and ongoing costs of implementing your proposed solution? (These are part of the cost analysis.) I What are the drawbacks in your recommendation? (Refer to the risk analysis.) I What are the next steps? Chapter 3 presents the essential elements of a business case for IT investment along with recommendations for presenting the case to various audiences of decision makers and stakeholders. 31 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Chapter 3. Preparing a business case This chapter will help you pull your analysis together into a convincing argument, or business case, for your project. We define a business case as a well-reasoned argument designed to convince an audience of the benefits of an IT investment, while educating them about the changes, costs, and risks that will be part of the effort. The goal of your business case is to inform key players about your initiative and convince them to support it in some specific ways. Choose a "good" problem How to Make Smart IT Choices Phase 1 Understand the problem and its context Specify your program or business objective Identify and assess stakeholders Analyze the problem or process to be tackled Phase 2 Identify and test solutions Find relevant practices, tools & techniques Develop and test alternative solutions Phase 3 Evaluate alternatives and make smart choices Compare risks Compare costs and expected performance Make and explain final choices Present results in a business case CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES First we outline the essential components of any business case, illustrating them from our experience with one project that aimed to improve the way a state agency supports the financial health of local governments. In the next chapter, we offer some guidance about venues and methods for presenting a business case. Essential Elements of a Business Case for IT Investment I Problem statement A mission or vision statement I Specific objectives I Preferred approach I Expected benefits I Performance and progress measures I Risks and ways to address them I Plan of work, timeline, milestones I Project management and staffing I Cost estimates and funding sources I Alternatives considered I Opposing arguments and responses I 33 A complete business case is a package of information, analysis, and recommendations. It includes a plain language statement of the problem to be solved, with key data to illustrate its public policy significance, as well as its severity and complexity. It also identifies customers and other stakeholders and how they are affected by the problem. The case clearly states assumptions, estimates, and other weaknesses in your underlying data. It presents the options available to the decision maker, comparing features, costs and benefits, and stakeholder impacts for each option. The case concludes with a recommended course of action and a justification that presents its strengths and weaknesses. The business case package includes a variety of presentations, both oral and written, with supporting media such as handouts, slides, or demonstrations. Your business case distills weeks or months of work. You need to be armed with all the data, but you will also need to present your findings and recommendations in a cogent, convincing, and interesting way. The best analysis can be entirely misunderstood if the presentation is disorganized, overly technical, or too mired in detail. Decide what the key points are and build your presentation around them. You can always add detail in response to questions. Essential elements of a business case A strong business case includes all of the following elements: I A brief, compelling, service-oriented problem statement I A mission statement or vision of the future that addresses the problem 34 I A description of the specific objectives to be achieved I A description and rationale for your preferred approach I A statement of the benefits that address the concerns of all relevant stakeholders I Measures for gauging improved performance or progress toward each objective I A statement of the likely risks of your initiative and how they will addressed I A basic plan of work with a timeline and key milestones I A project management plan and names and roles of key managers I Alternatives considered and how they would or would not work I Cost estimates and potential sources of funding I Opposing arguments and your responses to them Table 1 shows how the analysis described in Chapter 2 contributes data to the business case. The analysis has given you a great deal of information to use in the case-building process. Take advantage of it to help you tell a coherent story about the investment that will turn your ideas into action. However, the business case does not emerge automatically from the analysis. You still need to select and organize the material, put it in context, and do some additional planning (such as developing a high-level work plan and management approach). CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Table 1. Building a business case from your analysis Business case element Suggested data sources Problem statement Mission or vision statement “Good problem” description Hopes and fears exercise Strategic framework Visioning Environmental scanning Hopes and fears exercise Process analysis Self-assessment Stakeholder analysis Strategic framework Specific objectives Preferred approach Expected benefits Cost-benefit analysis Cost-performance analysis MAU model Process analysis Prototyping SWOT analysis Technology awareness sessions Cost-benefit analysis Cost-performance analysis Stakeholder analysis Performance and progress measures Baseline measures Benchmarking Best and current practice research Process analysis Strategic framework Risks and ways to address them News analysis Partisan analysis Positioning charts Prototyping Risk analysis Scenario building Stakeholder analysis Technology awareness sessions Plan of work, timeline, and key milestones Project management and staffing Cost estimates and funding sources Draw from all learning during the analysis Draw from all learning during the analysis Best and current practice research Cost-benefit analysis Cost-performance analysis Stakeholder analysis Benchmarking Best and current practice research Modest, moderate, elaborate alternatives Prototyping Technology awareness sessions Best and current practice research Cost-benefit analysis Cost-performance analysis Partisan analysis Positioning charts Prototyping Risk analysis Stakeholder analysis Alternatives considered Opposing arguments and responses CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 35 Problem statement A problem statement clearly defines the problem, need, or opportunity. When developing a problem statement for your business case, the key is to state the problem in terms of public service. You want to explain, for example, how the public is impacted by the inability of justice agencies to easily share information, or how the lack of network availability prevents local governments from receiving timely payments of state funds. Find several true stories that illustrate the problem and its consequences for real people. Draw on the process maps from your analysis to show how and why the problem occurs. Sample problem statement Division of Municipal Affairs is primarily responsible for overseeing the operations of local governments in NYS. It has a staffing level of approximately 210 employees to serve over 10,000 local governments. Historically, the Division has operated as a decentralized organization with seven regional offices and one central office. Consequently, information is received and distributed at numerous points throughout the Division. This has led to islands of information with little or no information sharing. As a result, Municipal Affairs staff who have contact with local government officials are not always aware of their previous contacts with staff in other offices. Field examiners, in particular, do not have access to all correspondence that has been sent to local officials. Duplicate paper and electronic files are being maintained by Municipal Affairs staff in each of the eight locations. Consequently, staff are not able to consider the implications of prior communications while providing current services. Staff cannot identify statewide policy and program issues, select the best services for a particular local government, or perform uniform risk assessment. Lack of information makes emerging local fiscal issues difficult to spot, wastes staff resources on duplicate or inappropriate services, and prevents them from assisting local governments to protect their assets. 36 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES A mission or vision statement Just as an architect has a mental picture of the building she wants to create long before she begins drawing, you must have a vision of how your project will impact the future. This vision of the future can be described by answering the question, “How will things be different when this problem is solved?” Illustrate how public health, child welfare, housing quality, or administrative efficiency will be improved once the problem you currently face no longer exists. Again, make sure to discuss your vision in terms of program and process objectives such as public service improvements, process efficiency, or paperwork reduction, not just technology. Sample vision statement The Division will have a single source of complete historical and current information on the status of all local governments, maintained in a standard way and accessible from any location. Staff who provide local services will understand the context and historical development of current situations and be able to select services that best meet local needs. Staff who oversee the municipal affairs program will be able to conduct trend analysis and program evaluations based on complete information about local fiscal operations. Statewide programs, policies, and resource investments will be targeted to those needs. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 37 Specific objectives Once you’ve described your vision of the future, you must define the project objectives that will help you realize that future. While “improved public safety” or “reduced taxes” are admirable goals, they are too general. You need to express your project goals in specific terms that people will understand. Using the results of your strategic framework or other detailed analysis, identify the key goals of your proposed project. State them briefly and in plain language, and then elaborate as needed to fully explain them. Sample statement of objectives To provide Municipal Affairs staff with remote and desktop access to up-to-date electronic indexed information about local government contacts that allows staff to: I conduct targeted and mass dissemination of information to local governments I assess the need for service delivery to local governments I document contacts between MA staff and specific local governments I develop and maintain a single reliable contact list As a result: 38 I local governments will receive useful information provided by or through the Division I staff can determine risk assessment I staff can maintain a contact history with local governments I staff have timely and accurate information in order to provide consistent services to municipalities CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Preferred approach The next step is to describe how you will solve the problem and achieve your vision. Write a brief statement that describes the approach you plan to take. A statement of approach includes the: I participants and their roles I customers or beneficiaries and how they will be affected I methods and strategies to be used I innovations and other changes needed to solve the problem Your statement should begin with a sentence or two that convey the essential elements of your approach. It then addresses those elements in more detail. When discussing your approach, describe the key factors that underlie your choice. For example, you may need to address the following issues: I how stakeholders will be involved I how decision making will be managed I how needed skills will be used, developed, or acquired I how prototyping or pilot testing will be used I where existing systems and processes fit with the proposed system, or how they need to be changed I how necessary data will be identified, shared, and integrated I how funding will be acquired, used, and managed Sample summary of a preferred approach While many different stakeholders have an interest in this problem, we believe an approach that first meets the information needs of the Division lays the best foundation for serving local needs. The underlying weaknesses of our current information resources lie in a lack of records-oriented policies, practices, and business processes. We therefore recommend an approach that begins with the appointment of a Contact Information Manager to oversee the development of an overall records management policy, an electronic records policy, redesigned business processes, and the selection and implementation of electronic document management software. The second element of our approach is the establishment of a Municipal Affairs Communications Center and a statewide 800 number, staffed by customer service representatives to handle or direct all incoming calls, mail and correspondence. Third, we should create a comprehensive customer information directory database available to all staff and to other agencies who deal with local governments. This approach entails reorientation and retraining of existing staff as well as the addition of some new positions. It also requires significant redesign of current business processes and practices and the development and field testing of software systems to support them. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 39 Expected benefits The benefits of solving your problem or reaching your goal are an integral part of your business case. People want to know how your project will help them. You should identify and discuss the benefits of change. Some typical benefits include reduced costs (perhaps from reducing redundant tasks such as data entry), better decision making at each step of a process (perhaps due to more accurate and timely information), or improved efficiency (thanks to fewer steps to process a transaction). While some benefits can be realized by all participants collectively, it is also important to identify benefits that are specific to each of your stakeholders. Wherever you have numbers or targets for improvement, but sure to include them. Sample summary of benefits Overall, the project we recommend offers the following benefits to the Division: I More consistent policy and action I Ability to set and monitor performance and impact measures I Triage capabilities for problem situations I Accurate and reliable local information for needs assessment and action I Historical and statewide information for planning and evaluation I Improvement in day-to-day internal communication I Reduction of redundant records and thereby storage needs Benefits to local governments include: 40 I Services delivered on the spot I Early problem detection I More timely service delivery I Consistent information I Better background information for services to be provided I More informed service selection I Better context for planning engagements I Increased local awareness of available state-sponsored services CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Performance and progress measures Performance measures give your stakeholders a concrete way to assess how the project is doing relative to their expectations, and identify where improvements are needed. Examples of performance measures include indicators of customer satisfaction, cost efficiency, time savings, dollar savings, improved accuracy rates, and quicker case dispositions. Remember that many initiatives have the potential to save money, but they can also be expensive to design and implement. These up-front costs make it especially important to capture the intangible benefits—such as increased public confidence. In order to retain support and funding beyond the initial approvals, state how and when you will give progress reports against the performance measures established in your business case. Example of performance measures When the project is completed we expect to have achieved the following performance targets: Service measures: Response to local inquiry: acknowledge, record, and assign action on the same day as the call Response to request for standard information: sent electronically the same day or by mail the next day Staff access to history of local situation: fully electronic, on demand Mission measures: Quarterly analysis of statewide and regional trends in local financial management issues Quarterly analysis of technical assistance activities and results CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 41 Risks and ways to address them Risks are inherent in the implementation of any project. Use your business case to demonstrate that you know the risks, and how to mitigate them. Based on your risk analysis, develop a statement of risks you’re likely to encounter on this project, and identify methods for addressing each one. Explain how the approach you have chosen reduces the risk or at least takes it into account. Anticipate the kinds of questions people will ask about risks and have ready answers based on your analysis. Sample discussion of risk This project challenges the entire Municipal Affairs Division to rethink its approach to its mission. We are already in the process of moving from a regulatory approach to one that is more focused on technical assistance. However, staff are very comfortable in creating their own ways of working, gathering, and recording information. The staff in the central office and especially in the regions must accept the value of adopting standard practices and of sharing information across individuals, regions, and functions. To help assure their acceptance, we have involved representatives of every region and all the associated functions in the planning and analysis phase of this project. Their needs have been identified and taken into account and their ideas for improvement are incorporated in the plan. The project team must continue to work in this collaborative way. In addition, some changes may need to be imposed when cooperation cannot be achieved in other ways. The Contact Information Manager and the Division Director will take responsibility for identifying these issues and recommending needed action. 42 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES A basic plan of work, timeline, and key milestones Like a blueprint that guides construction, a well thought out plan of work is a critical component of your business case. The plan of work must take into account the existing infrastructure, funds, staff, time constraints, and other changes required to make your vision of the future a reality. The plan of work allows you to begin to identify the management model that will be used and the resource implications of that model. The use of a collaborative management model, for example, will involve activities and milestones related to creating networks of participants and conducting project activities in a collaborative way. The statement about your plan of work should also include a section on efforts to coordinate resources with other information initiatives in the area. The plan of work becomes a data source for the identification and estimation of cost categories and time estimates for the project. Timelines are an effective way to show how long it will take to complete each step of the project. Fill your timeline with important project milestones, which serve as attainable shortterm goals and evaluation points that keep the project heading in the right direction and on schedule. These milestones also help keep people’s interest in your project, especially if it will span several years. Think about how you will demonstrate the achievement of each milestone as it occurs so stakeholders can see what has been accomplished with their support. Sample work plan excerpt By the end of phase one of the project, we will have completed the following: I Hired a Contact Information Manager I Developed internal records management policies for the Division I Presented and discussed these internal policies with the rest of the agency I Completed process improvements for the following Municipal Affairs activities: Technical assistance request Newly elected local official orientation Advisory financial review CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 43 Project management and staffing Your statements about the management of the project may focus on the key methods of coordination and decision making. This approach could include the formation of a coordinating body that represents the many interests involved; it also helps shape the project, and guides it through the complex world of power, politics, and bureaucracy. Describe how existing staff in every specialty will be prepared for changes by orientation, training, peer consulting, or other methods. Identify functions that are likely to be outsourced or handled by consultants and how these will be managed. Pay attention to internal competing or shifting priorities for human resources. A project director must take responsibility for the project, manage the activities, and direct the staff. Your project director must be capable of implementing the project effectively, and be acceptable to all parties involved in the effort. The qualifications and responsibilities of the project director must be carefully described in the business case. Consider the management approach for the project in terms of project management and human resources implications. If this is a collaborative, interorganizational effort (as more and more government initiatives are), relationship development, joint decision making structures and rules, and a variety of communication methods will be needed. These needs, in turn, suggest the kinds of staff skills and management techniques you will require. Pay special attention to the “people” components associated with your initiative. Explain how you will deal with the general shortage of IT professionals and the fierce competition for skilled people posed by the private sector. Sample project management excerpt The project will be led by a newly appointed Division Contact Information Manager who will report to the Division Director. The Manager will work with a team drawn from all the regions and the related functional areas of the agency (e.g., IT Division, Communications Office). An advisory committee made up of local officials and outside records management experts will be appointed and consulted quarterly. For the first phase of the project, work can be accomplished with existing agency staff. The system design and development phase may require the use of contractors assisted by in-house staff who will assume responsibility for ongoing operation and enhancements. 44 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Cost estimates and funding sources Anyone evaluating your project proposal will have questions about it, but two questions you will hear often are: “How much will this cost?” and “Where will the money come from?” An evaluation of costs and benefits is essential information to provide in your business case. Your cost estimates should cover all elements of the project: human resources, technology, consulting, training, physical plant changes, and so on. The analysis must also assess the impact of ongoing costs, such as training and maintenance, and related activities. Securing funding for your project is likely to be a complex and creative process. While there are often several state and federal sources of funds, you may not find one single source of funding for your project. A multiple source funding model may be the only way to accomplish your goals. Making your case to several “funders” and assembling a mix of resources is sometimes your best road to success and may also help ensure the long-term viability of the project. Example of a cost summary Total project costs are estimated at $783,000 for development and implementation. Thereafter, recurring annual costs will be approximately $190,000. The initial costs include contracts for system design and development and customer service training for the Communications Center staff. New staff positions include the Communications Information Manager and one Research Analyst in the Municipal Affairs Division, and one Systems Analyst in the IT Division. All other positions are existing, although extensive retraining will be needed in some areas. Funding will be requested in the upcoming state budget. Costs can be offset by early implementation of the 800 telephone service for internal communications, cutting the costs of long-distance telephone service to the regions. Early implementation of the consolidated contact list will reduce duplicate printing and mailings. We will take advantage of the agency’s existing plan to deploy a statewide Intranet service and desktop upgrades to provide the technical infrastructure for the new system at no additional cost. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 45 Alternatives considered Even though your analysis points to a particular approach, it may have competitors. Detail any acceptable alternative approaches that will achieve your future vision. It is also helpful to describe your decisions about some potential approaches that were considered and discarded. Sample discussion of alternatives We considered several document management system designs to assess their ability to improve performance and turnaround time on local requests for assistance. Our analysis showed that none of the systems, no matter how powerful they were technically, would generate much improvement without first making significant changes in our underlying information and records management policies and in our business processes and practices. We therefore moved away from a technical solution as the first step, and will implement a system after the needed policies and process improvements are in place. 46 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Opposing arguments and responses In addition to all the questions you’ll hear, you are likely to face some opposition to your project when you present your business case. Your earlier analysis that identified points of contention and alternative ways of looking at the issues will help you prepare to defend your decisions. Expect those issues and alternatives to be raised by one audience or another. Anticipate their reactions and be prepared to respond to them in as informed a way as possible. Have solid data to back up your position and show how the advice of recognized experts or the experience of other jurisdictions supports your project. You should also listen carefully to concerns and be willing to hear and consider new ideas that might improve your plan. Examples of opposing views and responses The project offers benefits to all stakeholders, but there are some opposing views: 1. The project should wait until the entire agency has considered its records management policies. Response: This project can be considered the pilot for an agency-wide effort and will generate near-term benefits for one of our most visible programs. 2. Local governments will lose the personalized service they have been receiving from the regional offices. Response: Services will actually become better targeted to local needs when complete information is available to staff in the field. In addition, best practices will become known and may be applied more widely. Division staff will still apply their individual expertise and continue to have close working relationships with local officials. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 47 The work of generating a business case has benefits beyond the business case itself. For example, we have seen the process of building a business case result in a total redirection of effort when one team realized belatedly they were trying to provide a service for a constituency that didn’t need it. Another team discovered that the nonprofit organizations that would use their planned system wanted to be considered more than system users— they wanted to become project partners and offered to participate in the analysis and the development of the case. In other less dramatic instances new insights into the nature of the problem or the resource situation emerged. Bad timing showed up in the recognition that top management attention was focused elsewhere, perhaps on an upcoming election or its aftermath. The 48 case-building process is a last opportunity to look hard at the data and see gaps or weakness in your thinking, as well as to identify possibilities that had not been obvious before. Finally, think of your business case as a portfolio of layered and related information rather than as a single document. As you work with different constituencies to make your case, you will draw from your portfolio the information and level of detail that is most suitable to each one. Sometimes a single-page briefing is right. For others you need a full blown, detailed justification. Your case portfolio should support whatever kind and level of information you need for each situation. Chapter 4 suggests ways to use this portfolio of information to present your case to a variety of audiences. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Chapter 4. Presenting your business case This chapter offers advice about ways to approach key audiences, get on their agendas, and make good use of opportunities to present your case. To do this effectively, you need to customize your basic message by focusing on selected elements of the case that you know interest or concern specific stakeholders. You also need to decide what medium and venue will best enhance the delivery and reception of your message. Your aim now is to get your case on the agenda of all your audiences—whether they are agency executives, budget officers, elected officials, fellow professionals, community organizations, the media, or the public. By presenting your business case, you will CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES educate stakeholders about your initiative and how it will improve services, administration, or quality of life. This is your opportunity to turn your business case into support—in the form of funding, staffing, advocacy, and energy—from various stakeholders. You are probably very familiar with the decision processes, norms, and expectations of your own agency, which of course is your first audience. The techniques in this chapter will help you make a strong case in those internal processes. Often, you need to engage external audiences as well. We highlight some of them here and offer specific advice about working with them. A more detailed description of Audience and Presentation Skills can be found in Part Two beginning here. 49 Identifying the audiences for your business case Understanding the political culture of your environment is important here. Your stakeholder analysis should have shown how political decisions are made, who is likely to make or influence them, and how to get access to the decision process. If the prevailing political culture puts a premium on public meetings, then a “knock-out” public presentation may be in order. If a crucial decision maker establishes a position on an issue by studying it herself, then you need time to talk with her. If she relies on staff to gather and assess information, then you need to find the person who plays this role and sell him on the issue. Here are some practical tips that will help you schedule your presentation with these other stakeholders. Elected officials and policy staff The schedule keeper is your key resource for setting up a meeting with an elected official or key policy advisor. Call to set up a meeting a week or two in advance. Be prepared for questions about what you’d like to talk about, the group you represent, and who would be attending the meeting with you. Once your meeting is set up, you may want to fax or e-mail a brief fact sheet and a list of people who will be attending. While most constituents want to meet personally with the elected official, it’s often just as important to meet with the policy advisor or budget staff member who works in your particular issue area. These individuals have the expertise and necessary access to advise the elected official on the best course of action. Often the appointed staff influences a policy maker’s opinion on any given matter. Thus, your ability to shape the staff’s views on a subject may matter much more than a brief face-to-face meeting with the elected official. Plan for presenting your business case Prepare a separate plan for each audience that answers all of the following questions: 50 I What are the key program and policy concerns of this audience? I What related activities do they engage in today? I Who should I call to set up a meeting or presentation? I How much time will I have and what type of presentation should I prepare? I What logistical preparations need to be done for this meeting or presentation? I Who should I invite to the presentation/meeting? What role will each person play? I What materials do I need to provide? I How, when, and with whom should I follow up? CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Professional staff The support of other professionals who will interact with or be affected by a system is also needed. Their understanding and support for it is therefore essential. One way to formally present your case is at professional organization or union meetings. Sometimes staff meetings present a good opportunity. Many staff also receive publications from the various organizations to which they belong and these may be good outlets for presenting your case. This group is often overlooked or under-appreciated and the consequences can be very severe. Be sure to cultivate and listen to their concerns. Community organizations When a new or changed system affects people at the community level, you should identify representative organizations and reach out to them. Contact the presidents or one of the leaders of the community organizations you wish to speak to, give them a brief overview of what your business case is about, and tell CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES them you’d like to speak at one of their meetings. These groups can help you market your business case in the community by participating in events and building the grassroots network. Involve these groups on an ongoing basis—their support and encouragement will be needed throughout the project. The news media The most effective way to deliver your message to the widest possible public audience is through the news media. Generally, your agency’s public information office is the conduit for this kind of communication. You can assist in this effort by helping to prepare press releases, op-ed articles, letters to the editor, and press conferences. In working with the press, anticipate the questions and problems they will raise. For example, “Why did you let this problem develop?” or “With all the things government should be doing, why should money be spent on this project?” Be ready to provide answers or solutions for each one. 51 Presenting your business case Once you get on the agenda, do your homework. You know a lot about your own agency audiences and decision processes, but you may need to address others as well. By doing some research on the Internet, in the public library, by looking through a legislative directory, or reading newspaper clippings, you can find out a number of things about other audiences before you walk into a room to make your presentation. Your presentation should address the concerns that they may have with your policy area in general and with your IT initiative in particular. structure in his district. If you are meeting with a specific government agency, you should have a good understanding of their day-to-day operations and what their role would be in the project. If you are presenting to a community organization, ask what types of projects the group has sponsored in the community and request a list of recent speakers. If you are trying to get your story picked up by a media outlet, you should have a good understanding of how they cover stories in your policy area and how they feel about spending on government IT projects. If you are meeting with an elected official, you should know his political party, the committees he sits on, his occupation, and the governance Table 2 summarizes the kinds of presentations that are well suited to each kind of external audience. Table 2: Presentation methods suitable for external target audiences Audiences Preferred delivery methods Elected officials and policy advisors Presentations – address a legislative committee or a group of elected officials Meetings – elected officials and/or their staff members may prefer to meet in their office or a conference room; the meeting is likely to be short Related professionals Presentations – at regularly scheduled meetings of professional organizations, associations, unions Newsletter articles – written for inclusion in regular professional publications Informal networks – actively encourage your staff to talk about the project with their colleagues Community groups Presentations – at their regularly scheduled membership meetings Informal networks – actively encourage community group members to talk to family, friends, and community residents about the benefits of the project Local and regional news media Press conferences – creating newsworthy events for your project Press releases – announcing newsworthy events Op/Ed (opinion/editorial) articles – enlisting community leaders to write articles that appeal for public support for your initiative Letters to the editor – clear and brief letters highlighting key points of the business case 52 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Presentation meetings. Most often the venue will be a meeting. When planning the meeting be sure you know who will participate, who will speak, what they will say, what you want to accomplish, what specific actions or decisions you want from the official, and any other important issues. Create and send out an agenda to all participants, and plan to arrive at the meeting place with enough time to set up and become familiar with the meeting room. Be sure to brief all those who are attending the meeting with you about their roles and what you expect them to do. Assume you will only have a few minutes to present your case. It is important to give a brief overview of your case, highlighting the problems, solutions, and benefits associated with the initiative. The details of your case— the perceived risks and the nuts and bolts of your initiative—should be included in the printed materials you leave behind. Be prepared to answer those questions in case they come up during your presentation. Have someone in your group record the comments and questions raised, and the main points of the discussion. Leave a brief, well-organized information packet behind. Include such items as news stories and editorials, a cost/benefit analysis, and proposed legislation or memorandums of support. And be sure to send a letter of thanks in which you briefly restate the issue and relate any progress that has been made since your meeting. Give a name and telephone number to call for any questions or additional information. Computer-enhanced presentations In certain venues, you may want to present your case using presentation software. Regardless of the technology you use, keep your presentation simple and direct and focused on the concerns of your specific audience. It helps to have an outline from CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES which to build your slides. One general rule to follow when creating your slides: less is more. Screens crowded with words or special effects are difficult to read. Stick with one typeface, or two at the most. Make your key points with simple, short bullets. Be sure to test the computer and projection equipment carefully, at the presentation site if possible. Always have either backup equipment or alternative media (such as overheads) available. Op-ed articles and letters to the editor Local newspapers often provide space for community leaders to voice their opinions on topics. An op-ed piece provides an avenue to clearly present an argument for a particular project and backs it up with facts. An op-ed piece will be most effective coming from a recognized leader in the community. A published op-ed piece is a good addition to the press packets and printed materials you use as you continue to market your business case. Letters to the editor can help increase awareness of your project and keep it fresh in readers’ minds. Letters to the editor should be brief, and they should cover the main themes of your message that need to be repeated often to take hold with the public. Press releases Newsrooms receive dozens of press releases every day, so reserve these for newsworthy events and limit them to one or two pages. A press release can be used to announce key milestones in your effort, such as the support of a new champion, the introduction of legislation, the receipt of funding, or the implementation of the project. Press releases should include the details—the who, what, where, when, why, and how—of your story and a contact name and number for reporters to call if they have questions. 53 Articles for specialty publications Publications like professional association newsletters provide a captive audience for your business case. These publications look for articles on current events affecting their members. Contact the editors and offer to write a short article for inclusion in their next issue. 54 Be prepared for all kinds of questions One good way to know that you are fully prepared to present your case is to ask yourself questions you think your audience will ask. Put yourself in their places and look for gaps, mistakes, confusion, past experiences, and points of view that could lead to questions about your idea and its feasibility. Table 3 lists some common questions. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Table 3. Frequently Asked Questions Experienced professionals, elected officials, and public policy leaders say you can expect any of the following as questions when you present your business case. Can you answer them? I What is this about? Why is it important? I What are the risks? I What are the benefits? I How will this improve services? I What exactly is the problem? How can it be resolved? I What are the long-term vision and goals? I What is the time frame for completing the project? I How long will it take to see results? I Can you define the scope of the project? I What are your milestones for showing progress? I Whose bureaucratic empire is being built by this initiative? I Who will manage this initiative and how? I Who else supports this initiative and why? I Who is against it and why? I Are all the participants on board? I Where have projects like this one been successful? I Does legislation need to be written to accomplish the goal? I Can you specify the policy hurdles? I What levels of government will it affect and how? I In view of data privacy laws and issues, how much and what kind of information should/can be shared? I What do the relevant elected officials think about this initiative? I Where does this project fit in with the Governor’s / Mayor’s / County Executive’s policy agenda? I Have you been to see the Legislature / City Council yet? I What role will your agency head play in this initiative? I What are the opinions of other agencies, constituencies, employees, and unions? I How much will it cost and where will the money come from? I Didn’t we [your audience] fund this already? I What about the millions of dollars we’ve already invested in this program in the past several years? I What other funding is available? How are you pursuing it? I What is the Total Cost of Ownership (TCO)? (building, maintaining, training, etc.) I What will be the Return on Investment (ROI)? I What are the true total costs? I Is there a way to share the costs? I What do you want me [your audience] to do? I How will this project help achieve other policy goals that I [your audience] care about? I CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 55 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Making Smart IT Choices Part Two: Smart IT Skills, Techniques, and Tools Introduction In this part of Making Smart IT Choices, we briefly present an array of skills, techniques, and tools that can help you develop the information that you need to build a solid business case and make sound decisions. First, we present general skills for working with groups and presenting to an audience. These skills apply at every stage of a business case development project. We tell you what they are, what they can do for you, and what their limitations are. Then we discuss a series of techniques that can help you begin gathering the information you will need to make a case for an IT investment decision. Next we present a collection of tools that help you work through each of the three phases of analysis that lead to the creation of a business case. Some of these tools will be familiar to you already while others may be new. For each tool we tell you a little about it is and what it can do for you. We also point out its limitations, since no tool is perfectly suited to every need. For some tools, (ones we have found easy to apply and valuable for everyone to know) we also offer brief how to’s that lead you through the basic steps. For other tools (those that need special training or the assistance of an expert), we give you overview information and some advice about where real experts can help you. For almost all of the skills, techniques, and tools, we point you to print or electronic resources to learn more. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Skills for working with groups In almost all government work settings, groups of people must work together to solve problems or achieve a common goal. Skills that help groups work together are invaluable in gathering information, identifying differences, exploring alternatives, and focusing support on a specific project. Group facilitation Government IT projects typically involve dozens of people making hundreds of decisions. People with vastly different work styles, backgrounds, and talents are often brought together, asked to form cohesive groups, and charged with solving problems. But people’s differences and group dynamics can make it difficult for the group to fulfill its mission. That is why a skilled facilitator can be helpful in leading a group through the necessary steps to make effective decisions. What is it? Intensive working meetings. Facilitated processes typically involve working meetings that use structured decision processes. These decision conferences allow for rapid elicitation and a combination of expert judgment and baseline empirical data from multiple people and points of view. Specialized group management roles. A facilitator interacts directly with the participants by leading a group through activities and discussions designed to elicit ideas, encourage discussion, or lead to a decision. A facilitator is someone trained in group processes and methods to build a group’s capacity for managing its own activity. The facilitator typically works with a group for a limited time to build its capacity for effective work or to accomplish a specific task. A variety of tools and techniques are used by the facilitator to keep the group on task and moving through the process. A way to help a group establish and work through an agenda. The facilitator is responsible for the quality of the group interaction and for helping the group reach satisfactory its goals. Thus, the facilitator has to leave his or her biases, opinions, and ideas at the door and concentrate on the group’s needs. What is it good for? Carrying out a variety of group tasks. You can use facilitated group meetings for a variety of tasks, including: generating alternatives, coming up with priorities, describing cost-benefit scenarios, allocating resources, developing budgets, devising strategic plans, identifying potential problems and solutions, and planning project timelines. A skilled facilitator can make it easier for a group to work through these often complex tasks. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 59 Managing the process. Often, people who are invested in a project need an outsider to help them work through the tasks duscussed above. Freed up from process issues, which are handled by the facilitator, the group can concentrate on the tasks at hand. Improving information quantity and quality. A group, as a whole, has more information than any one individual has, and groups are better at catching errors than individuals are. Most important, the group process helps participants identify important terms and concepts, and it helps them explore their differences before formulating judgments or making choices. Managing conflicts. Some conflict is a given in any group work situation. Often, conflict signals a need for more information. Whether conflict is caused by differing experiences, opinions, personalities, or missions, a good facilitator can explore conflicts while diffusing tense situations and keeping the process moving in a productive direction. Some limitations and considerations Cognitive and judgmental biases. Group decision conferences are subject to known cognitive and judgmental biases. “Group think” for example tends to reduce the number and variety of ideas that are expressed and explored. Research has shown that an accumulation of individual thinking usually results in more ideas for consideration than when a group does not allow time for individual reflection. Consequently, a facilitator should include some time for individual issue or idea generation as part of the overall process. Blind spots. Facilitated meetings may also be hindered by “blind spots.” Depending on the composition of the group, individuals may be less willing to be candid with respect to issues and opinions than they might be if their anonymity were ensured through some other type of issue identification process. Sabotage. A decision conference can be sabotaged by one participant or a subset of participants who are either uninterested in the problem being addressed or unwilling to consider the potential solutions being discussed. Lack of information. Participants may lack the information required to complete the task successfully. Participants may have difficulty envisioning the interactions between the units of a complex process or system, or they be unable to assess the level of effort that will be required to bring about substantial change. In addition, people may substitute assumptions for facts without being explicit (or sometimes even aware) about these substitutions. 60 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES For more information Debus, M. (1990). Handbook for Excellence in Focus Group Research. Washington DC: Academy for Educational Development. Hackman, J. Richard (2002) Leading Teams: Setting the Stage for Great Performances. Cambridge, MA: Harvard Business School Press. Morgan, D., R. Krueger, and J. King (1998). Focus Group Kit. Volumes 1–6. Thousand Oaks,CA: Sage Publications. Reagan-Cirincione, P., S. Schuman, G. Richardson, and S. Dorf (1991). “Decision modeling: Tools for Strategic Thinking.” Interfaces 21, 52-65. Rohrbaugh, J. (1992). “Cognitive Challenges and Collective Accomplishments” in R.P. Bostrom, R. Watson, and S.T. Kinney (eds.), Computer augmented teamwork: A guided tour. New York: Van Nostrand Reinhold. A comprehensive list of reviews of group facilitation books is available at reviewing.co.uk/reviews/group-facilitation.htm [Retrieved July 7, 2003]. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 61 Consensus building, collaboration, and decision making Projects do not happen in a vacuum. Working with groups from other agencies and organizations is often required to successfully plan and implement an information technology project. These tools are ways of answering the question, “How can we help work teams function effectively?” Consensus finding and building tools are often needed to help a team explore different views and conflicting objectives or interests. Teams also frequently need models for collaboration, especially if they have never worked together before. What are they? Meeting management methods and ground rules. Effective meeting management involves ground rules, agendas, clear purposes, facilities planning and preparation, careful recording of results, notification of members, and communication. Methods for conflict resolution. Conflict is a normal part of group work that can result from adversarial relationships, different interests, or both. Techniques for conflict management include diagnosing the causes of the conflict, mediation, negotiation, and problem solving. Mediation involves helping the parties understand the possibilities, communicate effectively, and recognize opportunity for compromise. Negotiation provides a framework for finding a mix of compromises that will resolve the conflict. Well understood rules and guiding principles can be particularly effective in conflict negotiation. What are they good for? Eliciting information, brainstorming. Facilitators often use nominal group process to elicit information from group members. In a nominal group process exercise, the facilitator provides an opportunity for each group member to contribute to the discussion and share ideas. The results can be prioritized by voting methods in which all group members have equal influence on the results. Multi-voting, where each group member can vote for more than one choice or has multiple votes to distribute, can be very effective for complex decisions and help avoid forming factions within a group. Understanding issues, resolving conflicts. Consensus building tools are useful in facilitating the two key requirements for reaching agreements in a work group setting, including identifying and understanding issues and resolving conflicts. Some of the tools for identifying and understanding the issues—SWOT (Strengths, Weaknesses, Opportunities, and Threats) analysis, hopes and fears exercises, strategic framework, and stakeholder analysis—are described in this handbook. 62 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Some limitations and considerations Effective group process takes time. It is unreasonable to expect new groups to accomplish substantive work immediately. It is usually necessary to invest in building skills, sharing understandings, and commitment to the group process. Vulnerable to disruptive, subversive behavior. Individual members (who want to do so) can wreak havoc on a group’s efforts to work collaboratively. Without effective internal controls and norms, such behavior can derail group efforts. Right people at the table. Good decisions about who is involved in group processes are often critical to success. It is possible to have too much as well as too little participation. Choosing the most effective group size and composition requires careful consideration of the needs of the group and the participating organizations. For more information Chrislip, D. D. and J. Parr (2002). Collaborative Leadership. San Francisco, CA: Jossey-Bass. Johnson, D. W., and F. Johnson (2002). Joining together: Group theory and group skills. Boston, MA: Allyn & Bacon. Schein, E. H (1998). Process Consultation, Vol. 1. Reading, MA: Addison-Wesley. Schwarz, R. M. (2002). The Skilled Facilitator. San Francisco, CA: Jossey-Bass. Schweiger, D. M., and W. R. Sandberg. (1991). “A team approach to top management’s strategic decisions.” In H. E. Glass (ed.), Handbook of Business Strategy, 6, 1-20. New York, NY: Warren, Gorham and Lamont. Straus, D. (2002). How to Make Collaboration Work: Powerful Ways to Build Consensus, Solve Problems, and Make Decisions. San Francisco, CA: Berrett-Koehler. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 63 Audience and presentation skills It is an old Boy Scout saying, but particularly true here: “be prepared.” In order to effectively present your project, you need to know about the audience. Make sure you are prepared to address the concerns that they may have about your project and convince them that it is worth their time, attention, and money. What are they? Ways to prepare good presentations. The first step on the road to giving a good presentation is being prepared. Ask yourself questions you think your audience will ask. Put yourself in their place and look for gaps, mistakes, confusion, past experiences, and points of view that could lead to questions about your idea and its feasibility. Methods for identifying audience concerns, biases. Think carefully about your audience. Your presentation style and content may change depending on who is in your audience. Some of the audiences you may encounter include elected officials, appointed policy makers and key staff, government colleagues, professional organizations and unions, community groups and organizations, private sector interests, media organizations, and the public. Ways to manage meetings. When planning a meeting to deliver your presentation, be sure you know who will participate, who will speak, what they will say, what you want to accomplish, and what specific actions or decisions you want from the audience. Create and send an agenda to all participants, and brief all those who are attending the meeting with you about their roles and duties. It is important to give a brief overview of your case. The details should be included in the printed materials you leave behind. Be prepared to answer questions. And have someone in your group record the comments and questions raised, and the main points of the discussion. Computer-enhanced presentations. It helps to have an outline from which to build your slides. One general rule to follow when creating your slides: less is more. Stick with one typeface, or two at the most. Make key points with simple, short bullets. Be sure to carefully test the computer and projection equipment at the presentation site if possible. Have backup equipment or media available, since technology failures can occur at any time. Effective deliveries. Be yourself and be enthusiastic about your project. Think of your delivery as an interaction with your audience, rather than a presentation to the audience. Make eye contact with the members of the audience instead of looking at the overheads. Speak clearly and at a pace somewhat slower than normal conversation. Let your confidence, conviction, and support show. 64 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES What are they good for? Effective communication. You already know all about your project. But knowing about your audience helps bring you much closer to securing the support you need for it go forward. Allow your project and audience knowledge to drive your presentation. Knowing what to present and how. A thorough knowledge of the audience, its concerns, biases, and priorities will help you tailor your presentation. The contents and presentation style will vary depending on your audience. It is helpful to have a core set of basic facts and materials that you can customize for each audience. Some limitations and considerations Information-gathering problems. You may run in to roadblocks when it comes to gathering information about certain agencies and organizations. No matter the cause, you may have to make-do with what you can find. The key is to use whatever information you have to your advantage when designing your presentation. Running out of time. No matter how prepared that you are for your presentation, you still can not control the amount of time that you will have with your audience. The meetings and presentations that went before yours may run long, or other circumstances with your audience may squeeze the amount of time with your audience. It is vital to be prepared for abridging your presentation and providing material that you can leave with your audience for when they have more time to focus on your initiative. For more information www.nwlink.com/~donclark/leader/leadpres.html is a site devoted to giving presentations [Retrieved June 11, 2003]. www.public-speaking.org/public-speaking-articles.htm is another comprehensive public-speaking site [Retrieved June 11, 2003]. www.presentations.com/presentations/index.jsp this is a site with many articles about presenting [Retrieved June 11, 2003]. www.ukans.edu/cwis/units/coms2/vpa/vpa.htm University of Kansas’ “Virtual Presentation Assistant” [Retrieved June 11, 2003]. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 65 Information gathering techniques The following are ways to collect quality information before making an IT investment. Library and document research Many information management problems and proposed IT solutions are not unique. Organizations and project teams can benefit greatly from the experience of others in both government and in the private sector. In addition, researchers have conducted and documented many studies that can offer insight into your project and ways to approach it. Information about all these kinds of experiences is published in periodicals, books, journals, or databases. What are they? Systematic searches of print and electronic publication sources. The purpose of these searches is to identify, review, analyze, and evaluate material that might inform thinking about a problem facing your organization. Library-based searches. A quick consultation with a reference librarian at the outset will help you focus the search, identify the most relevant print and electronic access tools, and be sure that you are not overlooking any new information sources. You may want to access the library’s magazine holdings, databases, newspaper articles, or government document sections in addition to looking at professional journals and books. Subscriptions. Many professionals subscribe to periodicals or news services that help them stay on top of developments in their fields. Electronic news services are becoming increasingly popular and deliver daily or weekly summaries to your e-mail inbox, often with links to more complete stories. What are they good for? Identifying best practices. A good review of relevant publications should help you uncover best practices, as well as analytical and theoretical frameworks. Learning more about possible solutions. Your review should reveal something about current “state-of-the-art” solutions that may help you solve your agency’s problem. Research journals in particular should present objective evaluations of the performance of a given management strategy or technology. Identifying potential expert advisors. There are probably a number of organizations and individuals whose experience you can draw on for advice or consultation. Use your library and document review to identify these possible advisors. Also look for organizations that may allow you to see in operation a particular technology or process that you are considering. Avoiding mistakes. The more you learn about the potential pitfalls in system design or implementation, the easier it is to avoid them. 66 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Identifying potential partners and vendors. Use this kind of research to identify and locate partners in government and academia, as well as potential vendors and consultants. Some limitations and considerations Need to define correct scope, keywords. One of the keys to effective and efficient literature reviews is identifying an appropriate search scope and keywords. Starting too narrowly is often more effective than starting too broadly. If the scope is too narrow and nothing can be found, you can easily make your search less specific. This is preferable to wading through hundreds of potentially unrelated documents in hopes that one or two relevant items will surface. Time lags. There is often a substantial time lag between the completion of a document or report and its appearance in print. For scholarly paper-based journals, this can be as long as 24 to 36 months between submission of the finished manuscript and its publication. Electronic peer-reviewed journals involve a much shorter time lag. Basic research skills are needed. The organization of indexes, abstracting journals, library catalogs, etc. is complex, and the inexperienced searcher can easily overlook relevant material or sources. For more information Gerstenfeld, S. “Chapter 6: Literature Review,” in Handbook for IQP Advisors and Students. www.wpi.edu/Academics/Depts/IGSD/IQPHbook/ch6.html [Retrieved June 11, 2003]. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 67 Internet research The Internet can be a powerful tool in your effort to develop and implement IT solutions for your business problems. Whether you want news about the latest privacy policy or e-commerce applications, a comparison of available software packages, or contacts in other government agencies, the Internet is the place to look. Developing your ability to quickly and easily search the Web will allow you to tap into its vast resources. Search the Internet to identify other organizations that are solving problems similar to yours. Try to dissect these examples to see if the way you are thinking is on track. Compare the example with your project. Look for dissimilar organizations with similar problems, as there is often a lot to be learned here. Finally, think about the technologies or management methods that might be useful in your project and search for organizations that are known to use them well. This kind of investigation is designed to leverage your research effort with known experience from a variety of other places. What is it? Using search engines. Web searching involves using Internet search engines to locate pertinent information about a given topic. Using various keywords and phrases, these sites search the Internet for Web sites, publications, listservs, and other resources about your topic. The Internet is also a valuable source and may actually yield more useful information than a library search in terms of identifying people and organizations working on IT implementations or specific business problems. Information found on the Internet may also be more current than those journal references that have gone through a lengthy review process. Many Web pages often provide direct e-mail access to the source of the information, making it easy to contact them. The search capabilities of these engines are constantly enhanced. Take advantage of what technology has to offer and let the Web work for you. Listserv searches. A listserv is a subscription-based electronic mailing list where individuals who share similar interests can post and respond to messages. Relevant listservs and their respective subscription instructions can be identified through Web-based searches. A posting to a listserv asking for information on a specific business problem or technology is a very effective and low-cost way of reaching a large targeted audience. If you are unfamiliar with the codes of conduct in using listservs, either subscribe to the list and watch for a while before your first posting, or refer to one of the many resources available that discuss “netiquette.” What is it good for? Finding best/current practices. Use the Web to find out how other organizations have dealt with problems similar to the ones you want to solve. Groups like to publicize their successes. Online magazines, newsletters, and journals often contain success stories and case studies. 68 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Identifying contacts. The Internet is a good way to locate other government and IT professionals who can be great sources of advice and inspiration. Use search results to identify knowledgeable people you should talk to. Most web sites include contact information. Get in touch with these professionals by e-mail or phone and interview them about their experiences. Information technology options. All kinds of hardware and software are available to fix business problems. By conducting a Web search, you can get information about all the different technology options available to you. You can prepare yourself with the knowledge to make informed choices. Some limitations and considerations Write a good query. The information you get out of a Web search is only as good as your query. You must write a targeted and well-defined search query in order to locate relevant resources. Variability of results. Since search engine databases vary in size, frequency of updates, and search capabilities, using different engines can lead to results that vary considerably. Information quality and accuracy. Anyone can create a Web site and fill it with whatever content they wish. It is important to make sure you know the source of information you access on the Internet. Try to find data from recognized and respected institutions, such as universities, corporations, government agencies, media publications, and professional organizations. How to search the web Formulate your query. Take a few minutes to think about your problem or goal. Try to break it into key elements or sub-topics. Write down a series of words or phrases that describe your topic using different synonyms. Also think about issues that are associated with your goal. This will help you develop a more comprehensive picture of the issue that has some reasonable limits. Conduct a broad Internet search. Start by accessing some of the major search engines and use different keywords and word combinations. Once you have searched a topic, narrow your results by searching within the search results or formulating a more advanced query. Remember how easy it is to follow link after link as you pursue a lead until you have forgotten where you started. Have a pencil and paper handy to keep track of the sites you visit. Use the bookmark feature of your browser to easily return to the most useful ones later. Search within discussion groups. Do not forget to search the discussion groups of main search engines to see if the issues you are interested in have been debated within these groups. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 69 Join listservs. Look for a listserv corresponding to the subject you are researching. Subscribe to it and use this forum to ask who knows about or has had experience with your issue. Regularly monitor lists that seem to provide good ongoing discussion of your topic. Tips for an efficient search. The results you get from a search on the Web are only as good as your query. By using the following tips to write a query, you may get more targeted and useful results. Quotation marks. If you are looking for an exact phrase or group of words, such as American Marketing Association for example, make sure to use quotation marks: “American Marketing Association.” Otherwise, the engine will search each word separately. AND. Use AND (uppercase) when you want to make sure your results contain two terms. If you use AND in your query, the search engine will retrieve only documents that contain both words. OR. Search your subject using different synonyms to maximize your chances of getting results. Using OR (uppercase) will allow you to enter several synonyms for the search engine to use as it will retrieve any of the words you typed. +Sign. Some search engines will ignore short words included in a query, such as: in, of, a, out, with. Using the plus sign will ensure that all words are searched. For example, if your search is: women in government, make sure to type: +women +in +government. - Sign. Use the minus sign if you want to make sure your results will not contain a certain word. For example, if you are looking for information on marketing strategy but do not want to get results from consulting companies, you can type: +marketing +strategy – consulting. * Sign. You can use the star sign to truncate a word. For example, if you are looking for marketing consultants or marketing consulting companies, you can type: +marketing +consult* and the search engine will retrieve results with any word starting with consult. Combination. You can use a combination of signs or words to do your search by putting terms in parenthesis. For example, you can type (American or U.S.) AND presidency. 70 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES For more information A detailed overview of how to conduct Internet research: www.tbchad.com/resrch.html [Accessed June 27, 2003] We have found the following search engines to be particularly helpful. Alta Vista: www.altavista.com Yahoo: www.yahoo.com Google: www.google.com Hotbot: www.hotbot.com Teoma: www.teoma.com The following metasearch engines, which provide more extensive coverage by searching simultaneously in several of the largest search engines’ databases, have proven to be useful.: Debriefing: www.debriefing.com Dogpile: www.dogpile.com Profusion: www.profusion.com Ask Jeeves: www.askjeeves.com CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 71 Surveys When you need structured information from a large number of people, surveys are the way to go. They allow you to quickly collect data from many people in different locations. They can be used to inform project direction, focus effort, and to build empirical support for a hypothesis. Surveys usually employ short, simple questions, and they generally offer a series of answers from which participants choose. For example, a survey could ask respondents how strongly they agree or disagree with statements about the potential impact of a new project. Or you might offer multiple-choice questions that ask people to choose among alternative responses. What are they? Structured questionnaires. Surveys can be self-administered questionnaires that are distributed by hand, mail, fax, or e-mail. They can also be administered face-to-face or over the phone. Regardless of what form they take, surveys are designed to gather information from a representative group of people. Methods for getting information from a defined population. Sampling is an important part of survey planning and design. For example, if you want to learn what people over 65 years of agethink about your new statewide initiative, you could try to find and ask every person in this age group. However, this is not likely to be logistically or financially feasible. Instead, you will probably ask a random sample of the population of senior citizens. If your sample is drawn with statistical precision (a topic much too complicated for this note), you can then generalize the results from this smaller group to the entire population of people in that age group. Sometimes the population you want to know about is accessible and small enough to be surveyed directly. For example, you may want the opinions of all the people in your bureau about their usage of e-mail. In this case, you would send a survey to every person – a 100% sample. What are they good for? Gathering opinions and demographics. Surveys allow you to assess user, client, or stakeholder opinions and evaluations. You can also collect demographic descriptions of these groups. Supporting qualitative analysis. A survey can help you obtain quantifiable results of subjective concepts such as attitudes, opinions, and values. Low cost data collection. Surveys often require less time, cost, and commitment from the participants compared to other forms of data collection methods. 72 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Keeping respondents anonymous. Like some other methods of information gathering, surveys allow you to preserve the anonymity of respondents. You are more likely to increase the number of responses when people know their opinions are anonymous. Some limitations and considerations Careful design and field testing. The survey questions must be carefully designed and field-tested. You must know what questions to ask and how to word them. Even slightly imprecise wording in the question or the answer choices can give results very different from the ones you seek. The cost of design and analysis. While the cost of a survey is relatively low for participants, a good survey design, execution, and analysis may be expensive for the researcher. Questions must be carefully designed and the answers thoughtfully analyzed, both of which take expertise and time. No chance to discuss answers. Surveys provide little or no opportunity for participants to discuss, elaborate, or explain answers. Respondents who do not understand a question may answer improperly or not at all. Potential problems with the sample. Biases or flaws in a sample can make it difficult or inappropriate to generalize results to the larger population. Absence of sampling. If you do not use statistical sampling, you cannot generalize your findings beyond the group you actually contacted. This is fine when the group is relatively small and well known. For example, if you want to know what every person on your staff thinks about the new proposal, you can ask them all, and report your results without worrying about statistical validity. For more information American Association for Public Opinion Research (AAPOR). “Best Practices for Survey and Public Opinion Research.” www.aapor.org/default.asp?page=survey methods/standards_and_best_practices/best_practices_for_survey_and_public_opinion_ research [Retrieved June 9, 2003]. Council of American Survey Research Organizations (CASRO). “CASRO Guidelines for Survey Research Quality.” www.casro.org/guidelines.cfm [Retrieved June 9, 2003]. Babbie, E. R. (1973) Survey Research Methods. Belmont, CA: Wadsworth. The Survey System had an overview for beginners of survey and questionnaire design at www.surveysystem.com/sdesign.htm [Retrieved July 7, 2003]. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 73 Interviews When you want to gather detailed information about people’s impressions, experiences, ideas, and attitudes, interviews are often the best method. Face-to-face or telephone conversations yield rich information and are a good way to understand complex topics. What are they? Conversations with a person or group. There are several types of interviews. All of them can take place with individuals or groups. You may conduct a structured interview with a series of preset questions and answers from which the participants choose. A semi-structured interview uses preset questions, but allows people to answer them in their own way. Unstructured interviews rely on a general idea of the types of questions you want to ask, but use no predetermined phrasing or order. A computer-mediated conversation, such as an online chat, is another medium used for conducting interviews. Ways to gather and record opinions and experiences. Having a record of the conversation allows for thorough and accurate analysis and interpretation. This may involve the interviewer taking notes by hand or on a laptop computer. The interview sessions may also be recorded with audio or video equipment. It is good practice to ask participants before the interview if you may record the interview. What are they good for? Collecting and recording complex responses. Engaging in a conversation with one or more people is an effective way to gather information about their experiences, opinions, attitudes, needs, and ideas about your project. Encouraging full discussions and explanations. Interviews allow the interviewer and participant to have full discussions of the questions being asked and answers being provided. This give-and-take yields rich, detailed information. Promoting active involvement. Effective interviews encourage participation, establish rapport, and capture observations of non-verbal behavior. Some limitations and considerations Greater time and expense. Interviews are often more expensive and time consuming to conduct than other information-gathering processes. High level of involvement. Interviews require considerable commitment and involvement from participants. 74 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Skilled interviewer needed. Training is required to conduct good interviews. Unstructured interviews, in particular, demand special skills. Higher cost of analysis. Transcription and analysis of interview data can be complex, time consuming, and expensive. For more information Doyle, J. K. “Chapter 11: Introduction to Interviewing Techniques,” in “Handbook for IQP Advisors and Students.” www.wpi.edu/Academics/Depts/IGSD/IQPHbook/ch11.html#11 [Retrieved June 11, 2003]. Fowler, F. J. (1990). Standardized Survey Interviewing: Minimizing Interviewer-Related Error. Newbury Park, CA: Sage. Goldman, A. and S. McDonald (1987) The Group Depth Interview: Principles and Practices. Englewood Cliffs, NJ: Prentice-Hall. Rubin, H. and I. Rubin (1995) Qualitative Interviewing: The Art of Hearing Data. Thousand Oaks, CA: Sage. Seidman, I. E. (1998) Interviewing as Qualitative Research: A Guide for Researchers in Education and the Social Sciences, 2nd ed. New York: Teacher’s College Press. Steinar, K. (1996) Interviews: An Introduction to Qualitative Research Interviewing. Thousand Oaks, CA: Sage. Survey Research Center (1983) General Interviewing Techniques: A Self-Instructional Workbook for Telephone and Personal Interviewer Training. University of Michigan, Ann Arbor: Survey Research Center. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 75 Experiments The essential purpose of an experiment is to learn about what influences the way some process or activity works. The data are typically a result of direct observation of behavior, albeit in a contrived and controlled situation. Experiments put you one step closer to understanding what might happen in a natural setting. The natural setting involves a combination of many interacting influences that make it very difficult to sort out the independent effects of one factor or another. So an experiment is designed to control enough of the factors to allow an assessment of the impacts of the specific factors that are of greatest interest or importance. For an IT system or prototype, an experiment can become part of testing or evaluating system performance. The experimental design would have to provide for the system or prototype to function in an essentially natural way. For computing systems, these experiments often take the form of running a set of highly standardized and tested procedures or software routines that simulate actual use in a controlled way. The experimenters can apply the same procedures under systematically varied conditions, such as running the same simulation on varying hardware configurations. Experiments may also involve hypothetical work or service delivery situations. In such an experiment, carefully selected persons perform a standardized set of actions on a system under controlled conditions. The experimenter can observe and record the results of realistic work behaviors or client transactions. If well designed, such experiments can yield highly useful data for assessing systems and prototypes. What are they? Ways to study what impacts performance. Experiments are artificially constructed and controlled situations designed to study what affects the performance of some system or process. On occasion, a so-called “natural experiment” can be useful as well, such as when a change in the natural setting occurs that impacts the process in the same way as a deliberate experimental manipulation of the situation. For example, if an organization changed a work procedure, but kept the workers, technology, incentives, and work setting constant, a comparison of productivity before and after the procedural change would be a natural experiment. Direct observations of a situation under controlled conditions. Experiments allow you to directly observe and/or measure a situation, such as service delivery or system performance, under controlled conditions. Experimental controls can eliminate or account for the influence of all but the most important components of a system. This allows direct testing and evaluation of these high priority components. 76 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES What are they good for? Observing and measuring activity. The activity of users, clients, and system components can be observed and measured under realistic, controlled conditions. These include: assessing how system performance may be affected under conditions of significantly increasing scale of operations, providing benchmark data for use in evaluating system performance in natural settings, and repeating activities and assessing performance under consistent conditions to test system reliability, stability, and performance. Assessing system influences. Conducting an experiment on your system will allow you to assess the influence on performance or system behavior, including critical components or operational factors. By controlling for, or eliminating the effects of, other low-importance factors, an experiment can illuminate the role of the most critical components in overall performance. Evaluating reliability and stability. Experiments also allow you to assess a system’s performance under low-frequency or extreme conditions. You can apply varied tests or operations systematically to evaluate performance under a pre-determined set of circumstances. Some limitations and considerations Can be costly. Experiments can be expensive to design and conduct. The construction of realistic, controlled conditions may require extensive laboratory facilities, equipment, or similar resources. Materials and protocols must be carefully designed. Participants must be recruited and prepared. The observation, recording, and analysis of experimental data may be very complex and time consuming as well. May require unrealistic assumptions. When conducting experiments, you may have to make unrealistic assumptions in order to accomplish the necessary controls. These can compromise the validity of the resulting observations. For example, experiments often call for participants to assume particular roles, such as business owner or teacher, to include the necessary range of transaction or clients. The ability of the participant to accurately play that role may be quite limited, and the resulting behavior may not be truly typical of people in that occupation. May have ethical constraints. The actions that can be taken in an experiment may be governed by ethical or policy constraints. For example, some designs may be prohibited because they involve unacceptable costs or risks for participants, such as divulging sensitive or potentially damaging information, or being subjected to highly stressful conditions. Validity depends on the controls. In any experiment, the validity of the data depends directly on the effectiveness of the controls. All potential influences on the outcomes must be taken into account or provided for in effective ways. This requires detailed and extensive knowledge of the processes involved, and all the components of the experiment itself. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 77 For more information Babbie, E. (2004). The Practice of Social Research, 10th Edition. Belmont, CA: Wadsworth Publishing. Cook, T. D. and D. T. Campbell (1979) Quasi-Experimentation: Design and Analysis for Field Settings. Chicago: Rand McNally College Publications. Morgan, D., ed. (1993) Successful Focus Groups: Advancing the State of the Art. Newbury Park, CA: Sage Publications. Morgan, D., R. Krueger, and J. King (1998) Focus Group Kit. Thousand Oaks, CA: Sage. 78 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Simulations In the world of IT planning and design, the word “system” is often used to refer to a technical system—a collection of electronic components that function together. Another use of the word “system,” however, refers to the collection of human actions and interactions that create the social and managerial systems within which technical systems operate. These social and managerial systems are often characterized by a high degree of complexity, variability, and uncertainty. System simulations provide a structured approach to analyzing and understanding how complex social and managerial systems give rise to problem behaviors, as well as what types of solutions might be applicable to those problems. What are they? Ways to describe and understand feedback. System simulations are structured, computer-based, analytic approaches to describe and understand the feedback complexity in social and/or managerial systems. They show, for example, how a policy influences behavior which in turn influences costs. System simulation models are constructed using specialized simulation software that help users understand these complex, often hidden, relationships. Time plot simulations support to planning. System simulations explicitly model delays in system functioning as well as feedback loops. This can help you predict and plan for the future behavior of a system under a wide variety of circumstances. What are they good for? Examining complex systems. System simulations help create models of a problem or a solution when the level of social or managerial complexity surrounding a system is high. Identifying problematic behaviors. Simulations allow you to identify patterns of inter-agency or inter-organizational decision making that are causing or contributing to a problematic or undesirable pattern of behavior. Understanding how a technical system fits into a social and managerial system. System simulations can help your project team understand how a technical system will fit into, and function within, a complex social or managerial system. Often, the simulation can help to identify forces within the social system that will result in a system that does not solve the identified problem. This can lead to a redesign of the technical system or to a redesign of the business processes that support the managerial system. Considering causes and possible solutions to problems. Your project is more likely to succeed if you understand the root causes of the problem, and how one or more proposed solutions may (or may not) act over time to solve or reduce it. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 79 Understanding your part. These models will help members of your management team understand how their decisions contribute either to the creation of problem behavior within a system or to the resolution of it. Some limitations and considerations Expense. System simulations can be expensive and time-consuming to construct. They require specialized expertise and tools to build and analyze. Mixes hard and soft data. Because system simulation models integrate a diversity of data sources, ranging from hard data about time to softer and more qualitative estimates of human factors, the output of such models can be confusing and hard to interpret. Requires high involvement of key actors. Usually all of the key actors in a system need to be involved in the construction of a system simulation. Bringing all the key people together for the amount of time that is required to build a system simulation as a group exercise may be logistically impossible or prohibitively expensive. How to know if the time and expense of a system simulation are justified Because system simulations require assistance from a skilled modeler, and because they are more complicated and expensive than some other analytic processes, it is useful to have some criteria for knowing when such modeling efforts are warranted. Here are some indicators of when the benefits of such an exercise will probably outweigh the costs. I when complex feedback is involved, I when actors from different parts of the system don’t readily agree on the root cause of the problem(s) or the possible solution I when the predicted impacts of a system on organizational performance are uncertain or when serious risks are involved I when the costs of making a mistake in implementation are unacceptably high I when the problem is an important one that has attracted the interest of top management For more information Morecroft, J. (1988) “System Dynamics and Microworlds for Policymakers.” European Journal of Operational Research 35 (3): 301-320. Roberts, N., D. Andersen, R. Deal, M. Grant, and W. Shaffer (1983) Introduction to Computer Simulation: A System Dynamics Modeling Approach. Reading MA: Addison-Wesley. Senge, P. (1990) The Fifth Discipline. New York: Doubleday. 80 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Senge, P. (1990) “Catalyzing Systems Thinking Within Organizations” In Advances in Organization Development, ed. F. Masarik. Norwood, NJ: Ablex. Wolstenholme, E. (1993) Evaluation of Management Information Systems. Chichester; New York: J. Wiley. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 81 Tools for Phase 1: Understanding the problem and its context These tools can be used individually or in combination to assist an individual or organization during the analysis that precedes the development of a business case for an IT investment. Visioning Visioning is a tool you use to establish an image of what you want your organization or project to look like in the future. The time frame associated with the vision depends on the needs of the group and may range from months to years. What is it? A way to stretch your thinking. Creating a vision is a way to stretch your organization and establish a vision of a “preferred state.” Growth in terms of size or scope of operations may form part of a vision, but doesn’t always constitute a vision. The circumstances facing your group need to inform the vision. Being realistic is important, as is remembering the concept of stretch. Ultimately, the vision should guide or frame the work that all participants will need to do in order to accomplish the desired outcomes. Various methods. The task of visioning can be completed in several ways. Regardless of which method you use, your main focus is to develop ideas. You must get everyone to share their ideas, reach a common understanding, build consensus, and craft a meaningful vision statement. What is it good for? Developing shared goals. Vision statements are often very good at getting everyone on the same page. The process of constructing a vision statement involves discussions and interactions that will help the group reach consensus on ultimate goals. Reflecting interests, needs, and skills. Remember that vision statements should reflect your interests and be attuned to their specific needs and capabilities. Otherwise, the likelihood of accomplishing the vision will be greatly reduced. Team building exercise. In short, a well-crafted vision statement that has buy-in from everyone involved is often a crucial first step in the beginning of any group project. Some limitations and considerations We’ve done this before. Almost everyone has been through a process like this at one time or another. Some of the most prolific buzzwords around involve the words “vision,” “mission,” “empower,” and “group consensus.” Depending on previous experiences, the level of cynicism may be very high when an exercise like this begins and may remain high even when a vision statement is developed. 82 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Address skeptics. Perhaps the best advice is to directly address participants’ cynicism. Let them know that they are in the room to make things different. Participants have to find a way to cooperate and take responsibility for the outcomes of their efforts. Predict the future. The final pitfall associated with the visioning process is that people are often poor prognosticators. Time and experience may require revisiting and modifying the vision. The real key here is to see the vision as a dynamic statement and not simply as a motto meant for hanging on the wall. Consequently, visioning should be used periodically as a project unfolds. How to use visioning for a project 1. Use a round robin facilitation format to elicit people’s thoughts about the characteristics they want to see embodied in the project. 2. Display all of the responses from step 1 (adhere them to a wall or project them to a screen). 3. Clarify what is being expressed in each characteristic, but avoid debate at this time. 4. Establish one or more small groups to take the characteristics and develop a vision statement(s) that reflect the key ideas. 5. Encourage the full group to discuss all of the statements and begin developing a common vision upon which the group can agree—this is when debate begins. 6. Repeat steps 4 and 5 until you produce a vision statement that satisfies your needs. For more information French, W. L., and C. Bell, Jr. (1999, 6th ed.) Organization Development: Behavioral Science Interventions for Organization Improvement. New Jersey: Prentice Hall. Kakabadse, A., F. Nortier, and N. Abramovici (1998) Success in Sight: Visioning. Boston, MA: International Thomson Business Press. Quinn, R. E., S. Faerman, M. Thompson, and M. McGrath (2003, 3 rd ed.) Becoming a Master Manager: A Competency Framework. New York: John Wiley & Sons, Inc. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 83 Hopes and Fears exercise Hopes and Fears exercises are techniques that help members of working groups share their perspectives on the task at hand and build a common understanding of goals and potential problems. This technique is similar to a number of other group and team building facilitation exercises. It is a way to help answer the question, “What’s on the minds of those who have to accomplish this thing?” What is it? Ways to share hopes and fears for project outcomes. Hopes and Fears exercises are used to make explicit the various hopes and fears associated with the project. The discussion generated from this activity can include ideas about how to ensure that the most important hopes are realized, as well as how to prevent the most important or costly fears from being realized. What is it good for? Building common understanding of goals, barriers, and enablers. These exercises are most useful in building a work group’s shared perspective of its expectations and potential problems. This is particularly important near the beginning of a project. It is at these early stages that the members are likely to have the greatest differences of opinion about what they are supposed to do. Sharing perspectives for effective work. Coming to a more thoroughly shared perspective is necessary for effective group work and communication among its members. It is also useful to identify where hopes held by some members may be unattainable or even inappropriate to your overall goal. Preventing sources of frustration. If unrealistic or inappropriate hopes are identified early in the process, they are less likely to become sources of frustration and resentment that can interfere with your group’s effectiveness. It is also reassuring to some members to learn that others in the group share their respective fears. Simple and effective icebreakers. Hopes and Fears exercises are simple, unthreatening activities that are useful as icebreakers for new groups. They allow the members to learn about each other and begin group interaction smoothly. The process of eliciting individual members’ hopes and fears, and giving them credence also emphasizes the value of each person’s contributions and can promote more enthusiastic participation in subsequent activities. Drawing on intuitions and knowledge of the environment. These exercises bring out the background, experiences, and personalities of participants in ways not possible through formal modeling techniques. 84 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Some limitations and considerations Reluctance to reveal feelings. Since these exercises are commonly used for new groups, members may be hesitant to reveal their hopes and fears to an unfamiliar group of people. Skillful facilitation is a necessary ingredient. A good facilitator is required to get the hopes and fears process moving. Even with skillful facilitation, it is likely that some members will withhold information for strategic purposes or simply from embarrassment. The full range of hopes and fears may be unavailable for discussion. Smaller groups are most effective. Because the process depends on active and relatively free-flowing discussion, it is inappropriate for very large groups. How to conduct a Hopes and Fears exercise 1. A facilitator or team leader first asks each member of the group to articulate his or her hopes for the outcomes of the project. 2. The hopes are then organized into related clusters on a wall or other display, and each item is discussed to be sure that it is well understood by the members. 3. The same process is then used to elicit and discuss the fears. Each member is asked to articulate what undesirable outcomes they fear might occur, followed by clustering and discussion. 4. The facilitator may use rating or prioritizing techniques to help the group identify the relative importance of various clusters of hopes and fears. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 85 Service objective A service objective is a structured way to express the goals of your project. The process of creating a commonly understood and agreed upon service objective often reveals differences in thinking, different assumptions, and conflicting perspectives that all must be discussed and resolved before the service objective statement is complete. What is it? Clear statement of project beneficiaries and goals. Once the group has developed a vision or mission statement, the service objective sets a course for them to reach specific goals. Developing a service objective is both an individual and a group activity. What is it good for? Reaching agreement on intent. This exercise provides a framework for discussing the similarities or differences in team members’ intentions or expectations. You can use it to determine if you are expressing the same intent in different ways or if members of the group have different objectives in mind for the project. Developing common language. Once common intent is agreed to, expressing that intent in common language increases the likelihood that group members will talk about the project in the same terms to people who are outside of the group, such as agency colleagues and decision makers. Creating a foundation for future action. The service objective provides powerful reminder about key goals that each member can work toward when completing his or her specific tasks. Some limitations and considerations Agreeing too soon. Often people hear what they expect to hear when talking about broad project objectives. Failure to explore the underlying intent of each participant can lead to a service objective that is not really a shared understanding, but just a placeholder for each person’s individual ideas. Beware of too-quick agreement and encourage debate and discussion before adopting a service objective. How to create a service objective using a “Madlib” exercise A Madlib is a fill-in-the-blanks sentence that we often use to generate a service objective statement. The sentence we use is this: Our service objective is to provide (who) with (what) that allows them to (action) so that (outcomes). 86 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES This format forces the group to specify who the project is designed to help, how it will help them do something specific, and how that specific action will result in something of value. When all the blanks are filled in, a service objective emerges. It usually takes many iterations before the group is satisfied that they understand and agree on the objective. The following steps will take you through the process: 1. Each individual should write down what they believe the objective of the project is using the Madlib format. 2. Each individual shares their Madlib with the group by reading it and posting it on the wall. 3. After all of the individual objectives are shared, similarities and differences (and their underlying logic and assumptions) are discussed. 4. Sometimes additional information will be needed to explore and evaluate different versions, so often multiple meetings are needed to allow for this. 5. Finally, the group develops a single agreed upon service objective or a set of related service objectives for the project. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 87 Strategic framework A strategic framework is a structured way to understand a project proposal by helping you clearly define each key service objective and its customers. The framework then helps you identify the resources, partners, and innovations that might contribute to success. To be most effective, the strategic framework should refer to one project-specific objective at a time. Strategic frameworks can be devised by one person and then presented to and reviewed by others, or they can be created through a facilitated group decision conference. What Is It? An analysis of the internal and external factors. The framework leads to an initial identification of potential resources, including partners, and to a closer look at potential uses for information technology and other innovations. To be most effective the framework should be used with one service objective at a time. The strategic framework helps project teams identify four factors (customers, resources, innovations, and partners) that will influence the definition, development, and operationalization of their project goals. I Customers are people or organizations who make use of the service you intend to provide. I Partners are willing participants in a joint enterprise who invest staff time, equipment, money, or credibility in the creation and operation of the service. Partners share costs, risks, and benefits and engage in active, trustful working relationships with one another. I Resources are something of value necessary to the success of the service. When using the strategic framework, it is usually useful to specify what resource(s) are associated with an organization, rather than just an organization’s name. I Innovations are products and services that could be used to design, develop, or deliver a new service or to offer an existing service in a new way. What is it good for? Taking a high-level view. The framework lays out the full array of internal and environmental factors that can support a particular service objective by: 88 I identifying existing and potential partners to help achieve those objectives, I identifying needed information and other resources, I identifying relevant innovative products and services, and I getting more specific about the customers of the service. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Thinking “outside the box.” Its focus on brainstorming resources, partners, and innovations pushes you to think more broadly about what is possible. Recognizing multiple roles. In using the framework, you are likely to find that specific people or organizations appear in more than one place. For example, if your service objective is “to create a repository of geographic data for use by state and local governments,” a particular state agency could supply data to the clearinghouse (i.e., be a resource) and use the clearinghouse to find data provided by others (i.e., be a customer). The same agency might also play a major role in the design, operation, or financing of the clearinghouse (i.e., be a partner). As a result, you will eventually need to discuss issues related to developing and managing relationships with an organization that may play a combination of roles. Often these different roles will be played by different people or units within the same organization. Refining objectives in light of what customers need and what the environment has to offer. Another strength of the strategic framework is its capacity to reveal different “points of view,” or POVs. You can experiment with this idea by completing separate frameworks from the POV of two different customers. (In the clearinghouse example above, these might be state agencies and local governments.) Compare the results. You are likely to find that the resources, partners, and innovations that would make one customer happy, are not entirely the same as the ones that would satisfy the other. Understanding this disparity will help you sharpen your service objective and will become an important point of departure when you move on to define the scope of your entire project. You may need to narrow your scope in a way that makes it customer-specific—or broaden it to include features needed to satisfy additional kinds of customers. Some limitations and considerations Focuses on “enablers,” but ignores barriers. You are more likely to identify barriers through modeling, prototyping, and best practice reviews. Lacks the detail needed to craft a project plan or design a system. Most importantly, this tool does not deal directly with the availability or cost of identified innovations, resources, or partners. It focuses your attention on what is possible rather than what is practical. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 89 How to construct a strategic framework 1. State your service objective as clearly as possible in the center box. If you have more than one objective, create more than one framework. 2. Then fill in the factors that are important for achieving that objective. I Who are or will be the customers of the service? Are they external, internal, or both? I What information and other resources (human, material, financial, political) will you need? I What innovative service approaches, technologies, or other products might be useful? I Who might be your partners in this endeavor? Keep in mind that the same people or organizations can appear several times in different roles. A customer might also be a resource supplier, for example. 3. Look at the results and ask yourself the following questions: I Are we trying to serve more than one kind of customer? If so, which is most important? I Who needs to be on the project design and development team? I Do we have or can we get the required resources? I Is there a good match between our customers’ capabilities and the technologies we propose to use? I How will we engage in partnerships? I Have we pushed ourselves to think broadly about each factor, or are we staying with what we already know best? I Does this picture make sense? 4. Based on your answers, refine your approach and decide when and how to proceed with your project. Strategic Framework Resources © Service Objective © Customers © © Partners Internal External Inside the Agency External Information Other Resources Products 90 Innovations Services CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES For more information Andersen, D., S. Belardo, and S. Dawes (1994) “Strategic Information Management: Conceptual Frameworks for the Public Sector.” Public Productivity and Management Review, 17 (Summer) 4, 335-353. Center for Technology in Government (1996) Developing & Delivering Government Services on the World Wide Web: Recommended Practices for New York State. Center for Technology in Government, University at Albany, SUNY. See Chapter 4, Case 3, Example 3A for a completed strategic framework and Exercise 1, Using a Strategic Framework to define a problem. www.ctg.albany.edu/publications/guides/developing_on_the_web/ developing_on_the_web.pdf. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 91 Stakeholder Analysis Stakeholders are individuals and groups who are affected by or have influence over your initiative. Every project needs a careful assessment of stakeholders in order to understand who cares about it, how they can affect it, and how they will be impacted by it. What is it? Structured examination of who cares. Stakeholder analyses are structured examinations of the relationships between a proposed project and key players in the environment. These analyses are ways of answering the question, “Who cares about this project and why?” Anyone who has an interest (a “stake”) is considered a stakeholder. Stakeholders can be examined in terms of their roles, degree of support for the initiative, influence over decisions or resources, or the ways in which the project will affect them in both positive and negative ways. A variety of tools can be used for this analysis–two (positioning charts and partisan analysis) are described in the sections below. What is it good for? Refining project scope. The results of a stakeholder analysis may expand or reduce the scope of a project design and strategy. Often planners have a good understanding of internal stakeholders, but less appreciation for the needs and influence of external ones. A stakeholder analysis gives them information to take all important stakeholders into account. Refining service objective and strategic framework. A stakeholder analysis tests the usability of the service objective and strategic framework that has been set by the group. Because a service objective focuses on a single user or beneficiary, planners often find that a stakeholder analysis reveals additional users (with different needs). Consequently, additional service objectives are needed for a complete understanding of the project. Examining impacts in order to design a better plan. There are many stakeholders in the environment of a government program, and most information systems have multiple features or products that will affect stakeholders in different ways. Some will see increased access to services, or better quality service. Others may experience higher costs or more competition for scarce resources. A stakeholder analysis can help you anticipate these effects before a full-blown project gets underway. Expanding understanding of the environment. Most organizations are better at understanding internal dynamics than external ones. The stakeholder analysis forces you to look outside your organizational boundaries to estimate the impacts and outcomes of a new initiative. Predicting potential results. The stakeholder analysis requires you to be specific about how various elements of a proposal will affect stakeholder groups. It helps you move from very general descriptions to more specific and measurable ones. 92 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Identifying high-priority features and stakeholders. Once you understand the different ways the proposal will affect different stakeholders, you should be able to see which areas need priority attention. You should also be able to identify measures of how your initiative will impact different stakeholders and estimate the magnitude of those effects. Some limitations and considerations Assumptions required. The analysis requires assumptions about causal relationships and processes. Since you have imperfect data, make educated guesses about causes and influences. Keep testing these assumptions as your project proceeds. For more information Mitchell, R., B. Agle, and D. Wood (1997) “Toward a theory of stakeholder identification and salience. Defining the principle of who and what really counts,” Academy of Management Review, vol. 22, pp. 853-866. Tennert, J. and A. Schroeder (1999) “Stakeholder analysis,” presented at 60th Annual Meeting of the American Society for Public Administration, Orlando, FL. The Management Sciences for Health and UNICEF have a guide on how to conduct a basic stakeholder analysis. erc.msh.org/quality/ittools/itstkan.cfm [Retrieved May 27, 2003]. The UK Department for International Development has an extensive guide to doing a Stakeholder Analysis at www.dfid.gov.uk/foi/tools/chapter_02.htm [Retrieved May 27, 2003]. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 93 Positioning Charts Positioning charts show the relationships among people, groups, or other elements of a problem in terms of their positions. A positioning chart is a good first stakeholder analysis tool. The chart usually shows two factors important to the problem, support and power. Stakeholders are arranged in the chart according to where they fit on the two dimensions. As shown below, placing stakeholders on a positioning chart helps identify what different approaches or strategies will be most effective for the different positions. For the sample chart, different strategies can be chosen for dealing with different stakeholders according to whether they have high or low support for the proposal, and by their criticality to its success. This kind of analysis can show that resources could be wasted on trying to generate greater support from those with little power to help, or failing to recognize antagonistic stakeholders who could damage prospects for success. What are they? Exercises to classify stakeholders along key dimensions. Positioning charts can be useful in mapping where particular stakeholders fit into the project. Knowing who has the inclination to champion the project and who has power and/or inclination to bring it to a halt can be helpful as you develop a project management plan. Stakeholders Support High Low Priority Supporter Low High Power Low 94 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES What are they good for? Understanding potential influences. This type of chart allows you to better understand how various stakeholders can influence your initiative and give you a start on developing strategies that take those influences into account. Communication. Representing this kind of analysis in a positioning chart is not only a good planning exercise, it is also an effective device for communicating the results to others. Equally important, because it tells you important things about various stakeholders, it helps you devise communication strategies for working with each one. Some limitations and considerations Somewhat arbitrary process. Placing stakeholders on the chart is often an inexact, even arbitrary process that relies on experience and judgment. Wrong assumptions and lack of confirming information can lead to substantial errors in positioning, which results in flawed conclusions. Oversimplify relationships. A chart may also oversimplify relationships in a complex setting, especially when more than two dimensions are involved or the relationships are not stable over time. For more information Bryson, J. M. (1995) Strategic Planning for Public and Nonprofit Organizations. San Francisco, CA: Jossey-Bass. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 95 Partisan analysis A partisan analysis recognizes that competing interests and conflicts are natural and unavoidable in any significant government action. Any new project requires careful attention to the partisan or political nature of the process. What is it? An inexact science. A partisan analysis uses a number of different approaches and ways of thinking about interactions. Like many of the tools presented here, a partisan analysis is more a craft than an exact science. However, some basic questions can guide the analysis. Way to determine wants and needs of participants. Partisan analysis includes finding out what participants want in general, or what they stand to gain or lose. Method for understanding a wide range of issues. Partisan compromises often involve negotiation over a wide range of issues that may be unrelated to the immediate concern. In legislatures, this is referred to as logrolling. It is also important to understand both individual and organizational interests and desires. Those who speak for a group or organization do not necessarily share all the group’s desires and objectives. Framework for identifying key relationships. Projects typically involve parties with existing relationships and histories. It is important to know who are friends and enemies, where natural alliances and rivalries exist or may form, and what kinds of coalitions are possible or desirable. Consider where trust has developed or been betrayed and where old friendships or wounds will shape current perspectives and actions. These issues are often critical to forming the coalitions necessary to move forward. An activity to find out who has power. A partisan analysis considers what power resources the parties bring to the table. These include: official status or authority; ability to punish or reward other participants; special expertise, status, skills, or reputation; and access to information. It’s useful to know participants’ preferences for different kinds of power and how they have acted in the past. Way to analyze rules of the game. Effective strategies for managing projects depend on knowing what kinds of actions are acceptable and what tactics are the most successful in your organizational and political culture. These include preferred styles of negotiation or influencing others, limits or penalties for actions, and understanding the importance of signals and symbols of play. Wild cards. Uncertainty plays a part in any environment. One major element of uncertainty is whether any outside actor or force will affect your plans. Partisan analysis often involves scanning the environment for possible external factors that may become involved. This scanning can also include analysis of the risks and probabilities of these kinds of events and the potential range of impacts. 96 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES What is it good for? Planning. Use partisan analysis to plan how to present your ideas (and your business case) to participants and outside audiences, and what to emphasize as your main selling points. You can also use it to decide the timing and format of presentations, what groups to make them to, and when. Collaborating. This approach is an effective planning strategy for forming collaborations and work groups. Strategizing. Use partisan analysis to develop a strategy for addressing potentially controversial decisions and for mobilizing support among participants and stakeholders. Some limitations and considerations The quality and amount of available information. The value of your partisan analysis depends in large part on the quality and amount of information available about the people and groups involved in your project. In most environments, people tend not to announce their true objectives and strategies. In fact, there can be substantial incentives to mask or deliberately misrepresent their true goals and interests. Judgments based on inferences about other people’s goals and interests should be evaluated and tested against actions and other evidence. Lack of definitive answers. Assessing the goals and interests of others involves a lot of uncertainty. There may be discord among groups about their goals and interests. It is often difficult to evaluate the accuracy and stability of statements and actions expressed by all groups involved in the project. No history. Historical information may be an ineffective basis for judgment. In new initiatives, histories may be absent. Information about past actions and events may be unavailable, unreliable, inconsistent, or badly distorted by selective memory or interpretation. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 97 Process analysis The process of studying how work is done is called business process analysis or process modeling. The study of business processes, or the flow of work (workflow), is a very important part of gaining an understanding of how organizations do their work. One of the best ways to document and analyze a business process is by talking to the people who actually carry out the work. Through facilitated discussions, a work process can be documented using graphical representations or models to formulate a collaborative view. These graphical representations (or models) of business processes allow organizations to learn more about the specific steps within a process. This baseline knowledge allows organizations to understand and potentially measure existing processes and to consider alternative ways of improving or changing them. What is it? Diagram of business activities. A business process model is a diagram that visually depicts the sequence of activities (or steps) involved in a particular work process. The goal is to document every step of the process, including decision points, action steps, information flow, documents produced or changed, and products developed or services delivered. It is important to include the appropriate people in the process in order to document their perspective. Documentation of every step in a process. People often assume that everyone in a work group understands the business processes of that group in the same way. Often, process modeling reveals these assumptions and shows that they are incorrect. Activities represented by symbols. Process steps are mapped out using a series of standard symbols that represent various activities. For example, a circle represents information or action that starts or ends a process. Boxes indicate an activity of some kind. Diamonds mark decision points or questions. And arrows are used to show flow within a process. Using these symbols, business processes can be mapped out a variety of different ways. The most straightforward approach is through the use of white boards, flip charts, or paper. Computer-based modeling packages are available for modeling more complex and sophisticated processes. The modeling tool used and the symbols selected are of secondary importance to capturing information from those involved in the process and to using that information to create a model for group review. 98 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES What is it good for? Identifying and analyzing problems. A model helps organizations recognize the underlying causes of a problem, bottleneck, or breakdown in the process. The step-bystep nature of the model allows analysts to look at the individual elements and figure out how to change them to solve problems. The model becomes the focal point of discussion, allowing those involved to walk through the steps necessary to capture the institutional memory or knowledge of the work process. Convening the people connected with a process and documenting it through the creation of a process diagram (or model) results in a broader and more consistent understanding of the process. Assumptions are made explicit, expert knowledge is documented and different perspectives are presented and reconciled. These models can then help you predict where problems may occur and provide you with the capability to add or clarify activities that eliminate or avoid problems. Investigating improvements. Once you have mapped out the steps, you can analyze a process to determine areas that can be improved. A model will show unexpected complexity and unnecessary steps. Identifying these areas can lead to improved workflow and performance. Developing “what if” analyses. Business process models help you compare the perceived process to the actual and the ideal. You can then engage in “what if ” analysis that may lead to changes and improvements. Standardizing to help make the implicit explicit. This type of analysis allows you to standardize activities and decision points to improve workflow and decision making. It also ties implicit assumptions about how the system works to explicit performance measures and structured analysis. Training and communication tool. Business process modeling is a great training tool to help people understand a complex process. It helps people who are new to the process or the organization better understand the flow of work. It also helps managers communicate effectively and efficiently with each other and with those who are associated with the process. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 99 Some limitations and considerations Different levels of complexity. Developing a business process model involves varying levels of complexity. You may only want to map out the key components of a process for historical purposes, or you may need to fully describe every minute detail of the process to support the design of an automated workflow. The completeness and complexity of your model will vary depending on the type of analysis you want to conduct. The answer to “how much is enough” is not always evident at the onset of the analysis. It may only become clear as the modeling begins to unfold. Potential for bias. Typically, the people who operate or manage a process participate in the development of diagrams or models that analyze the process. This personal investment in the work can lead to bias in reporting and analyzing the workflow. It is important to guard against this in the construction of any model. Expensive and time consuming. Mapping out all the steps in a business process is tougher than it may seem. It takes a lot of time and effort to accurately describe all steps required and to come to consensus about that model within a group. You must involve enough people who know about different aspects of a process to get a full accounting of it, and you must have enough of their time to allow them to review the models and improve on them. For more information Fischer, L., ed. (2003) Workflow Handbook 2003. Lighthouse Point, FL: Future Strategies. Hammer, M. and J. Champy (1993) Reengineering the Corporation: A Manifesto for Business Revolution. (First ed.). New York: Happer Collins Publishers. Modell, M. E. (1996) A Professional’s Guide to Systems Analysis. 2 nd Edition. New York: McGraw Hill. Brint.com, a business technology portal, contains a section of resources about business process reengineering and innovation at www.brint.com/BPR.htm [Retrieved June 2, 2003]. The Business Process Reengineering Online Learning Center, hosted by ProSci, contains helpful tool kits, articles, tutorials, and other resources at http://www.prosci.com. [Retrieved June 2, 2003]. 100 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Ericson, J. (2003) “Starting Over With BPM.” Portals Magazine. May 29. www.portalsmag.com/articles/default.asp?ArticleID=4695&TopicID=7 [Retrieved June 4, 2003]. Wreden, N. (1998) “Model Business Processes”. Information Week Online. September 28. www.informationweek.com/702/02iuprc.htm [Retrieved June 2, 2003]. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 101 Self-assessment tools Self-assessment tools include a wide range of methods to gather information about a current situation or performance. They are designed to answer the question, “How are we doing?” These tools range from something as simple as a newspaper survey asking readers to rate their knowledge of the Internet to the kind of year-long institutional selfassessment procedures used in hospital accreditation. The essential element in all self-assessment tools is that only the group being assessed answers the questions. What are they? Self-administered questionnaires. The simplest form of self-assessment is a self-administered questionnaire or test. In order for the assessment to be effective, the questions must be sound and the participants must understand what is being asked and provide valid responses. Informal evaluations. For informal self-assessments, neither the questions nor the resulting interpretation scales have any particular scientific basis. They are at best rough guides. Formal evaluations. More comprehensive self-assessments and accreditation procedures usually involve the participation of those being assessed in setting and reviewing goals. In the typical institutional assessment, the evaluation criteria are a mix of external standards developed by the accrediting body and the institution’s own goals and criteria. What are they good for? Evaluating performance. If well designed, a self-assessment tool can be a highly efficient performance or status evaluation. Once developed, the tools may be used repeatedly without the need for outside intervention. Building consensus and morale. The deliberative processes in setting goals and reviewing performance can have positive effects on the organization by building consensus, enhancing morale, and increasing understanding of operations. Screening for problems. Self-assessment instruments can also be screening devices to identify possible problems or areas for further attention. Self-administered surveys are often used in organizational development work to identify these areas and issues. Planning. Self-assessment can also be used as a planning tool. It can assess past and current progress to inform strategic project management planning. 102 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Some limitations and considerations Hard to validate. Self-assessment tools are difficult to validate. The fact that they produce a measurement or evaluation result does not mean that it is accurate or meaningful. Interpretation of the results must be done with careful attention to the validity of the tools and how they were used. Distorted results. While all data collection results can be distorted in a number of ways, the nature of self-assessments may be more susceptible to distortion. Those being assessed may deliberately provide false or misleading information to promote their own interests; provide inaccurate data due to their own biases, faulty memories, or flawed perceptions; or be unduly influenced by others in the process, either deliberately or inadvertently. Bias of the tools. The tools themselves could be badly designed or insufficiently tested, or there may be accidental but serious flaws in the information produced. The assessment tools could be intentionally designed to favor a particular point of view or desired outcome. Resource intensity. The kinds of information called for by assessment tools can be quite extensive and complex to assemble. While you may avoid the costs of external consultants and analysts, the kinds of internal deliberations involved in institutional self-assessments require considerable staff time. For more information Strategic Planning Process Self-Assessment Questionnaire for Federal/Government Agencies. www.strategicfutures.com/articles/stratpln/gov/quesgov.htm. Retrieved May 27, 2003. Building Smart Communities (1997) San Diego, CA: International Center for Communications, San Diego State University. www.smartcommunities.org/index2.html. Retrieved May 27, 2003. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 103 Models of problems After an IT system is designed, built, and installed, you may discover that it solves only a part of the initial problem or that the problem was poorly understood to begin with. This happens when participants have different mental images, or models of what the problems are and how they should be remedied. A formal model of the problem makes these perspectives more explicit. A model represents a small-scale simplification of the problem being addressed. It can take the form of a system diagram, a process flow chart, a spreadsheet, a set of equations, or even a computer model. What are they? Collective visions of the problem. The formal model is a public visualization of a problem. It allows people to see how their own assumptions of the problem fit into a larger view shared by all. Reproductions of the problem. A good model can represent the essence of the problem. This helps ensure that the problem is understood well enough to begin investing resources in framing, testing, and evaluating solutions. More than one thing. A good model is a collection of perspectives that work together to create a precise description of a problem to be solved. Although there are many different types of formal models, most of them share a number of common features including: 104 I “Stories” of how it works and what’s wrong with it. These accounts typically reflect many different points of view. They can be gathered either through a group process or individual interviews. I Common visuals of the system that needs fixing. These visuals usually take the form of business process maps (see next tool) or structured system flow models. I Numbers and other measures of key variables. Quantitative models need to be explicit about which measurements are key indicators of the problem. I Analyses of sources of the problem. The logical reasons for a problem are often revealed by drawing a diagram of a process or system. The elements of the process can be extracted from verbal accounts of the system and can lead to measures of system performance that reproduce the problem. I Lists of proposed solutions. Ironically, many managers define problems in terms of their solution (e.g., the problem is that we need more people in the field). By collecting lists of what managers think the solutions ought to be, you can more clearly define the problems that are implicit in those solutions. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES What are they good for? Making the implicit explicit. Modeling makes individual implicit assumptions and mental pictures explicit. It also imposes a process on a team’s thinking about a problem that helps prevent premature conclusions. Creating common definition of the problem and possible solutions. The process of modeling serves to focus discussion on the root causes of observed problems in order to develop common definitions and potential solutions. Identifying underlying forces. Models can help analysts and managers come to grips with the causes of problems. Measuring. Modeling pushes the team to identify which key performance variables really count. Developing “what if” analyses. Models help participants see how problems would get better or worse under different sets of possible circumstances. Communicating with external audiences and decision makers. Models help project team members communicate their reasoning to the external audiences that need to be involved in solving the problem or who need to make investment or other decisions. Some limitations and considerations Expensive and time consuming. Some approaches to modeling a problem (or its solution) may require specialized knowledge and techniques that are hard to find or expensive to apply. Level of complexity. Sometimes the models themselves can get so complicated that they can’t easily be understood. Overly complex models don’t help illuminate the core of a problem. But models that are too simple fail to capture enough complexity of a problem to provide new understanding. Bias. Sometimes the modeling approach that a team chooses has a subtle and biasing effect on how they will look at the problem. For example, spreadsheet models emphasize the financial aspects of a problem, whereas process maps tend to look at workflow. Validity. Models can be wrong. When this happens, you have a whole group of people aligned around a view of the problem that won’t yield solutions. Fortunately, this probably occurs less often with models than without them. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 105 How to model a problem: one approach Because there are so many different approaches to modeling a problem, no single way of getting started always works. However, a number of common sense steps can get you far enough down the path to decide whether you can finish a problem-centered model yourself, or need to call in some expert help. 1. Using a facilitated group process, gather each participant’s concerns on paper (or on flip charts or white boards). 2. As a group, create a system flow diagram, business process map, or some other explicit picture that captures these concerns using common vocabulary. This creates a common view of the problem. 3. Once the common view (often a diagram) of the problem has been constructed, use it to elicit discussion about what is and is not important in the system. These discussions can help a group simplify the model or identify its most important features. 4. If you and your team have arrived at a coherent and complete view of the problem you need to solve, and if the model you have developed seems adequate, then proceed toward exploring alternative solutions. 5. If you are developing a quantitative model, now is the time to get some numbers to help tie the emerging model back to the real system. (Getting numbers can be as easy as using group process and expert judgment to calibrate key variables in the system or as complex as using extensive surveys, interviews, experiments, and data analysis exercises to measure critical aspects of system performance.) 6. Once you have a common view of the problem tied to some preliminary numbers, you can begin to test the model by changing some of the numbers or revising part of the process to try to understand what those changes would do to the overall system. 7. If your preliminary analyses are turning up questions that lack answers, if members of the team are arguing about the details of the model, or if a clearly defined problem is not emerging, then you may need to enlist the help of someone with more experience in modeling problems. You may have hit upon one of the many complex issues in the public sector that requires detailed analysis at the early problem-finding stage. For more information Gaylord, R. J. and L. D’Andria (1998) Simulating Society: A Mathematical Toolkit for Modeling Socioeconomic Behavior. New York: Springer. Stokey, E. and R. Zeckhauser (1978) A Primer for Policy Analysis. New York: W. W. Norton. 106 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Tools for Phase 2: Identifying and testing solutions Best and current practices research Best and current practice research involves learning both what works and what doesn’t, based on the relevant experience of others. Often, you may find that your business problem has already been dealt with, in whole or in part, by other government agencies, private or nonprofit organizations, or academic researchers. Identifying and evaluating these solutions are important early steps in your business case development. An abundance of information and expertise in the IT community, as well as elsewhere in the public sector, can contribute to solving similar problems in other organizations. In particular, there is a great deal to learn from those cases where things did not go as well as expected. What is it? A way to find potential solutions. Best and current practice research involves identification and consideration of various approaches to a problem, or the components of a problem, that a project is intended to address. A method for learning from others’ success and failure. Such research may take different forms, but the ultimate goal is to learn from the experience of others so you can avoid “reinventing the wheel” or replicating the mistakes of others. An early project task. This research should be conducted during the business case development process and continued over the life of the project. What is it good for? Understanding the problem. By finding out how other organizations tackled a similar problem, you can develop a better understanding of your problem from multiple and varied perspectives. Learn more about how the problems were addressed, and the success factors, as well as the advantages and disadvantages of the solutions. Finding potential solutions that have already been tried. You can identify individuals and organizations that have solved, or tried to solve, problems similar to yours. You can learn from their experiences and gain feedback on your proposed and ongoing project activities. Identifying methods and resources. Use this tool to identify methods and mechanisms for evaluating a range of solutions, from IT solutions used by different organizations to how departments are structured to deal with problems, to the strategies used for organizing partnerships across agencies. In addition, current practice research is an effective way to identify sources of relevant policy, process, and technical expertise and technology in order to explore the range and variety of technical solutions. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 107 Investigating other resources. Universities and professional organizations have an array of conferences, journals, books, and published studies that may help you identify relevant current and best practices. Classifying all parts of the problem. By identifying all relevant components of a problem, you can avoid the trap of treating the symptoms of the problem instead of the problem itself. Some limitations and considerations Assumptions about others’ work. When gathering data about other organizations’ solutions, you must make assumptions as to the appropriateness or relevance of their experiences to the problem you’re facing. Reliance on published data and people’s memories. In order to get information about current and best practices, you must rely on published reports and the recollections of people involved in those projects. This may limit the kind and amount of information you are able to acquire. Reluctance to discuss failures. Organizations and individuals are more likely to share stories about their successes than their failures. But both kinds of stories can provide valuable information. Defining “best” is subjective and anecdotal. What is best for one organization may not be best for another. Defining best practices in any field can help direct discussions about alternatives, but be careful about assuming anything is “best” for everyone, everywhere. Be selective and rigorous in judging whether you should adopt a practice from another organization. For more information Center for Technology in Government (July 2000) Using the Internet to Find Current and Best Practices. Albany: Center for Technology in Government. Available online at www.ctg.albany.edu/publications/didyouknow/dyk_2000_jul/dyk_2000_jul.pdf. Cortada, J. (1997) Best practices in information technology: How corporations get the most value from exploiting their digital investments. Upper Saddle River, NJ: Prentice Hall. Eglene, O. (2001) Conducting Best & Current Practices Research: A Starter Kit. Albany: Center for Technology in Government. Available online at http://www.ctg.albany.edu/ publications/guides/conducting_best/conducting_best.pdf. Rocheleau, B. (2000) “Prescriptions for Public-Sector Information Management: A Review, Analysis, and Critique.” American Review of Public Administration 30, (4) 414-35. 108 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Technology awareness reviews Technology awareness activities help to identify what technologies make sense to use given a specific problem. These activities are used to educate people about the capabilities of the technology so they can begin to think creatively about transforming the way the agency operates. Becoming aware of the capabilities of specific technologies helps to inform analyses of alternative solutions and helps narrow investment choices to those that will work best for your organization. What are they? Educational activities focused on technology. These activities include reviewing the relevant literature in both trade and technical journals, visiting trade shows, hearing presentations by organizations with exemplary systems, visiting organizations that have installed similar systems, arranging vendor demonstrations, or developing and demonstrating one or more prototype versions of a proposed system. Conversations about technology options. Technology awareness sessions presented by experts or experienced users help you find out about different classes of technology and, more specifically, about certain products within classes of technology. What are they good for? Learning about new classes of technology. In order to use a technology effectively it is necessary to understand thoroughly the capabilities and limitations of that technology. Technology awareness activities are designed to accomplish that purpose. The amount or kind of educational activities that are needed in a project depends on the size of the gap between the staff’s current knowledge and optimal familiarity with the proposed technology. Some limitations and considerations This is only an introduction. Understanding and adapting to a new technology is often a slow and difficult process for a number of reasons; reengineering processes commonly requires cultural, organizational, and interorganizational changes. In addition, there is a difference between knowing about something and actually experiencing it. Some of the benefits and limitations of a technology can only be appreciated after years of experience by an organization. Keeping up is a challenge. Technology changes constantly making it difficult to stay ahead of the curve. Try to meet this continuous learning challenge by regularly reviewing publications or attending conferences. Your entire organization benefits if someone stays abreast of new developments and emerging tools. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 109 For more information DB2 magazine www.db2mag.com. Federal Computer Week’s Technology Reviews www.fcw.com/techtest.asp. Government Computer News’ Product pages www.gcn.com/prodcentral. Government Technology. Solutions for state and local government in the information age. Includes case studies, best practices, and a government technology library www.govtech.net. Association for Computing Machines info.acm.org. Comprehensive information on technology news, technology reviews and case studies www.zdnet.com. 110 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Benchmarking In benchmarking, you compare yourself to the best-known example of a process, product or service. This example provides a reference point against which to evaluate your own performance. A narrow benchmarking framework may be used to compare organizations that are similar in mission and basic technology. A wider framework, which can be achieved by looking outside your own domain, can also provide important lessons or improved methods that would be missed by looking only at organizations just like yours. Identifying and selecting the appropriate benchmarks is a critical part of the process. The news media, professional publications, and competitions are good ways to identify possible benchmark candidates. What is it? A way to compare yourself to the best. Organizations that develop effective innovations and approaches to a particular problem typically publicize it. Most professional organizations and many publications sponsor annual competitions for best practices and noteworthy innovations. There are also databases of benchmark and best practice information for the public sector. A way to build consensus, support, and partnerships. Selecting a benchmark also requires consensus and support within your organization. In addition, you may have to establish a partnership with the benchmark organization. A thorough analysis and understanding of business process. For benchmarking specific technology applications, you need a thorough analysis and clear understanding of the business process and/or product to be evaluated. Without it, the lessons or innovations revealed by using the benchmark may be missed or misapplied. What is it good for? Learning how to improve efficiency and performance. The central benefit of good benchmarking is learning how to improve efficiency and performance. Benchmarks achieve their superior performance by innovative, often highly creative ways and offer rich opportunities for learning and gaining new perspectives. These new ideas, perspectives, and techniques can be learned through benchmarking much more efficiently and quickly than by self-study alone, formal research, or evaluation projects. Taking advantage of other group’s investment. By using another organization as a benchmark, you’re benefiting from its considerable investments in research, testing, training, and experimentation. Use the knowledge you acquire to help avoid mistakes and achieve higher performance. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 111 Information sharing and collaboration. Benchmarking also involves information sharing and potential for collaboration. The process may start an ongoing exchange of performance ideas and innovations among organizations; a relationship that may result in a partnership that provides greater opportunities for performance improvements. Getting positive publicity and recognition for participants. Successful benchmarking efforts can also lead to public recognition for the participants. The potential for performance gains can be substantial, resulting in opportunities for increased public support and rewards. Some limitations and considerations Once-in-a-lifetime experience. The outstanding performance of the benchmark may be due to special circumstances or factors that you can’t replicate. Can you live up to this standard? The high standards and great success of the benchmark organization can raise unrealistic expectations among your project participants. Need solid support and good resources. Trying to replicate the success of the benchmark requires political support and consensus within your organization. In addition, you need adequate resources to respond appropriately to the challenges produced by using a benchmark for assessment. For more information Andersen, B. and P. Pettersen (1996) The Benchmarking Handbook. London: Chapman & Hall. Bogan, C. and M. English (1994) Benchmarking for Best Practices: Winning Through Innovative Adaptation. New York: McGraw Hill. Harrington, H. J. and J. S. Harrington (1996) High Performance Benchmarking: 20 Steps to Success. New York: McGraw Hill. Keehley, P., S. Medlin, S. MacBride, and L. Longmire (1997) Benchmarking for Best Practices in the Public Sector. San Francisco, CA: Jossey-Bass. Rocheleau, B. (2000) “Prescriptions for Public-Sector Information Management: A Review, Analysis, and Critique.” American Review of Public Administration 30 (4) 414-35. 112 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Environmental scanning Organizations use environmental scanning to monitor important events in their surrounding environment. It is a way to answer the question, “What’s happening in my environment that will affect my future?” Scanning involves identifying the issues and trends that have important implications for the future. Scanning includes analysis of the information about these issues and trends to assess their importance and determine their implications for planning and strategic decision making. What is it? A way to discover emerging trends of strategic importance. Scanning is different from ordinary information gathering in that it is concerned primarily with the future, emerging trends, and issues that have strategic importance for your organization. A method for gathering information from a variety of sources. It involves gathering information from publications, conferences, personal and organizational networks, experts and scholars, market research, and any other source that appears to be useful. Organizations may have formal, continuous processes for environmental scanning with a permanent unit of the organization responsible, or the effort may be episodic and organized in an ad hoc manner. Data analysis for planning purposes. Simply gathering the environmental data is insufficient. It is also necessary for you to interpret the data correctly and make it useful for planning and decision making. What is it good for? Taking advantage of opportunities. Environmental scanning can help capitalize on emerging opportunities. It can be an important part of strategic planning by helping you shape strategy to better fit emerging conditions. Anticipating developments to avoid costly mistakes. Scanning can also help avoid costly mistakes by helping planners and decision-makers anticipate changes in the environment. This is particularly important in any planning that involves information technology, since the capabilities and costs of IT are evolving at a rapid pace. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 113 Some limitations and considerations Level of resources required. It’s hard to judge the appropriate level of resources to devote to environmental scanning. Where environmental conditions are turbulent and full of potentially significant changes, large amounts of resources may be justified. Interpretation an inexact science. More importantly, the interpretation of trend information and forecasting is an inexact science at best. The further into the future a scan probes, the more careful you must be with the interpretation. For more information Abels, E. (2002) “Hot Topics: Environmental Scanning.” Bulletin of the American Society for Information Science and Technology 28 (3) 16-17. Cornell University’s Cooperative Extension has a list of resources pertaining to environmental scanning. www.cce.cornell.edu/admin/program/documents/scan.htm [ Retrieved May 27, 2003]. Choo, C. W. (2001) “Environmental scanning as information seeking and organizational learning.” Information Research, 7(1). Choo’s article provides a detailed academic view of environmental scanning. informationr.net/ir/7-1/paper112.html [Retrieved May 27, 2003]. Mafrica, L. (2003) “From Scan to Plan: How to Apply Environmental Scanning to Your Association’s Strategic Planning Process.” Association Management 55, 42-47. 114 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Models of solutions When the stakes are high and uncertainties are great, it pays to build a model of your idea and test it in any and every way that you can. By modeling a process, system, or program before it is designed and implemented, you can more clearly think through how it will impact overall organizational processes and performance. When the idea works in the modeling stage you can be more confident that it will succeed in real operation. That is why building models and testing them thoroughly before getting to the final design and implementation phase is an effective way to hold down development costs and minimize risks. What are they? Extensions of a formal model of a problem. Review the earlier section of this handbook on “Models of Problems,” because solution models share many features with problem models. Representations of operation. Like the problem model, solution models often represent processes, information flows, decision points, and relationships—but this time they are created to solve the problem. The solution model should show how the new process or system will function within the whole organizational context. Representations of organizational and customer-oriented effects. These representations are just as critical as the ones that show how the proposed IT system itself will function. What are they good for? Simulating full system operation. Models of solutions help you describe and simulate how a full system will operate within the context of organizational and human factors. Thinking big. These types of models help you see the implications of a limited prototype when it is expanded to full-scale operations. Managers are forced to think through technical, organizational, and policy issues in designing these models. Exploring costs and benefits. You can delve into the costs and benefits of proposed solutions by linking the model to financial data. Asking “what if” questions. A new or revised system or process will have various organizational and human factor effects. By asking “what if ” questions, you can anticipate issues and problems before they are encountered in a real world system implementation. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 115 Some limitations and considerations All depends on the data. Models of solutions are no better than the data and relationships upon which they are built. Your model must be built upon a foundation of solid data and analysis if you want it to accurately forecast the impacts of the new system. Expense. These models can be very complex, expensive and time-consuming to build. Models of solutions may require specialized expertise, which may be unavailable in your agency. For more information Wolstenholme, E. (1993) Evaluation of Management Information Systems. New York: Wiley. 116 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Modest, moderate, elaborate alternatives When building a business case for an IT project or innovation initiative, you will want to consider alternative solutions. Investigating modest, moderate, and elaborate alternatives during the analysis process allows you to identify a range of possible choices. You can then compare the costs and benefits of the alternatives and make more informed decisions. What are they? Levels of choices. Developing modest, moderate, or elaborate alternative solutions can help outline the range of choices available to you. The features, functions, and technology that go into each level depend on your goals and resources. Modest. This level involves a minimum investment in effort, time, and resources. Moderate. At this higher level, you may include additional features or options and a wider range of internal and external information sources. Elaborate. This ultimate level involves advanced features, technologies, or options for the most ambitious project you could undertake to solve your organization’s problem. What are they good for? Identifying features and functionality. Place possible features and functions at the appropriate level. Think about the modest, moderate, and elaborate answers to questions about customers, services, features, information sources, and resources. You will be better able to make decisions when the options are categorized. Characterizing benefits. Describe the major benefits you expect to result from the implementation of your project. Benefits typically fall into one of three categories: better, cheaper, or faster. Classify the potential benefits as modest, moderate, or elaborate. Assessing and measuring performance. Once you have outlined project features and benefits appropriate for each level, you need to figure out a way to measure performance. Devise specific, objective methods for measuring the outcomes and results of your project. After a project is operational, these performance goals can be used to evaluate how well the project performed with respect to its overall specific goals. Determining the basis for cost estimation. Determining the cost of your project is easier because of your modest, moderate, and elaborate descriptions. Each of these alternative suggestions cost categories and degree of needed investment that you can build on in the more detailed cost estimation phases. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 117 Some limitations and considerations Know what resources are likely to be available. Have a good handle on what resources will be available to implement the modest, moderate, and elaborate project options. Be as realistic as possible when devising these alternatives. “Elaborate” does not mean the sky is the limit, unless that is actually a realistic expectation for your situation. Keep your focus on outcomes and results. When talking about the potential benefits of your project, describe them in terms of outcomes and results instead of inputs and outputs. This helps you focus your attention on the service you’re providing rather than the delivery mechanisms. Devise concrete measures. You must describe your performance measures in explicit, objective terms. Be as specific and clear as possible to avoid future confusion about how project features and services will be measured and assessed. Make forecasts. While you may be uncomfortable doing it, you should try to forecast the impact of each approach. Make educated guesses about how the modest, moderate, and elaborate options will affect stakeholders, service delivery, and the business process. Forecasts will help you make better decisions about which option to implement. How to present modest, moderate, and elaborate alternatives 1. Gather a group of people with strong knowledge of the issues and goals of the project and work through this exercise together. 2. Make a table with four columns and as many rows as you have features, functions, or other bases for comparison. Label the columns “features & functionalities,” “modest,” “moderate,” and “elaborate.” 3. In each row, briefly describe the modest, moderate, and elaborate alternatives for one feature or function. For example, user support features might range from on-line help to a business hours help desk, or even 24/7 support staff. 4. Make a second table with four columns, labeled “benefits,” “modest,” “moderate,” and “elaborate.” This table contains three rows labeled “cheaper,” “faster,” and “better.” Briefly describe the cheaper, faster, and better benefits that are likely to accrue from the modest, moderate, and elaborate alternatives you have just specified. Some examples: 118 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Cheaper: Reduce duplication in areas such as data collection and program development generate revenue savings in non-personal services: telephone, printing, mailing savings in personal services Faster: Reduce the number of steps in a process staff members get access to information in a mroe timely manner citizens get access to services in a more timely manner Better: Improved responsiveness to citizen need through 24-hour access more satisfied clients because information is more accurate and consistent ability to reach more customers with existing services Be as specific as possible in defining expectations for system performance (e.g. “90 percent of telephone inquiries will be completed on the first call”). While this may prove difficult to do at first, quantifying system performance expectations will help to clarify project goals and objectives, and provide a basis for evaluation when the project is completed. 5. Compare the alternatives in terms of the features they would offer and benefits they would generate and keep this information handy to compare with cost estimates. For more information Pardo, T. A., S. S. Dawes and A. M. Cresswell (2000) Opening Gateways: A Practical Guide for Designing Electronic Records Access Programs. Center for Technology in Government. See the “Program Design Tool” www.ctg.albany.edu/publications/guides/ gateways?chapter=5. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 119 Prototyping Often an IT project involves a relatively new technology or combination of technologies with which you and your staff have little familiarity and even less expertise. This is especially true in state and local government agencies, which traditionally opt to stick with tried-and-true technologies that may be several generations older than the proposed technology. To apply the new technology successfully in a cost-effective manner, all project participants—from end users who are specifying system functionality to developers who deliver the ultimate implementation—need to be thoroughly familiar with the potential benefits and risks inherent in the technology. Prototypes help you understand the proposed technology fully in order to reengineer processes successfully and take maximum advantage of the new system. What is it? Proof of concept. We define a “prototype” as a system built for a proof of concept. It is not a full-scale system or even a pilot. A prototype essentially identifies, demonstrates, and evaluates the key management, policy, technology and costs implications of a desired system. A prototype is also built to identify the value proposition all participants. An education and communication tool. Prototypes help educate end users, managers, and system developers about potential applications of technology, and how it can help solve their problems. Prototypes are powerful tools used to bridge the gap between what project team members currently know about a new technology and what they will need to know to apply it successfully. They also help bridge the gap between developers and potential users because both groups can look at and discuss the prototype in concrete terms. Not a pilot system. The purpose of a prototype is to show prospective users how a system might work so that they may think creatively about its potential usefulness. By contrast, a pilot is used in a limited real-life setting. Pilots are much more costly than prototypes to build because they have to work well enough not to hinder the activity of people who have to get real work done. This requires an attention to quality control and performance that typically drives the cost of development up substantially. What is it good for? Educating projects participants. The primary value of a prototype is to educate the project participants. Often, end users and managers have the least awareness of a technology’s potential because they may not have been exposed to it through their day-today activities. Therefore, prototypes typically emphasize the user interface portion of the system. Prototype development may also address data preparation costs, maintenance requirements, technical support requirements, end-user training requirements, and infrastructure needs. 120 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Stimulating both imagination and realism. A prototype can push people to dream of potential innovative applications of the technology. At the same time, seeing the technology in the concrete leads to a more realistic assessment of costs and benefits. Seeing a mock-up of the application helps guide the analysis to factors that are relevant to the real work of an organization. Interviews, model-building, surveys, and experiments all become more accurate if the participants have personally experienced how a system might work. The prototype itself can be used to gather data about the likely impact on processes and operations. Setting the stage for implementation. Prototyping activities can be aimed at all levels of staff. If the system being prototyped is ultimately procured, training and other costs may be lower because of your organization’s experience with the prototype. Assessing risk. The reactions that people have to a prototype can help you assess the risks involved in the project. Risks may be inherent in any of the internal or external factors that could affect the success of the project. These may include such potential barriers as staff and client resistance to change, immaturity of a new technology, personnel limitations, technology failures, and expected changes in the technical, political, or management environment. Some limitations and considerations Won’t work like a real system. Because a prototype is only part of a system, it won’t work like a real system. It will have very limited functionality, few attractive presentation features, and limited or fake data. The limitations of the prototype may not be apparent to naive users, and their experience needs to be moderated by expert advice. Can be costly. Depending on the educational needs of the project and the technologies involved, developing a prototype may be a costly proposition. If custom development is necessary, you may need someone experienced in a particular rapid application development environment. Specialized hardware and software may be necessary to support even a small prototype. For more information Lewis, R. (2002) “The Value of a Test Drive.” Information World 24 (13) 48. Sommerville, I. (2000) Software Engineering, Sixth Edition. New York: Addison-Wesley. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 121 Tools for Phase 3: Evaluating and making smart choices Risk analysis Risk analysis covers a range of techniques and analysis tools used to assess the likelihood of failure or undesirable outcomes from decisions or policies. As one researcher put it, risk assessment “is the application of…knowledge of past mistakes in an attempt to prevent new mistakes in a new situation” (Wilson and Crouch, 1987). The methods rely primarily on mathematical modeling, statistics, uncertainty, and decision analysis. What is it? A way to identify threats that can derail success. As applied to business case development and decision making in IT projects, the most impor tant elements of risk analysis are identifying the threats to success and assessing the likelihood of those the threats materializing. A method for learning from past mistakes. A variety of modeling, statistical, and analysis tools can be used to examine past projects, determine where mistakes were made, and devise methods to avoid repeating them. What is it good for? Identifying threats and possibility of damage. Careful risk analysis is needed to provide two kinds of information. One is a clear and detailed identification of threats or possible mistakes that can damage an initiative. The other is an estimate of the likelihood of each kind of damage actually occurring. Outlining potential process problems. A number of important risks are associated with innovations in business processes. These include internal resistance to change or even subversion of objectives by unhappy participants. The costs and complexities of needed changes may be underestimated, leading to insufficient resource commitment. An inadequate or inaccurate model of the business process may be used, or inaccurate data about that process may lead to mistakes. Differences in the cultures of the organizations involved may produce conflicts that undermine success. Competition or lack of trust can inhibit communication and collaboration. And it may be impossible to generate the support from top leadership to sustain large-scale changes. Identifying potential political opposition. Political opposition can lead to problems and barriers. Risk analysis should involve the positioning analysis described above, with special attention to estimating the strength of likely opposition from influential players. Risks can include failure to manage expectations about success or immediate results, as well as missing the possible influences of other large initiatives on the political agendas of supporters and champions. 122 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Defining IT risks. A number of risks are associated with the use of information technology, including rapid obsolescence and emergence of alternative technologies after investments have been made. Avoid the tendency to over-promise the benefits of technology or underestimate the effor t of implementation—both lead to disillusionment and loss of suppor t. Describing environmental and organizational risks. Planning and risk analysis should take into account the kinds of policy shifts, as well as the sources of suppor t and opposition to such policy changes, that constitute the greatest threat to your initiative. Demands and costs of human resources can also shift, due to labor market forces, and put a project in jeopardy. Careful environmental scanning can help mitigate or anticipate these possible threats. Some limitations and considerations Technical problems. The technical problems of statistical risk analysis can be substantial, since they depend on models of threats and probabilities. For complex projects, such models may be unavailable or even impossible to construct. In addition, statistical risk analysis often depends on historical information that may be unavailable for new projects, technologies, or collaborations. This problem may be mitigated in some circumstances by tools, such as system dynamics models or other simulations that allow for exploration of various scenarios or alternatives. Long-term perspectives, short-term adaptability. This basic dilemma in mitigating and managing risk is especially acute in technology projects. IT plans and system designs based on current knowledge and technologies are unavoidably at risk. Systems built with smaller components or modules can provide for more flexible response to rapid changes, but their success depends in large par t on accurate anticipation of technology trends, which is demanding and error-prone at best. For more information Chapman, C. and S. Ward (1996) Project Risk Management: Processes, Techniques and Insights. Chichester: John Wiley & Sons. Bedford, T. and R. Cooke (2001) Probabilistic Risk Analysis: Foundations and Methods. Cambridge: Cambridge University Press. Kammen, D. and D. Hassenzahl (1999) Should We Risk It? Exploring Environmental, Health, and Technological Problem Solving. Princeton, NJ: Princeton University Press. Kemshall, H. and J. Pritchard (1996) Good Practice in Risk Assessment and Risk Management. Bristol, PA: Jessica Kingsley. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 123 Kemshall, H. and J. Pritchard (1997) Good Practice in Risk Assessment and Risk Management 2: Protection, Rights and Responsibility. Bristol, PA: Jessica Kingsley. Stern, P., H. Fineberg, and the National Research Council (1999) Understanding Risk: Informing Decisions in a Democratic Society. Washington, DC: National Academy Press. Wilson R., and E.A. Crouch (1987) “Risk Assessment and Comparisons: An Introduction.” Science, 236-267. 124 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Scenario building and forecasting Scenario building and forecasting tools help design the future of a project. They are a way of answering the question, “Where should we be going and how will we get there?” These strategic planning tools, as distinct from the others, are usually about the big decisions organizations face about their future. They are strategic in that they involve decisions and actions with major consequences that extend over long periods of time, and attend to the shor t and long term environmental factors that may affect events. What are they? Scenario building. This is a process of designing a hypothetical situation in a way that helps you predict the consequences of decisions and actions. For example, Massachusetts has proposed legislation to require all state agencies to consult a database of outstanding arrest warrants when a citizen is seeking a service or benefit. Officials could examine the possible consequences of such a new policy by creating a scenario. This scenario would assume reasonable values for the number of times the policy would generate arrests of various types, and compute the increased demand on jails or law enforcement officers. Forecasting. This tool is also used to predict future events, but it uses calculations based on historical data. Forecasting typically uses data that have been collected on some events over time and uses them to project trends into the future. Populations, crime statistics, and budgets often have ample historical data for forecasting. The mathematical models used in forecasting may take into account the forces that influence trends to adjust the predictions. What are they good for? Simplifying reality for testing. These kinds of models provide a simplified version of reality against which to test ideas and explore consequences. They are most useful in the kind of complex situations characteristic of justice systems and their information flows. Exploring possible actions. A model can be a very powerful tool to explore possible courses of action or decisions. Consequences can be explored in hypothetical rather than real situations, so the costs of errors or bad decisions are limited. Developing a common understanding. The development of models also provides a way of creating a shared understanding of complex systems among those that work in them. This shared understanding can be of great value as an aid in collaboration. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 125 Some limitations and considerations Require advanced technical skills. The kinds of models described here require relatively high levels of technical skill for their construction and interpretation. If these skills are not available in your organization, it will require the intervention of external experts, usually at considerable cost. Quality depends on data. In addition, the quality of the analysis resulting from the model is no better than the model itself and the data on which it is based. Careful testing and validation are necessary to avoid conclusions or actions based on a flawed model. Presentation, communication. Models of this sor t often pose problems of presentation and communication as well. They frequently involve complex mathematical operations or graphic images that are hard to understand and explain to non-technical audiences. A well-designed interpretation and presentation must accompany the modeling work for non-technical audiences and policy makers. For more information Ward, J., P. Griffiths, and P. Whitmore (1990) Strategic Planning for Information Systems. New York: John Wiley & Sons. 126 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Cost estimation What is it? A way to assess initial and future total costs. Working out the costs of a project requires careful attention to what cost information is relevant, what’s available, and how it can be interpreted and used. Although it can seem like a straightforward task, a comprehensive cost analysis can be quite complex and demanding. A cost is something of value that is given up or exchanged for a par ticular reason. It might be as obvious as the financial outlay for some new equipment or as subtle as the extra time it takes a supervisor to explain new procedures to a staff member. And costs continue beyond the initial star t up phase of a new system or program and must be estimated for recurring annual expenses. Detailed breakdown of costs. Cost estimation helps specify all the different categories of cost associated with both the start up and the ongoing operation of a system or service. The list of categories can be quite long if the project is large and complex. The cost estimate should include the costs of developing and maintaining the system, preparing the agency computing infrastructure to support it, and training staff and other end users to use it. It should also include the cost of all of the staff time involved in the planning, decision-making, and training for the project. Both one-time costs and ongoing costs should be included. Determining the costs of alternative approaches. The goal of cost estimation is to approximate the costs of alternative plans as thoroughly and explicitly as possible so they can be compared. What is it good for? Avoiding underestimation. People often underestimate the costs of IT initiatives. Many times there are so many different kinds of expenses that some are overlooked. Frequently, the true cost of human resources is not made explicit, especially if most or all of the people to be involved are already on the payroll. The ongoing annual costs of hardware and software maintenance, upgrades, and training are often missed. By listing all the categories of cost that can be reasonably attributed the project, you are in a better position to later judge which alternatives give the best pay back. Project evaluation. After a project is operational, initial cost categories and estimates can be used to evaluate how well the project performed with respect to its budget. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 127 Some limitations and considerations Failure to consider indirect and opportunity costs. Although these kinds of costs are often more difficult to identify and many not lend themselves to quantification, they should at least be identified and described so they can be factored into later choices about the cost-benefit of various alternatives. Direct costs of a new project are usually the easiest to identify and analyze, since they typically are the financial costs that are part of ordinary budget making and planning. Indirect costs are usually based on estimates or pro-rating of shared resources, such as portion of infrastructure maintenance and depreciation or overall administration expense. These costs are usually more difficult to identify and analyze, since the estimates they require are often based on uncertain assumptions and limited knowledge of actual impact. Opportunity costs are the losses or costs to the organization that result from implementing the new system rather than the alternative uses of those resources. These costs are real and can be important, but are very difficult to measure and document. Participants in the development and implementation of a new system are often very sensitive to opportunity costs, since these affect their day-to-day work. But these costs are not part of any formal accounting system and so may be ignored by planners and budget makers, often to the detriment of implementation. At the very least, you should attempt to identify the possible opportunity costs involved in your project and discuss ways to ameliorate negative impacts. Difficult to estimate future costs. The cost of out years cannot be estimated with the precision or confidence that pertains to the first year. Some informed guessing is inevitable, especially in an environment of significant technological change. How to conduct cost estimation 1. Begin your cost estimation by making a comprehensive list of the cost components of the project. Typical categories include project management, equipment, contractual services, facility maintenance, travel, and so on, but these usually need to be broken down into sub-components before they can be estimated with any accuracy. In most categories, you can also partition costs in a second way: costs for human resources and costs for other purposes. You are likely to find that the human resources costs dwarf other costs. In making your estimates, you should account for all the staff time necessary to plan, launch, and operate the service. 128 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 2. Next, make a best estimate of the costs for each expense category across all alternatives—from modest to moderate to elaborate for both start up costs and ongoing operation. A worksheet, such as the one illustrated below, can be very useful. In each category, the worksheet allows for “one-time and “annual” costs. One-time costs are incurred during development and implementation only, while annual costs recur for as long as the service continues to be delivered. For more information Pardo, T. A., S. S. Dawes and A. M. Cresswell (2000) Opening Gateways: A Practical Guide for Designing Electronic Records Access Programs. Center for Technology in Government. See the “Cost Estimation Tool” www.ctg.albany.edu/publications/guides/ gateways?chapter=6. Boehm, B. W. (1988) “A Spiral Model of Software Development and Enhancement.” Computer, (May), 61-72. Sample Cost Estimation Worksheet MODERATE MODEST First First Year Year Annual Annual ELABORATE First Year Annual Project leadership Project management Organizational readiness training Equipment for users Program development and maintenance Hardware acquisition Hardware maintenance ..... ..... ..... HUMAN RESOURCE SUBTOTAL INFRASTRUCTURE AND OTHER SUBTOTAL TOTAL CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 129 Cost-benefit and cost-performance analyses Cost-performance and cost-benefit analyses estimations are used to determine the level of investment that is appropriate for your project. They are ways of answering the questions, “Is this worth doing?” and “How will we know whether it was worth it when we’re done?” These tools are methods for assessing the value of a project by comparing its costs to measures of its performance, or more generally to the value of benefits it produces. The analysis requires accurate cost data, as well as measures of performance in appropriate units and overall benefits. Cost-benefit analysis tends to emphasize quantitative evaluation, while the broader concept of cost-performance analysis more readily accounts for qualitative results. Cost and performance data can be obtained from operational records, direct observation, surveys, or group meetings at which those who perform the operations report and discuss costs and performance measures. Both one-time costs and ongoing costs should be included. If you have used the tools above to generate modest, moderate and elaborate alternatives and to estimate costs, then you are ready to make comparisons on a costperformance or cost-benefit basis. What are they? A way to understand the value proposition. Many books and articles have been written about cost-benefit analysis and it can be extremely intensive and elaborate in implementation. At its heart, however, this kind of analysis helps decision makers understand the value that can be obtained (in terms of goals achieved) for the investment to be made. A way to compare competing alternatives. Cost-benefit or cost-performance analyses that compare alternative approaches on the same criteria help decision makers choose the best alternative given desired goals and resources available. What are they good for? Making informed choices. When decision makers look at a business case, information from these kinds of analyses are important in choosing whether and how to go ahead with a project. They lay out the value proposition for various alternatives by showing how they can achieve program goals and what it will cost to reach them. Decision makers can see the strengths and weaknesses of different approaches and make the trade-offs that lead to a final decision about whether and how to proceed. Identifying new alternatives. By explicitly comparing the costs and benefits of the various approaches against a given set of criteria, you may be able to spot additional alternatives that combine different elements of the original set. For example, you may see a way to take advantage of elements of one approach early on and then move to a second approach later. 130 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Evaluating a project. The results of your cost-benefit or cost-performance analyses form an important basis for evaluation. After a project is implemented, these cost and performance measures can be used to evaluate whether the initiative actually achieved its goals within its expected budget. Some limitations and considerations Complex environment. A comprehensive analysis of your project’s impact may be difficult to prepare because of the complex environment in which public sector programs reside, and the many factors that may affect the intended outcomes of the project. It is often difficult or impossible to attribute outcomes and impacts directly to one specific initiative to the exclusion of all other efforts that are seeking similar goals. Can over-emphasize quantitative data. Classic cost-benefit analysis seeks to place a quantitative value on every aspect of cost and benefit. For example, if a project saves lives, classic cost benefit would put a dollar figure on every life saved. In the public sector, many efforts are aimed at qualitative goals that are difficult or impossible to quantify. These goals may improve convenience for citizens, improve quality of life, or act in concert with other programs so the direct effects are impossible to discern. Nevertheless, these benefits need to be identified, described and taken into account. For more information Bloniarz, P., and K. Larsen (2000) “A Cost and Performance Model for Supporting Web Service Investments.” Communications of the Association for Computing Machinery, 43 ( 2) 109-116. Boardman, A. E., D. H. Greenberg, A. Vining, and D. L. Weimer, eds. (2000) Cost Benefit Analysis: Concepts and Practice. 2nd Ed. Upper Saddle River, NJ: Prentice Hall. Boehm, B. W. (1988) “A Spiral Model of Software Development and Enhancement.” Computer, (May), 61-72. Center for Technology in Government. (2003) Combining cost and performance assessments for decision support. Available at www.ctg.albany.edu/publications/guides/ costperfmodel?chapter=8&PrintVersion=2 [Retrieved August 10, 2003]. Center for Technology in Government. “Return on Investment in IT: A Guide for Government Decision Makers.” www.ctg.albany.edu/publications [Retrieved August 10, 2003]. Levin, H. and P. McEwan, eds. (2000) Cost Effectiveness Analysis: Methods and Applications. Thousand Oaks, CA: Sage Publications. Nas, T. F. (1996) Cost Benefit Analysis: Theory and Application. Thousand Oaks, CA: Sage Publications. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 131 Multi-attribute utility (MAU) models Multi-attribute utility (MAU) models are mathematical tools for evaluating and comparing alternatives to assist in choosing among them. They are designed to answer the question, “Given the factors we care about, what’s the best choice?” MAU models are based on the assumption that the desirability of a particular alternative depends on how well its attributes measure up against key evaluation factors. For example, if you are shopping for a new car, you will prefer one over another based on how well each one scores on the factors you think are important, such as price, reliability, safety ratings, fuel economy, and style. These models can be applied in all kinds of decision situations and are often used in the technical and programmatic parts of procurement evaluations. What are they? Methods to evaluate alternatives. MAU models offer a structured way to weight, evaluate, and compare possible alternatives. They offer a quantifiable method for choosing among options. Ways to conduct sensitivity analysis. A MAU model can also be used to explore the consequences of changing the attributes, their weights, or the scores they receive. Since the model usually is embodied in a simple spreadsheet, it is possible to make any number of changes and review the results. For example, if it appears that some attribute is too important in determining the results, the weights can be adjusted to produce different overall scores and to see if those differences really matter to the final decision. What are they good for? Clarifying the nature and importance of evaluation criteria. One of the most useful benefits of using a MAU model is that it makes clear to all involved the basis on which the alternatives are being evaluated. This is particularly important in group decisionmaking situations in which many different points of view and decision alternatives have to be reviewed and taken into account. Managing complex comparisons. Some choices need to reflect evaluation of many criteria. A MAU model helps manage that complexity by converting the evaluation to a numerical score while still presenting the logic behind the score. 132 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Some limitations and considerations Requires group consensus. MAU models are typically used in a group situation. To be effective, the group must be able to come to consensus on the attributes in the model and on the weights to be used to indicate their relative importance. It may be very difficult and time consuming, or even impossible to achieve consensus on very controversial decisions. Conflicts often arise. The level of detail and specification necessary in the discussion of attributes, their overall importance, and the extent to which each alternative meets them can result in considerable conflict and contention, rather than the move toward consensus. However, conflict often indicates that some important criterion has not been made explicit and this possibility should be explored through candid group discussion. For more information Edwards, W. (1982) Multiattribute Evaluation. Beverly Hills, CA: Sage Publications. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 133 Prioritizing methods Prioritizing methods establish the relative value of choices or alternatives. They answer the question, “How do these items rank in importance?” You can prioritize answers to this question in a rank-ordered list of the choices to show what should be done first, what requires the greatest attention, and what needs the most resources. What are they? Objective-based priorities. Actions or choices can be prioritized in terms of how they affect the achievement of an objective or fit into a structured process. These can be called objective priorities. Ways to identify the critical path. Program Evaluation and Review Technique (PERT) is an example of an objective priority-setting process. A PERT analysis shows which activities in a structured process are part of the “critical path.” This is the sequence of events that determines the overall pace of your project. Activities on the critical path usually receive priority attention because delays will affect overall progress. Triage activities. Triage is another objective-based priority-setting process in which choices are made according to whether they will affect the overall achievement of objectives. In medical triage, for example, cases that are not in immediate need of attention receive lower priority, as do cases where the likely success of action is small. Those that combine urgency with potential for success get top priority. Criteria priorities. Priorities can also be based on a set of criteria. Cost-benefit or cost-performance analyses are examples of this sort of priority setting. Whatever choices yield the greatest value on the criterion measure get highest priority. Results of a MAU model would also be a form of criteria-based priority setting, but one that may be linked to a group decision process. Group voting techniques. Priorities chosen through voting are based on the subjective preferences of the voters. These include one-person one-vote methods, where vote totals can determine priorities. Multi-voting methods are also used in which each voter gets some fixed number of votes to distribute among the choices. What are they good for? Investment decisions. These methods attempt to identify those factors that will have the greatest influence on progress or success and therefore point to the people, organizations, tools, and activities that should be highest on the list for investment of resources. 134 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Group decision-making. A single person can use these methods, but often they are used in a group setting. In these cases, prioritizing methods are used in situations where a variety of perspectives or preferences must be taken into account. In these cases, the prioritizing is an open process which reveals differences that can then be explored in an effort to achieve consensus. Some limitations and considerations Tough choices among desirable things. Priorities often involve choices among competing “goods.” The process of identifying and setting priorities will almost certainly involve conflict and controversy. Some planning and preparation are necessary to keep the work on track. More tough choices. Setting priorities does not end the tough decision process. Even though you know which choices are most important, you still have to figure out exactly how to allocate resources and work assignments. Existing preferences, policies. Priorities set by objectives or voting methods may become irrelevant if they fail to align with an organization’s preferences and policies. Therefore, the effort invested in priority setting activities may not always determine outcomes. Active consultation with top executives or policy leaders should be a part of the policy deliberations to avoid conflicts and wasted efforts. For more information Modell, M. (1996) A Professional’s Guide to Systems Analysis. 2nd Edition. New York: McGraw Hill. www.projectreview.net/prio.asp gives a quick guide on how to do a group prioritization matrix to narrow a team’s focus [Retrieved July 7, 2003]. For a more detailed explanation of the Program Evaluation and Review Technique (PERT), Critical Path Method (CPM) and Gantt charts, see studentweb.tulane.edu/~mtruill/devpert.html [Retrieved July 7, 2003]. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 135 SWOT analysis (Strengths, Weaknesses, Opportunities, Threats) SWOT analysis is a simple framework to help answer the question, “What are the prospects for success?” The approach recognizes that any project should be examined for both positive and negative influences from internal and external perspectives. A SWOT framework prompts you to look in detail at both sides of the coin. That is, the strengths and weaknesses of your project are only meaningful in terms of the opportunities and threats in its environment. What is it? A way to identify strengths and weaknesses. To obtain this knowledge about yourself (strengths and weaknesses) and others (opportunities and threats) requires identifying the SWOT elements and analyzing them in depth. This is typically done in interactive groups where people can discuss, assess, and elaborate on what is identified in each category. A method for maximizing the positive and minimizing the negative. The analysis and deliberation are designed to identify ways to take advantage of strengths and exploit opportunities, as well as minimize the impacts of weaknesses and protect against threats. What is it good for? Testing feasibility. SWOT analysis is best suited to a stage in business case development when the nature of the objective is reasonably well known. It is a useful way to test the feasibility of project objectives and methods. Determining how to move forward. This type of analysis helps you start to identify what is needed to move your project to the next phase armed with a good understanding of both internal capabilities and environmental factors and their interaction. Expressing different viewpoints. The interactive process can provide people with an opportunity to express their views about the project and discuss their implications. Advocates usually emphasize strengths and opportunities. Opponents tend to emphasize weaknesses and threats. Neither view alone creates the balanced or comprehensive analysis needed to make the right choices. The SWOT framework provides legitimate exposure for both perspectives and an opportunity to reconcile opposing points of view. A basis for further planning. The results of a detailed SWOT analysis also provide valuable material for continued planning and support-generating activities. The strengths can be presented and emphasized to potential supporters. Discussion of weaknesses and threats provides useful information for strengthening the project or business plan where possible. Conversely, problems and weaknesses that cannot be eliminated become better understood. As a result mitigation plans and contingency planning can take place. 136 CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES Some limitations and considerations Information quantity and quality. The key to effective SWOT analysis is the sufficiency and quality of available information. Participants’ understanding of your project, its resources, and weaknesses must be deep and detailed. Similarly, analysis of the environment in terms of opportunities and threats must be based on extensive experience, thorough scanning, and collection of data from a wide variety of sources. The future cannot be predicted precisely. Complete information about the environment is never available and projections about future events and trends are always subject to error. So the SWOT analysis must include consideration of the reliability of the information used and the conclusions reached. Considerable technical resources may also be needed in some circumstances to provide forecasts and projections for assessing the opportunities and threats in the environment. Assumes shared goals. The process of SWOT analysis is based on the assumption that the participants all share the same goals. This, of course, is not always true. Because the process depends information provided by participants, as well as their collaboration, the analysis may be vulnerable to disruptive or subversive behavior. For more information Collett, S. (1999) “SWOT Analysis”. Computer World 33(29) p. 58. Stewart, R., S. Mohamed, and R. Daet (2002) “Strategic implementation of IT/IS projects in construction: a case study.” Automation in Construction 11, 681-94. Management Sciences for Health and UNICEF “SWOT Analysis.” erc.msh.org/quality/ ittools/itswot.cfm [Retrieved July 7, 2003]. About.com’s site addressing how to perform a SWOT Analysis. businessmajors. about.com/library/weekly/aa123002a.htm [Retrieved July 7, 2003]. CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES 137 Center for Technology in Government 187 Wolf Road Albany, NY 12205 Phone: (518) 442-3892 Fax: (518) 442-3886 E-mail: info@ctg.albany.edu www.ctg.albany.edu CENTER FOR TECHNOLOGY IN GOVERNMENT: MAKING SMART IT CHOICES