The shift from the modern industrial era into the new, post-modern Information Age presents contemporary society with a rather significant paradox. On one hand, there is fairly widespread agreement that the governmental apparatus established to implement public policies – the bureaucracy – is not very efficient or effective. On the other, there is equally widespread belief that bureaucracy is necessary in order to successfully implement those policies. We are stuck in something of a love/hate, “can’t live with it, can’t live without it” dilemma when it comes to the presence of the large bureaucratic systems, at all levels of government, that are critical to the actual delivery of services that constitute the ultimate operationalization of legislative dictates.
This paradox is not new, although the dilemma it presents has become more pressing as the societal transition into a new era proceeds. Significant backlash to the dysfunctional features of bureaucracy emerged as early as the 1950s, when proponents of a more humanistic approach to organizational design began articulating how bureaucratic structures and processes could be revised to take into account the higher-order needs (i.e., self-esteem and self-actualization, in Maslow’s hierarchy) of the people working in bureaucratic organizations. To a considerable extent, the slow but steady evolution of this organizational form since that time has reflected the gradual integration of some of those ideas into our collective understanding about the best ways to manage organizations. These changes have been further stimulated by the dynamics of globalization, the diffusion of information/communication technology, and the differences among succeeding generations of workers. Taken together, the reforms over the last half-century can be seen as leading to a transformation in the bureaucratic organizational form itself, as it evolves into a new form more appropriate for the demands of a new era.
To better understand the nature of this evolution, it will be instructive to consider briefly the genesis of the bureaucratic model and its institutionalization in modern society. German sociologist Max Weber is credited with formalizing the principles of bureaucratic organizations in the late 1800s, as part of his broader work analyzing the effects of industrialization on society. New technologies were increasing the scale and scope of organizational activities, and thus new approaches to organizational design and management were needed in order to be able to maintain effective control over operations. Weber’s principles were intended to provide the basis for organizations that were more stable and predictable, more rational and efficient, than the small, traditional businesses prominent in older agrarian societies. Contemporaries of Weber in Europe and the U.S. contributed additional ideas to the body of thought now referred to as classical administrative theory, which provides the normative foundation for formal, hierarchical organizations incorporating bureaucratic properties. The “scientific management” movement popularized by Frederick Taylor utilized time-and-motion studies to determine in precise terms the most efficient ways for workers to do their work, leading to significant improvements in productivity and profitability.
A key focus of the Progressive movement that began at the end of the 19th century was the elimination of the governmental corruption associated with the powerful political machines of the day, and a primary strategy used to accomplish this objective was the reform of government organizations to be more scientific, rational, and efficient. These efforts to modernize government systems reflected what Woodrow Wilson characterized as the “politics-administration dichotomy,” i.e., the notion that policies should be decided through democratic political processes but then implemented by neutral public administrators with relevant expertise. While politicians are accountable to the voters who elect them, public administrators are accountable to the politicians who authorize their activities. A single chain of command from the top to the bottom of the organizational hierarchy is meant to insure democratic control over the government bureaucracy. The Progressive reformers initially worked to modernize municipal government, but with strong support from the middle class, the reforms spread through the state and federal governments as well.
Along with government, the Progressive movement had modernizing effects on many key sectors of society in the early 1900s, including education, science, medicine, finance, and business. Henry Ford’s development of mass production techniques during this period further stimulated industrial productivity, and two world wars provided ample opportunity and motivation for American industrialists to incorporate modern management systems and production techniques into their operations. With the end of WWII, the G.I. bill enabled millions of returning servicemen and women to get a free college education, after which they secured employment in innumerable private and public organizations throughout the country, comprising the best-educated workforce in the history of the world. It wasn’t long before American government, business, and education systems were widely recognized as the best in the world. This success served to verify the value of the rational, scientific orientation underlying the whole modernization movement.
Despite the many benefits resulting from the pervasive bureaucratization of organizational systems in the first half of the 20th century, criticism of these systems started to grow in the 1950s as their dysfunctional consequences became more apparent. For example, sociologists such as Robert Merton, Philip Selznick, and Alvin Gouldner pointed to various unintended outcomes of bureaucratic structures that undermine organizational performance by, for example, producing behavioral rigidity and a tendency to perform at a minimal acceptable level, focusing employees’ attention on rules and procedures rather than goals and results, generating conflict among organizational subunits, and resulting in close supervision perceived as “micromanagement.” Likewise, humanistic psychologists and organizational theorists such as Abraham Maslow, Douglas McGregor, Rensis Likert, and Chris Argyris critiqued the bureaucratic model for its negative effects on the human beings who work in them. The essence of their argument was that rational structures and processes serve to de-motivate and alienate workers, and preclude them from opportunities for growth and development.
Two primary recommendations were offered by those who sought to reform bureaucratic organizations to be more compatible with the needs and values of their human participants. The first was the idea of participative management, reflecting the basic premise that managers should take into account the information and perspectives of their employees when making organizational decisions rather than simply acting in an autocratic manner. Built into bureaucracy is the notion that managers should do the thinking and planning while workers should just do as they are told. Participative management helps to overcome this separation of planning and doing, and is more compatible with democratic principles of giving people input into decisions that affect them. Advocates suggest that the benefits of involving employees in relevant decisions include making better-informed decisions, gaining greater acceptance of those decisions and thus willingness to implement them, and increasing employee motivation and satisfaction. The Organization Development (OD) movement of the 1960s focused heavily on the goal of helping organizations become more participative, with a variety of “interventions” developed to bring about the necessary changes in organizational structures and management processes.
The second main reform strategy was the redesign of jobs and the work processes used to produce organizational outputs. The principles of bureaucracy and mass production logically result in narrowly defined jobs at the bottom of the organizational hierarchy that require very few skills or mental capabilities. Furthermore, these jobs are organized into work processes over which employees have very little control, either in their design or on-going management. Recognizing the challenges of motivating workers to perform these jobs well, as well as the negative performance effects that result from worker alienation, OD practitioners also helped organizations implement changes in their socio-technical systems that gave employees more variety and autonomy in their work as well as a better understanding of how their work fit into the bigger picture. These efforts were compatible with those oriented towards increasing participative management, and the two strategies came together in the concept of autonomous work groups. Adopted by just a small number of companies, these are essentially self-managing groups that are responsible for producing an entire product such as a car. Rather than working on a mass production assembly line with each worker doing the same task over and over all day long, a small group of employees would have responsibility for working together, however they see fit, to produce an entire car.
Despite the concerted efforts of the OD reformers, a common experience was that it was very difficult to achieve any significant change in the core structures and processes of a bureaucratic organization. This can be explained in part by people’s natural resistance to change, rooted in the fear that things will get worse rather than better, or simply in the fear of the unknown. But it also reflects the fact that bureaucratic organizations are not built to change but rather to maintain stability and predictability. In their classic 1961 book The Management of Innovation, Tom Burns and George Stalker pointed out that the stability derived from highly bureaucratic or “mechanistic” organizational characteristics is useful as long as the environment in which the organization functions is also stable. However, when the environment is more unstable, characterized by greater rates of technological or market change, organizations need to be more “organic” so as to be able to adapt to changing circumstances. Organic organizations are those with more decentralized decision-making, fewer rules and regulations, broader job definitions, and other features that make it easier for employees – and thus the organization as a whole – to respond to new conditions or unexpected events.
The post-war conditions of the 1950s provided a relatively stable environment for American industry, as an expanding middle class both at home and abroad made it easy for businesses to grow simply by doing more of the same. The mechanistic, bureaucratic organizations that emerged through the modernization period were quite successful in this context. However, the social upheaval of the 1960s started generating greater environmental turbulence, as the civil rights, feminist, environmental, and anti-war movements all challenged traditional mindsets and institutions that were getting in the way of the development of a peaceful and just society. The 1970s brought even more uncertainty and unpredictability, with the Watergate scandal undermining Americans’ faith in government and the OPEC oil embargo creating economic turmoil. A direct consequence of the sudden steep increase in the price of gasoline caused by the embargo was that many Americans stopped buying big American automobiles with low gas mileage and started buying smaller Japanese cars with more miles per gallon. This shift, along with growing global competition more generally, led to a condition of “stagflation” in which a high rate of inflation was coupled with a stagnant economy.
By 1980, American businesses were suffering, and starting to recognize the need for significant organizational change to adapt to the emerging conditions of a more globalized economy. But business executives then ran headlong into the dilemma that the large bureaucratic organizations they had created and maintained were not very amenable to the kind of transformation needed to adapt to these new economic conditions. With Reagan’s election that fall amid a resurgence of conservative ideology favoring market-based approaches and solutions, there was considerable sentiment that government had grown too big and that large public agencies had become too bureaucratic and dysfunctional. The concerns were that taxpayers’ money was being wasted as a result of too much red tape and inefficiency, too many staff doing too little work, too many resources squandered on ineffective programs, and too little innovation or adaptation to changing circumstances. Two key objectives of the Reagan administration were to downsize government and to deregulate business, based on the premise that these strategies would help the U.S. out of its economic malaise.
In both the private and public sectors, then, broad forces were exerting pressure on large organizations to de-bureaucratize so as to become more viable in the more turbulent circumstances of a globalizing world. In this context, some organizational theorists and management consultants introduced the concept of organizational culture, pointing out that the bureaucratic model of organization is based on underlying values and beliefs that must also change if bureaucratic organizations are to undergo meaningful transformation. This focus on the values embedded in organizational structures and processes harkened back to the humanist critique of the previous generation, and put the spotlight on the assumptions embedded in bureaucracies that undermine individual and organizational performance. A key lesson from studies of organizational culture was that an organization is likely to perform better if there is good alignment between its core values and those of its employees. Moreover, as Jim Collins and Jerry Porras demonstrated in Built to Last, organizations that are guided by a vision or mission that employees find attractive or compelling also have an advantage in terms of inspiring high-quality performance and bottom-line benefits. By the end of the 1980s, this cultural perspective had significantly influenced management thought, such that organizations and their managers were much more ready and willing to implement significant organizational changes than they had been a decade earlier.
This willingness came faster to private firms, as the competitive pressures of the marketplace required them to adapt or risk losing business. A number of techniques were used in an effort to improve performance, including quality circles and total quality management, reengineering and benchmarking. Taken together, these efforts to modify their structures and processes so as to enhance their adaptive capacity reflected a significant shift towards a more organic organizational form. In particular, organizations started incorporating three key features that deviate from the bureaucratic template. First, to overcome the autocratic tendencies of bureaucratic hierarchy, and building on previous practices used by OD practitioners, they implemented methods for increasing employee participation in decision-making processes; USC professor Ed Lawler provides an integrative perspective on these approaches in his book The Ultimate Advantage. Second, to overcome the coordination problems stemming from the bureaucratic division of labor, and again drawing on knowledge derived from previous efforts to redesign work, organizations began adopting more team-based structures; USC researchers Susan Mohrman, Susan Cohen, and Allan Mohrman discussed this trend in their book Designing Team-Based Organizations. Third, to overcome the inertia associated with bureaucratic standardization and routinization, they incorporated various practices intended to enhance organizational learning and innovation; Peter Senge’s book The Fifth Discipline added considerable impetus to this movement.
As these various reform strategies were being implemented more widely in the 1990s, a number of books were published that aimed to describe the new forms of organization that were emerging as a result of these transformative dynamics. According to the labels these authors used to describe these emergent forms, the new post-bureaucratic model of organization will be: centerless, boundaryless, and virtual; flexible, lateral, and connected; intelligent, knowledge-creating, and inventive; self-designing, self-managing, and self-renewing; living, adaptive, and evolving; new logic, democratic, and green; postmodern, quantum, and chaordic. Like the blind men and the elephant, each author offered his or her take on the organizational features required in order to be effective in the context of the new economy. Yet their collective insights describe a new organizational form that is radically different than the Weberian bureaucracy.
With many businesses engaged in the task of trying to transform their operations, the Clinton administration embarked on an effort to implement similar reforms in the federal government. Reflecting the kinds of ideas discussed by David Osborne and Ted Gaebler in Reinventing Government, Vice-President Gore led the National Performance Review that aimed to make federal government organizations more performance-based and customer-oriented by decentralizing authority, flattening structures, increasing employee involvement, and changing incentive systems. The “reinventing government” movement stimulated reform efforts in a number of states as well, and even some municipal governments tried to implement similar changes. Research evaluating the effects of the reinventing movement suggests that the reforms implemented tended to be rather modest and limited in scope. Undoubtedly, this is because public bureaucracies are even more mechanistic than their corporate counterparts, and thus more resistant to change. Furthermore, in the absence of any competitive pressure or real risk of failure, the felt urgency to change is lower in the public than the private sector. Despite these constraints, the popularity of the reinventing government agenda and the many serious efforts to implement corresponding organizational changes constituted a clear signal that, as the century drew to a close, the way government operates needed to change in response to societal conditions.
Despite these constraints, the popularity of the reinventing government agenda and the many serious efforts to implement corresponding organizational changes constituted a clear signal that, as the century drew to a close, the way government operates needed to change in response to societal conditions.
While specific, localized efforts to reinvent government may not have had significant impact, the broader agenda started by the Reagan Revolution was still in play, to the extent that government downsizing, privatization, and contracting out were resulting in what Brint Milward and Keith Provan described as the “hollow state.” Instead of reforming public organizations themselves, this “new public management” entailed shifting more and more public service delivery into the private sector, where organizations were assumed to be more capable and/or efficient. This entailed allocating public funds to private organizations to provide particular services, and while for-profit firms were the recipients of some of this money, much of it went to nonprofit organizations and as a result stimulated the growth and development of this “third sector.” A key role of the relevant public agencies in this context is to manage the relationships with these external parties, making sure they are performing as needed and thus providing value for the taxpayers’ dollar. This shift towards greater reliance on third-party service providers was happening not just in the U.S., but in Europe and the Commonwealth countries as well, where it came to be referred to as “joined up government.”
As the new millennium got underway, the term “collaborative governance” became the preferred moniker in the U.S., reflecting the new reality that government entities were now frequently collaborating with private firms and nonprofit organizations in the course of carrying out their operations. In the nonprofit sector, funders in both government and philanthropies began expecting recipients to develop partnerships and alliances that would enable them to leverage more effectively the funds they were being given to carry out their work. The value of collaborative arrangements also became more apparent in the policy arena, with the realization that difficult policy issues could be addressed most effectively by bringing the various stakeholders together and having them reach agreement about how to move forward. More collaborative approaches to conflict or dispute resolution, many of them grounded in the win-win orientation popularized by Roger Fisher and William Ury in Getting to Yes, began replacing the adversarial tactics that often lead to years of litigation that leave problems lingering unresolved. In short, governance was taking place in more and more contexts through decisions being made by actors from the public, private, and/or nonprofit sectors collaborating to identify the most satisfactory solutions.
Related to the emergence of collaborative governance was a new focus, in both the practice and study of public management, on the interorganizational networks that were being developed to pursue a variety of public purposes. Throughout the economic upheaval of the latter part of the 20th century, interorganizational networks had proliferated in the private sector as well, for an equally diverse set of objectives. In his seminal paper discussing this trend, Woody Powell pointed out that networks constitute a third type of economic organization distinct from both markets and hierarchies. They are different from markets in that the relationships among actors are not simply contractual exchanges, but entail the same kind of long-term, trust-based relationships that are useful in a hierarchical organization. They are different from hierarchies in that actors are all independent and autonomous, with no control systems that help to insure they fulfill their obligations and work towards collective purposes. Interorganizational networks are what Harlan Cleveland described as “uncentralized” systems, in which nobody is in general charge but everyone is partly in charge.
A considerable amount of public management research over the last 20 years has explored the issues and challenges public managers face in their efforts to “manage” networks in the pursuit of public objectives. A frequent focus of these discussions is the difficulty of making things happen and getting things done in the absence of any formal authority to rely on to ensure that others carry out particular tasks. While the ability to exert influence without authority is a skill that is useful in many contexts, it is especially important in collaborative governance settings where managers from different organizations, often with some competing interests and perspectives, must find a way to work together towards some common objectives. For public managers quite used to the power of their authority, and the various bureaucratic mechanisms that reinforce it, trying to guide or steer a network towards particular processes or outcomes often requires new ways of thinking and acting. Many such networks fail, or at least accomplish very little, because those leading them are not able to adapt to operating in a non-hierarchical context.
The obvious challenges of accomplishing goals through an uncentralized network reinforce a deeply ingrained belief in the necessity of hierarchy as a key feature of purposive organizations, i.e., groups of people working together for some overarching purpose. Most people seem to believe that any organized effort to accomplish a goal requires someone to be in charge, a decision-maker at the top of the pyramid who is ultimately responsible for making sure that organizational activities achieve that objective. From this perspective, the network activities public organizations are now engaging in may be useful, but they are essentially supplemental to the primary work of those organizations, which takes place through the hierarchical arrangements of the bureaucracy. Likewise, this perspective would suggest that, regardless of how prevalent these interorganizational networks get, they are still going to be comprised of bureaucratic organizations operating in hierarchical mode that are not going to go away any time soon. In short, it is widely assumed that, regardless of the emergence of collaborative governance systems, the bureaucratic organizational form is here to stay.
However, this belief in the endurance of bureaucracy does not take into account the fundamental changes to society that are and will continue to be instigated by the transition out of the industrial era into the Information Age. With the development of the internet in the 1990s, organizations started integrating into their operations the new capabilities it generated for information dissemination and online interactions. Businesses jumped on these new opportunities and quickly developed the infrastructure for “e-commerce” that allowed them to increase sales while reducing some of their costs. Government agencies naturally moved more slowly to adopt these new tools and technologies, given both constrains on funding needed to implement the changes as well as greater inertia due to a lack of the same incentives motivating private firms to establish an online presence. Inevitably, though, they too began incorporating new internet-based approaches and practices for interacting with the public. In general, these “e-government” reforms made it easier for the public to get information from and provide information to public agencies, and thus to engage in basic transactions with the government.
The belief in the endurance of bureaucracy does not take into account the fundamental changes to society that are and will continue to be instigated by the transition out of the industrial era into the Information Age.
The diffusion of e-government capabilities coincided with the growth of the third sector and greater involvement by community-based organizations in various collaborative governance arrangements. Coupled with the devolution and decentralization reforms associated with the reinventing government movement, the net effect was to stimulate the development of innovative mechanisms for enhancing citizen engagement in the work of government. For example, when voters in Los Angeles approved a revision of the city charter in the summer of 1999, it initiated a process of developing a system of neighborhood councils throughout the city. As a structural reform in the institutional arrangements through which Los Angeles is governed, these councils provide an opportunity for citizens and communities to exchange information with city government agencies pertinent to policy decisions being made and implemented. New processes for citizen engagement have also been implemented, such as the examples of deliberative democracy described by Edward Weeks. As the 21st century got underway, a trend towards greater involvement by citizens in government was readily apparent, with one Nancy Roberts calling it “the age of direct citizen participation.”
While the internet greatly expanded the opportunities for people to interact with government in a variety of ways and for a variety of purposes, most of these interactions were simply bi-directional exchanges of information. This is largely due to the properties inherent in what is now referred to as the first generation of the internet, which enabled one-to-one communications as well as one-to-many information broadcasts and many-to-one information aggregation, but did not readily support many-to-many interactions. With the popularization and proliferation of new social media tools in the last decade, however, the second generation of the internet – Web 2.0 – has produced platforms on which large numbers of people can interact with each other and in turn participate in and contribute to productive processes and creative activities. As Clay Shirky points out, the global population has about a trillion hours of “participatory value” per year to commit to shared projects, and if given the opportunity, some of this “cognitive surplus” could be allocated to tasks associated with improving government policies and services.
While most government agencies have not yet incorporated Web 2.0 technologies and capabilities into their operations, the diffusion of these tools into public organizations seems inevitable, in the same way that most of them have integrated the internet and e-government reforms to enhance internal and external communications. Looking ahead, a number of observers have adopted the label Government 2.0 to refer to the significant changes in governmental structures, processes, and services that will result as public organizations and managers learn how to take advantage of the innovative capacities and opportunities provided by these new tools. Building on the trends towards collaborative governance and citizen participation already apparent, governance in the digital era will be much more open, participative, and collaborative than the bureaucratic systems they could eventually displace.
Some indications of where this is leading can be seen in the “open data” movement and corresponding development of a “civic tech” community. Open data refers to the notion that data collected and/or held by government entities should be publicly accessible and freely available for anyone to use for their own purposes (with proper privacy protections as necessary). The Obama administration put a focus on open data with its Open Government initiative, and the data.gov website now provides access to over 130,000 data sets from a wide range of government agencies and other organizations. The civic tech community includes software developers and other technologists who are developing “apps” that use open data in ways that add value to users. More generally, this community is using a variety of innovative mechanisms, such as hackathons and design competitions, to identify and develop creative approaches for addressing existing public concerns.
A key feature of Web 2.0 is that it facilitates the process of crowdsourcing, which refers to the practice of incorporating input from a large number of people in an effort to improve the quality and/or creativity of a process, product, or service. Crowdsourcing practices reflect growing recognition of the “wisdom of crowds” as well as the power of “mass collaboration.” In Government 2.0, we can expect that governance mechanisms will readily utilize crowdsourcing tools to gather information and ideas pertinent to topics or issues of interest, to then incorporate into the development of policies and decisions regarding their implementation. There are already many examples of public agencies using crowdsourcing activities to get broad input from citizens, as well as of private actors taking the initiative to develop tools through which public organizations can benefit from some form of crowdsourced information. It seems inevitable that the use of these tools and diffusion of their capabilities will become as pervasive as the first generation internet is today. As this happens over the course of the next ten to twenty years, we should expect that the core structures and processes of public bureaucracies, and even the policies they implement and services they deliver, will be modified if not transformed as a result.
The purpose of this historical overview is twofold. The first is to support the supposition that, just as the bureaucratic form of organization was generated and institutionalized in the context of the modern, mechanistic worldview and industrial-era culture, a new and different form is being generated and may become institutionalized in the context of a post-modern, ecological worldview and information-age culture. The second is to suggest that the mechanistic bureaucratic organizational form has been evolving towards a more organic ecological form for a half-century already, and that the latest technological developments are likely to stimulate greater and faster movement in that direction. And just as organizations are changing to adapt to societal transformation, governance systems more generally will be transformed in ways that reflect the opportunities, demands, and expectations of a world that will soon be.
In an effort to articulate the key features of these emergent systems, I wrote a paper a few years ago called “Ecological Governance” (with Taehyon Choi as co-author). After outlining the key properties of ecological systems – they are interconnected, self-organizing, and co-evolutionary – the paper described how the purpose, design, process, and relationships that shape the functioning of governance systems could incorporate these ecological properties. The paper was published in 2010, in a special issue of Public Administration Review focused on the future of public administration in 2020. Admittedly, it seemed at the time that ten years was quite optimistic as an estimate of how soon bureaucratic systems would be replaced by ecological governance, and now that we are halfway there, it still doesn’t seem likely that this transformation will be complete by the end of the decade. However, while the timing is uncertain, we can be quite sure that continued diffusion of ecological thinking and practices, grounded in humanity’s evolution towards higher, unitive consciousness, will eventually generate the kinds of systemic reforms required to create an institutional framework for governance that can effectively respond to the complex challenges of 21st century society. Rather than believing this transformation isn’t possible, the future will be better served if public servants devote their energy and efforts to developing and implementing the kinds of mechanisms needed for Government 2.0 to become a reality.
Just as organizations are changing to adapt to societal transformation, governance systems more generally will be transformed in ways that reflect the opportunities, demands, and expectations of a world that will soon be.
The opinions expressed are those of the author, and do not reflect in any way those of the USC Bedrosian Center.