Mountain of Accountability
Wednesday, May 7, 2014
Mountain of Accountability
Pursuing mission through learning, exploration and development
While the Mountain of Accountability is not a definitive statement on accountability, we believe that sharing it may illuminate for our partners and peers Blandin Foundation’s thinking and processes. We appreciate your feedback.
Since 2007, Blandin Foundation has committed itself to building an organization-wide assessment system that contributes to improved performance and adaptation to a changing world. Blandin Foundation’s journey in pursuit of mission through learning, exploration and development is well summarized by an observation by Kathleen Enright, CEO of Grantmakers for Effective Organizations:
"Philanthropy still has a way to go before it can deliver on the true promise of evaluation to drive learning and deliver better results for organizations and the communities they serve. The challenge for grantmakers is to weave evaluation into the fabric of what they do every day, and to shift the focus of this work so it's about improvement, not just proof."
In 2013, Blandin Foundation engaged evaluation expert and author Michael Q. Patton to take its assessment work to a new level. Together we explored how the field of “developmental assessment” might strengthen the work of the foundation, and how all of the various assessments at play might work together.
A simple graphic has come to be valuable in understanding and describing what is, admittedly, a complex system. Blandin Foundation’s Mountain of Accountability© summarizes its three levels of accountability, and the interconnections among them. The journey to the summit (mission fulfillment) begins in the foothills of basic accountability. From there, the ascent leads up to the middle of the mountain where more complexity and commitment is involved. The final level leading to the summit, with its holistic and comprehensive panorama, offers no pre-set trail. This is first-ascent territory, where the conditions along the route and what has been learned along the way combine to inform further learning and guide the way to the summit.
Upward arrows along the left side show that routine data inform higher level inquiry, and downward feedback arrows along the right side show that what is learned at the top informs strategies and actions at the lower levels (e.g., that reflective practice informs future strategic planning and operations management).
Level 1: BASIC ACCOUNTABILITY
The first level of accountability focuses on fundamentals and answers two basic questions:
- Did we implement our work as planned and authorized?
- Did we meet basic quality standards in carrying out our work?
Answering these basic questions involves the following kinds of assessments:
- Financial audits
- Investment returns
- Staff performance management and assessment
- Basic management information systems reports
- Routine grantee reporting
- Community trend indicators for planning
- Basic program evaluations – who attended, how many, etc.
- Adherence to government and legal oversight
This first level of accountability assesses the extent to which resources are well-managed by adhering to generally accepted accounting standards, wise and astute portfolio management, quality personnel management practices, systematic grantmaking and program processes, implemented with due diligence and professionalism, that meet demands for public accountability and reporting. In classic logic model terminology, basic accountability focuses on inputs, activities, processes, and outputs. The data for basic accountability should be embedded in fundamental management processes. Basic accountability data support ongoing management decision-making and inform Board oversight.
Level 2: ACCOUNTABILITY FOR IMPACT AND EFFECTIVENESS
Once basic accountability processes and assessment reporting are in place, more specialized accountability assessments are possible, appropriate, and necessary to take programmatic effectiveness assessment to the next level. By assessing program and organizational outcomes and impacts, impact and effectiveness accountability answers three important and challenging questions:
- To what extent are we building strong, positive relationships with key stakeholders and partners?
- To what extent and in what ways are we attaining desired and intended program outcomes and impacts?
- What are we learning, and in what ways are we applying what we are learning to improve effectiveness?
These are more specialized accountability questions because answering them requires data that go beyond what can be generated by basic internal management and monitoring information. More specific assessment work is necessary, which includes:
- Major program evaluations
- External strategic plan evaluation
- Board survey & feedback
- Program Participant Surveys
- Grantee Perception Report on Foundation relationships and perceived effectiveness
- Synthesis of grantees’ reports on impacts
- Public perception studies
- Employee surveys
While the first level of basic accountability assesses the extent to which resources and processes are well-managed and of high quality, the second more advanced level assesses what differences the Foundation is making. As with basic accountability, the Foundation has multiple sources of data that document a broad range of positive impacts in the communities it serves.
Level 3: ACCOUNTABILITY FOR LEARNING, DEVELOPMENT, AND ADAPTATION
While the second level of accountability does an excellent job of evaluating at the program level and getting feedback from specific stakeholder groups, the third level takes on complexity and dynamic systems change, both in the Foundation’s internal operations and in the Foundation’s external initiatives, to bring about community change. At this level, “developmental evaluation” is utilized and the focus is on learning to adapt strategy, systems change, mission fulfillment, principles, and values.
Developmental evaluation is an approach that specifically supports innovation and adaptation through systems thinking and complexity science understandings. While traditional evaluation approaches (middle of the mountain) focus on improving and making decisions about projects and programs, developmental evaluation addresses strategy implementation and effectiveness at the overall organization and mission fulfillment levels.
Developmental evaluation integrates accountability, improvement and ongoing development with particular attention to changes in the organization’s environment ( for example, economic, social, demographic, policy, and technological changes) that affect strategic adjustments. Such an assessment approach supports adaptation and innovation through reflective practice. The major assessment questions for learning and development are:
- How are the world and the systems we work in changing, and how do we understand those changes so as to learn, adapt, and develop?
- What does it mean to take our strategic framework seriously and inquire deeply into its elements and their implications for how we do our work and the impacts we have?
- What are the interrelationships and interconnections between and among our diverse stakeholders and partners?
- To what extent do the many and diverse parts of the Blandin Foundation constitute a coherent and aligned whole?
These questions are particularly challenging because they don’t yield simple answers. Nor is the process of arriving at answers simple. The questions are often called “wicked” because inquiry into them is a journey of cumulative insights and adaptations rather than pound-the-fist-on-the table-with-certainty answers. But the increasing evidence from high-performing organizations is that the willingness to take on these wicked questions generates important insights and actionable learning that increase effectiveness and impact.
Accountability for Learning & Development involves:
- Deep reflective practice
- Developmental evaluation
- Strategic evaluation
- Focusing on systems change, innovation, adaptation, and making a difference in complex, dynamic systems
INTERCONNECTIONS AMONG THE THREE LEVELS OF ACCOUNTABILITY
Each of the three levels can be defined categorically and implemented specifically – and that is what most foundations and organizations do. They generate a series of separate and discrete assessments.
In contrast, the challenge – and opportunity – when seriously engaged in accountability for learning and development is to see, understand, and use the findings from three levels to inform each other.
- Findings from basic accountability assessments inform program evaluation priorities and questions.
- Findings from program evaluations provide the case data to ask deeper, more strategic questions in reflective practice when looking for patterns across cases, across programs, across stakeholder groups, and across communities.
- Knowledge gained from reflective practice is used to inform future strategic planning as well as ongoing learning and adaptation.
- Program-level impacts are synthesized to examine changes at the community and system levels.
Effectiveness of basic operations and support services within the Foundation are used to understand basic operations and support services among grantees. The ways in which basic operations and support services contribute to outcomes and impacts become part of the larger systems inquiry.
Permission and Citation
Blandin Foundation permits use of this document for non-commercial purposes, subject to full attribution (see the suggested citation reference below). For permission to use this material for commercial purposes, please contact the Foundation at 218-326-0523 or firstname.lastname@example.org.
Citation reference: Patton, Michael Quinn and Blandin Foundation (2014). Mountain of Accountability: Pursuing mission through learning, exploration and development. Grand Rapids, MN: Blandin Foundation, blandinfoundation.org.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. http://creativecommons.org/licenses/by-nc-nd/4.0/deed.en_US
What People are Saying
"I really appreciate the way this framework is laid out. It is really helpful to visualize the layers of accountability this way and be able to articulate the key questions under each one. World Savvy is engaged in reflective practice and developmental evaluation but we have not always been able to articulate where these practices fit in relationship to our commitment to evaluating our impact and effectiveness. I also appreciate that the framework includes both internal and external accountability. Too often these concepts are talked about and outlined in entirely separate places and ways. Looking at evaluation more holistically like this will help to make it more mainstreamed in organizations' work rather than an add on/afterthoughts." Charmagne Campbell-Patton, World Savvy April 1, 2014
"I really like it." Rebekah Sobel, U.S. Holocaust Memorial Museum April 29, 2014
"As the Director of Research and Evaluation for Sherwood, this makes evaluation use (what kinds of data and for what purpose) very clear within the foundation's ecosystem. I don't have feedback so much as to hear concrete details from you. Sherwood is a new foundation and I am a newly minted evaluator. There is a lot of mojo around the foundation and amongst my program officer colleagues for strategy discussion and the idea of the basic reporting functions (lower levels of the mountain) makes everyone very sleepy! We have a grants management system that people are using, but not for the larger strategic discussions. I noticed you have a synthesis of grantees' reports on the mid-level of the mountain. Do you pull these from your MIS? Do you have a pattern or framework from which you conduct the grantee report synthesis? This feels like an elementary question, but I'd be curious on your work to make these things meaningful in order to support the larger strategic thinking. Many thanks." Erin Bock, Director of Research and Evaluation, Sherwood Foundation, May 6, 2014
"This is fantastic work. Well done.
Here are the insights it generated for me: The capital stored into the base of the mountain can make it harder to truly pivot when developmental learning on systemic implications of the work leads to desires to shift things dramatically. Hence:
- Accountability for the top of the mountain cannot be left only to "after we've figured out" the base levels otherwise too much capital may be stored into how the organisation does things ; Yet accountability for the top of the mountain is certainly not enough on its own, as effective deployment of resources toward the vision is also paramount. This is a polarity.
- Finding ways to build sturdy but light structures of accountability at the base levels and crystallize the minimum level of capital into a given model/structure allows to be OK dismantling and redirecting if the developmental insights suggest a dramatic shift. How can an organisation stay amorphous enough to be flexible, while having strong systems at the base? This is another polarity.
Thanks for this framework. Very well done! Boris Martin, Engineers Without Borders Canada June 12, 2014
"I read with great interest your Mountain of Accountability report. What a great, straightforward framework to think about evaluation and assessment. Congratulations on this work and thank you for the valuable contribution it makes to the field!
I wondered whether you have developed reporting mechanisms or dashboards to track your progress along these various levels - how you report out to your board and community within this framework? We are in the process of looking more closely at our board reporting and it would be great to know whether you've developed any tools that align with this great framework." Mary-Kim Arnold, Evaluation and Learning Officer, Rhode Island Foundation Sept. 8, 2014
YES!!!!!!!!! This is exactly what we have been discussing and conceptualizing at our Foundation. We have called our model "continuous impact improvement" and have done our best to articulate what you have SO BEAUTIFULLY articulated here! THANK YOU!
An evaluation consultant sent this to me after hearing what we were trying to do at our Foundation. I am so delighted to have another model to refer to!
Leila Martini, Director of Policy and Research, Foundation for a Healthy St. Petersburg, Dec. 2, 2015
Assessment of BCBP using the Mountain of Accountability Framework
(PDF) - 3.6 MB
Joint report: Blandin Foundation use of Mountain of Accountability
(PDF) - 1.0 MB
Mountain of Accountability
(PDF) - 117 KB