If the Facts Are Against You

Religious News Service is carrying a screed by Tom Ehrich railing against the use of Average Sunday Attendance (ASA) as a metric. Well, actually he quickly lapses into a denuciad against measurement at all:

Now consider how many useless metrics we use to chart our nation’s course. Politicians fixate on the federal deficit, for example, because it’s an easy number to fling at opponents, even though it doesn’t truly measure the nation’s economic well-being. Same with the official unemployment rate — which measures one small flow in a deepening pool of economic despair. Same with the “approval ratings” that give shallow insight into popularity, but say nothing about quality of leadership. Many investors cling to the Dow Jones industrial average, even though averaging prices on 30 blue chip stocks says little about stock market trends. Business leaders look at earnings per share, while ignoring the research-and-development pipeline, innovation, customer service, employee turnover and capital spending. Politicians and edu-crats force public schools to measure performance through standardized tests that yield simple PowerPoints and talking points, while they ignore actual learning, as well as readiness for adulthood, teacher morale and return on investment. A one-day test, the SAT, is misused as the key to college admission.

Well, this is platitudinous rubbish. And while I can rail against the misuse of statistics and numbers better than he can, because I actually have a mathematical background, he protests way too much.

Few statistics are perfect, of course; there are inaccuracies in collecting them, and there are issues in interpreting them. But really, far-less-than-perfect data is almost always better than nothing at all, and Ehrich's airy-fairy alternatives are essentially that nothing. ASA is, in actuality, a pretty good measurement. It is objective, well-defined, and easy to collect: give the ushers a counter, and have them submit a slip in the offering plate. Its one weakness is the well-known "Christmas effect" wherein the movement of the Nativity through the week causes a certain cyclical variation in the numbers. But that effect is small and well-understood.

And as a measure of church activity, it's a reasonable indicator. If people will not come to church to participate in weekly worship, how strong is their attachment to the Church? When Ehrich says that "A much better quantitative measure would get at “touches,” that is, how many lives are being touched by contact with the faith community in its various Sunday, weekday, off-site and online ministries —and then, for a qualitative measure, asking how those lives are being transformed": this is so much twaddle. How can anyone measure this? They can't; they have to use a surrogate number. And getting people to come to church is precisely that surrogate. We count baptisms, confirmations, receptions, and so forth; if we need a record of "transformation" as it is recorded in the sacramental life of the church, well, we have that record already.

No, the real problem, as with every other statistic he denounces, is that the message the number gives is unfavorable. SAT scores and standardized testing show that we as a nation are not making headway against the failure to educate the disadvantaged; the federal deficit is part of a larger picture of political and economic dysfunction. None of these numbers is useless, and though they all may be abused, the information they provide is essential for understanding the social ills in which we are mired. And it is likewise with ASA: its steady decline shows a church that is slowly collapsing in on itself, a church in which, each year, 3% of the membership chooses to find something else to do on Sunday morning. It's bloody obvious that the reason for counting these "transformations" instead is that, since they cannot be counted, one can make up any positive story one wants instead of confronting the fact of relentless decline.

It would be something else if one could write cautionary notes about how an increase in ASA doesn't necessarily mean an increase in the Kingdom. But really, the facts are plain: as Kirk Hadaway has been wont to say, we have no positive statistic. The Episcopal Church is slowly withering, and he only possible positive in this is that large-scale departures are such a large part of the losses, so that there might be some hope for stagnation once the troglodytes are all driven off. But as it stands, our future is not to be recovered by counting vague "touches"; it can only be recovered by getting people to commit to Christ, become members of his Church, and participate in regular worship. ASA is a measurement of this last, and be it imperfect, its relentlessly negative message needs to be heeded and addressed, not brushed off with platitudes about the fallibility of statistics.