The Interesting Parts of the ABA’s Legal Technology Survey Are What It DOESN’T Say

Annual surveys make good (read: popular) copy, but unless they have insightful scope and keen survey instruments, those reports frequently turn out to be incomplete social science of limited practical value. Judging from the ABA’s press materials and Adrian Dayton’s summary, the “2010 ABA Legal Technology Survey Report: Web and Communication Technology” unfortunately follows that pattern. It appears to tally surface-level indicators of social media participation, but does not drill down far enough into that data to reveal meaningful, useful information.

Isn’t it time that legal marketing got past the “gee whiz” bandwagon stage of social media participation and started seriously studying its application and effectiveness?

The ABA’s “Book Briefs” blurb cites the following top-line findings:

“When asked whether they personally maintain a presence in an online community/social network such as Facebook, LinkedIn, LawLink, or Legal OnRamp, 56% of respondents answered affirmatively, compared with 43% in the 2009 survey and 15% in the 2008 survey.”

What does “maintain a presence” mean? Does it still count if they signed up for an account but have not updated it for weeks or months? How frequently do they post to or update each profile? Also, what does “personally” mean?’ That they write and post the content themselves, or that the account pertains to them personally (as opposed to their firm) and their marketing staff helps?

“The highest percentage of respondents report maintaining a presence in LinkedIn (83%), followed by Facebook (68%) and Plaxo (18%).”

Again, how frequently do they visit these sites, what do they do while there and what specific results is that participation generating? Much has been made of data indicating lawyers spend more time on Facebook than on LinkedIn, but that in itself is a poor predictor of its efficacy as a marketing platform unless you have data on their primary activities and expected benefits from that participation. There’s a lot of non-business stuff to do on Facebook; not so much on LinkedIn. In marketing parlance, you have a more “intentional” (there for a specific purpose) and “qualified” (likelier to be receptive to what you’re selling) audience on LinkedIn than Facebook. Participate in Facebook if it works for you, not just because it has buzz momentum and all the cool kids are hanging out there.

“Respondents in the 30- to 39-year-old age group are the most likely to report that they maintain a presence in an online community/social network (77%, compared with 72% in the 2009 survey), followed by 68% of 40- to 49-year-olds (compared with 58% in the 2009 survey), and 50% of 50- to 59-year-olds (compared with 35% in the 2009 survey).”

What useful conclusions can be drawn from that? It appears to be a fairly typical adoption curve by age group. Either confirm that or identify unexpected findings and tease out their meaning.

“Large-firm respondents are the most likely to report personally maintaining a presence in an online community/social network (63%, compared with 57% in the 2009 survey and 13% in the 2008 survey); 52% of solo respondents (compared with 37% in the 2009 survey and 15% in the 2008 survey) maintain such a presence.”

Now that’s interesting! Did they bother to mine the data for reasons why that might be? Maybe because large firms have marketing staffs to help “maintain a presence,” while solos lack the time/resources OR don’t anticipate sufficient near-term ROI to justify that investment.

“The most common reason respondents report for maintaining a presence in online communities/social networks is for professional networking (76%), followed by socializing (62%), client development (42%), career development (17%), and case investigation (6%). Three percent chose the “other” category.”

Aren’t socializing, client development and career development just broad and overlapping aspects of professional networking? More concrete options like “speaking opportunities,” “reputation and awareness,” “client leads” and “new job opportunities” would have yielded cleaner data, and “professional networking” could have taken the place of “other.”

“Far from being a time waster, nascent efforts at social networking are yielding fruit. Ten percent of respondents report that they have had a client retain their legal services as a result of use of online communities/social networking.”

Actually, that conclusion irks me the most because the data to support such a conclusion clearly isn’t there. To make such a determination you’d have to know how much time respondents put into social networking, how many direct leads those activities have yielded, how much of total new business is attributable to leads from social networking and how much the resulting client engagements were worth. Even the credible and creditable Adrian Dayton fumbled this data point, saying “While 10% may seem small, it represents a dramatic shift in law firm attitudes towards social media.” No, it reflects client behavior, not law firm psychology.

What if a lawyer who responded affirmatively to that question spent 10 hours a week on social media last year and got only one matter, but it represented nearly half of his billings? Then yes, that’s certainly not a waste of time. Conversely, what if another respondent, this one working at a Big Law firm, spent the same amount of time on social media participation, also got one retainer out of it, but it only amounted to less than five percent of his total billings? He might have gotten more — and more lucrative — leads after doing a free CLE session at a bar event.

Here’s another way to look at it: How “dramatic” would it be to learn that 10 percent of respondents report that they have had a client retain their legal services as a result of use of Yellow Pages advertising?

Legal marketing has thrown its lot in with social media for many good reasons, but also for some bad ones — uncritical acceptance of and enthusiasm for generalized, anecdotal data chief among them.

I’m willing to allow that the additional data and analysis I’m looking for might be found in the full report, but at $350 per copy ($300 for members), I’m not going to take that gamble based on what’s been reported so far.