DMI Blog

Amanda Hickman

The Great Database Caper

I've tossed and turned about my first guest contributor entry here on the lovely DMI blog. How do I slide technology in between The CUNY Compact and Predatory Lending? I could write about some very cool tools that are out there--I've never met a grassroots organizer who can't think of a use for a barcode scanner that talks to their database, or a database that can be loaded onto PDAs and synced with the server at the end of a day of door knocking[1])--but the truth is that again and again what community led organizations desperately need is a decent database and some training. All the little stuff -- basic security, network health, keeping the web site up to date--is training. The thing you can't get in a class is a really good database. I've taken to calling this the One True Database, which sometimes elicits laughs from fellow techies because of course half the problem is that there is no "one true database" when Community Voices Heard is trying to figure out how to keep in touch with hundreds of voters who pledged to turn out at polls without overloading their membership database, while the Asian Pacific Environmental Network has no use for a database that can't support multiple Asian alphabets. In my experience the truth turns out to be that most community led organizations with a healthy variety of active campaigns and an engaged community of members and leaders need something close to the same thing from their database, or they need it to be flexible in about the same way.

CVH and FUREE Families United for Racial and Economic Equality will tell you that their database is fantastic (sweet of them, since the LINC Project built and supports their databases and as a LINC Circuit Rider I appreciate the flattery) but the truth is that when FUREE wanted to know how to do a search for home child care providers who had come to at least three of their five recent meetings about challenges that home child care providers face in New York City, I couldn't come up with a clean way to do it.

There are great databases geared towards non-profit, membership organizations, but there is a catch: most non-profits use their databases (and their members) primarily to raise funds. Organizations that prioritize member engagement want to use their database to look at members as more than deep pockets and eventually they'll hit a wall using a fundraising database to do that. And most non-profit membership database packages are essentially fundraising databases. There are some great examples of organizing databases out there, but not one of them is perfect.

If you want to see some that I like, have a look at del.icio.us for reviews. If you are wondering just what this "del.icio.us" thing is you'll just have to sit tight and wait for tomorrow's "Nifty Tool Chest" entry.


[1] I'll confess, I am taunting you dear reader. If you want to know more about how this works you have to ask.

Posted at 6:32 AM, Dec 07, 2005 in Democracy | activists | Permalink | Comments (10) | TrackBack (0)


Comments

i think this is great! progressive issues need to know that there are tools that are inexpensive and effective!

Posted by: Noel Hidalgo | December 7, 2005 10:44 AM

Amanda points out the catch in nonprofit databases... they are geared toword fundraising. This is becuase any software project responds to its "customers." Commercial firms respond to folks that pay them money. Open source projects respond to people that participate.

No one is magically going to create the "One True Database" until grassroots organizers either spend enough money on databases to be a market (at which point commercial firms will respond) or participate in the "open sourcy" software development efforts underway (Metrix, BaseBuilder, ODB, CiviCRM, eBase).

The most exciting thing I have seen recently is the dot.organize project which brings community organizers (including Amanda) together to really hammer out what they need.

Once there is a critical mass of community organizers thinking about and using technology, it becomes very easy for them to impact the roadmap and software development of projects like CiviCRM.



Right now, one of the biggest influences on the CiviCRM platform is Canadian political organizations. Why? Becuase they post messages on the list, ask for features, and deploy CiviCRM in the real world. Simple as that. http://www.openngo.org/

Posted by: David Geilhufe | December 7, 2005 10:53 AM

Good work Amanda! Let me take the bait and talk a bit about CiviCRM. If an organization is looking ofor a robust and inexpensive online database solution, CiviCRM is a strong contentder--if the organization wants an online solution. I personally think that online solutions are great, however, for organizations that are technology infrastructure strapped, this might not be the best route to go in: low bandwidth connections would probably result in the organization's staff reverting to excel spreadsheets!.

CiviCRM has been thoughtfully developed by a community of folks who have worked with non-profits and databases before. The lead developer was responsible for Groundspring's toolset. Folks like myself who have provided DB support to non-profits, grassroots and political organizations contributed (and continue to contribute) feedback to the development team.

CiviCRM is an open source project. The source code is open for all to see and improve. It is also free (libre and beer) and is distributed with CivicSpace (an open source CMS based on Drupal). There are many consultants and firm like mine (CivicActions) who can install, configure and host CiviCRM.

What can CiviCRM do? Contact management, relationship management (relationships between contacts, as in people to people and organizations to organizations and people to organizations) each record can have mutiple relationships. Activity tracking, issue tracking, notes tracking, email list management and blast mailing, and now even contribution tracking (these last two with CiviMail and CiviContribute). The product is still BETA, but it very strong and in production use with many organizations already.

Posted by: Gregory Heller | December 7, 2005 12:09 PM

Thanks, Amanda, for mentioning the ODB software. We have put some screenshots of our upcoming version up at http://flickr.org/photos/organizersdb/sets . Make sure you click on the "All Sizes" button in Flickr to see the screens in their full sizes.


One thing that should be pretty clear from the screenshots is that this is not really an Access application; it is actually a Visual Basic application, which has made it possible for us to implement all kinds of interactivity that would not be possible or practicable with Access, or with a web browser based application.

For example you can set up an ODB report that opens up an Open Office document, and have the Open Office document access the ODB data via an ODBC connection.

We will be making an announcement on December 19 of our upcoming release; stay tuned!

Posted by: Rich Cowan | December 7, 2005 02:04 PM

amanda, as usual, you're telling it like it is! thanks for taking the time to illustrate what the sector is really struggling with. that's the first step in imagining new approaches.

Posted by: Leda Dederich | December 7, 2005 08:53 PM

Thanks for kicking off this great discussion. We're excited about our upcoming release of Metrix, about the other great database work that's going on in the sector, and about the recent communication amongst various non-profit database teams.

Metrix, free as in beer and as in speech, is built in Microsoft Access, which is already on so many Windows desktops, and is backed by a free (beer only) version of Microsoft's SQL Server.

Out of the metaphorical box, Metrix offers contact management, the ability to build connections between contacts, a query building wizard for simple queries, the ability to use the native query builder, Access's powerful report builder, simple calendar functionality, export to Excel and Word, and the ability to track simple "interactions" (a flag to indicate a role, the date a meeting was attended, etc.).

The power of Metrix, though, is its extreme customizability--those simple interactions can be extended in any way any other Access form can be: multiple tabs and subforms for related data, data validation code, code that prints reports, sends e-mail, and anything else you can do with VBA

Metrix is community software, and we're already seeing that pay off. We do more and more consortium builds where we work with several organizations to build a set of features they will all be able to use, so they can pool their resources and we end up with fewer customizations to support. And now we're beginning to share those customizations with other organizations--for free, since they've already been paid for.

To facilitate that kind of sharing, we're building a tool to extract customizations from the base Metrix build and a Forge on which to share those customizations. Version 1.2.1 of Metrix should be out by the end of January, and the extractor and the forge will be up later in 2006.

Posted by: Jeremy Wallace | December 8, 2005 09:28 AM

One thing I have to come clean on is that as a former vollunteer coordinator for political campaigns (and 1,000 other things) lots of my volunteers had a hard time learning the computer databases.
Mostly middle-aged and up vollunteers. But these aren't even retirees-- these are working people who use computers in their jobs who still found the software confusing. And yes, some of it was clearly phobic on their part.

But people developing this stuff need to include better directions on how to show others how to use it.

Simplified directions so that when my volunteers are making calls, they can enter their responses straight in to the system. There should also be "uglified" versions that can be used on the old computers we're all stuck using without causing everything to crash (as always happens because the computers we use are usually from the 5 years back or worse)

Posted by: grassyrootsy | December 8, 2005 11:03 AM

grassrootsy, thank you so much for an honest response. To the folks pitching products, well, to quote John Leguizamo from ER last night "bullcrap." Cause really, while it's nice to hear from coders, implementers, and intermediaries, I feel like the majority of the comments here have been sales pitches which is the last thing we need - can we hear from some non-techie folks actually using databases to support their organizing work?

Oh, right, they're out organizing. Ok, so maybe "bullcrap" is a tad bit strong. But, since we're in pitch mode, how about some honest "here's what this tool sucks at" or "here's where the negative feedback from users is clustered." Remember folks, the title of Amanda's post here is the Great Database Caper - and in case you've forgotten, a caper (wikipedia here) ". . . involves one or more crimes (especially thefts, swindles, or occasionally kidnappings) perpetrated by the main characters in full view of the reader."

The caper in this case is that all the tools the techies are referring to above each have their own unique frustrations and limitations - not that you'd know that from looking at what's been written about them here or most other places. And yes, that would the swindle I'm talking about, carried out in plain view of the users.

David's right though, the only way out is to organize our way out through an independent process driven by organizers. I have high hopes for dot.organize, but as always it's hard to say if there's any there there, till its there.

Posted by: anonymoustechie | December 9, 2005 04:52 PM

I think bullcrap is a bit strong -- I, too, would like to hear from any organizers in the DMI audience who are looking for something more than they've got, but all the folks who left long pitch comments are working on tools that are free of charge.

David is a lead on the CiviCRM Project with a vested interest in seeing it get wider use. Gregory has nothing particular to gain from wider use of CiviCRM except perhaps if the fallout from wider use is that it will continue to improve.

All of the databases mentioned above, CiviCRM, ODB, Metrix, are free to download and install, but to varying degrees fairly expensive to support at the present moment. Not as expensive as, say, Raisers' Edge, but then I don't think many organizations are sitting down to decide between Metrix and Raisers Edge.

Anonymoustechie, if you think that you won't know if there is "any there there" at dot.organize, how should organizers drive a process to organize (y)our way out of the database muddle? (In my own professional life I walk a fine line between being part of the organizing community that needs better tools and being part of a tech community that is trying to develop them. I am not sure whether 'your' or 'our' fits better there).

That is a huge question, and I don't think that progressive (or retrograde racist patriarchal) organizations are going to get good organizing tools until organizers and developers figure out how to communicate, and until organizers figure out how to contribute to software development projects. I like to think that LINC does a good job of bridging some of the communication gaps, but LINC can't do it alone.

Years ago, ebase published a Truth in Advertising statement that I think was an important step towards exactly that, but I haven't seen it replicated and I don't know if it went far enough.

Which brings me to another question: whose job is it to articulate what a tool can and cannot do? Is there room for some thoughtful side by side comparison of these projects, what they do and don't do and what it takes to get them going? Would anyone use that to make a decision or would it just make the techies feel better, knowing that something had been writ?

I should have looked up caper, I wasn't really thinking too deeply when I chose my words. I do think that, on the whole, when I've tried to explain to pretty much anyone that many organizations don't have the database that they need I get some variation on "What's wrong with ODB?" or "Can't they use CiviCRM?" and the answer is always a little different. They need to be able to track event participation or they are on Macs or whatever it is.

I don't think it is quite a "swindle, before our very eyes" but there is no question that, while there are some great tools that a lot of folks don't know about, there are also some real limitations with each of those tools and not nearly enough openness about what and where those limitations are.

And yet, as someone who doesn't spend a ton of time boasting of my own personal limitations (not quite true--buy me a beer and I'll cry in it for ya) I have a hard time stomping my feet and insisting that software developers do what no one else manages to.

Posted by: Amanda Hickman | December 9, 2005 06:24 PM

I don't think its a swindle.

I just think my vollunteers need help.
And that I need a new computer to run anything.

I do think that programs could benefit by being workshopped by organizers and stuff.

Posted by: grassyrootsy | December 9, 2005 07:04 PM