E is for Evaluation

[part of the Pagan Blog Project]

Evaluation is one of those words that seems to scare people. Which, all right: people don’t like feeling like they’re being judged. And for a lot of people it’s a word that has a great deal of formality and weight behind it – job evaluations, school evaluations. Senses of being weighed and found wanting.

But at the same time, evaluation is something we do every day.

Do you want to see this movie this weekend? That takes evaluation. Do I want to talk to a friend about going to a concert in a sort of nearby town? What should I eat for lunch? Should I spend twenty minutes working on this project or reading this thing or petting the cat? (The last, says my cat. Always.)

Evaluation, in short, is about making choices. Which we talked about last week. More specifically, it’s about making choices in context.

If I’m trying to decide what movie to watch on Netflix, I’m working with a large but fundamentally limited set of choices (what is available streaming on Netflix right now, modified by my taste and preference and mood.) If I am trying to decide which thing to do with my time, I am limited by what’s available in my immediate vicinity. (Swimming on a tropical beach might or might not be fun, but it is not going to be an immediate option if I’m sitting in the foothills of Maine. With my cat.)

And, as I evaluate my choices, I’m going to probably be asking myself some questions. What does this thing bring me that that thing doesn’t? If I make this choice now, what does it mean for later in the weekend? Maybe that’s about money, or about energy, or about balance in my life.  (I want to go to a concert on Saturday night and dinner with friends first, but it’ll involve 90 minutes of driving. Other choices might take less prep time or exertion or cash.)

Here’s the thing: we might make different choices at different times for perfectly good reasons. This Saturday night, I might decide to go to that concert. Next Saturday night, I might decide that I want to stay home, and get ready for a trip. Or write. Or knit.

That doesn’t make going to the concert a wrong choice. And it doesn’t make packing or writing or knitting a wrong choice. It just means that in context, at that moment, some of those choices are better than others.

Evaluating what we read is like that. It’s about context.

One of the things I find so very frustrating about discussions of Pagan books online is that people seem to forget this a lot of the time. The books or resources that help someone – or that help someone at a particular point in their learning – are going to be different from person to person, situation to situation. Of course, there’s going to be overlap (there are a finite number of ‘intro to witchcraft’ books in the world, for example, and even fewer that are widely available to a number of people.)

And thus, how we evaluate a book may change over time. There are books out there that are simplified, but do a good job of hinting at the complexities. There are books out there that simplify in ways that make it harder to go deeper later. There are books out there that are just plain complex, and are probably not the right place for someone new to the topic to start.

So when we’re looking at these books, we need to keep all of that in mind. Both when we’re looking for things for us to read – but also when we’re recommending things for other people.

How do we evaluate? Good question. One of the very first Pagan-related essays I wrote is about that (and I really do need to do an update, because it’s been 12 years…) – you can read it over on my Seeking website. But briefly, the things I look at go like this:

1) Who wrote this thing?

What’s their background (relevant to the topic)? What’s their – apparent or stated – purpose in writing it? Do they have any special expertise that’s relevant? Alternately, do they seem to be claiming that unrelated experience makes them a massive expert? (Watch out for those!)

2) Where does their information come from? How do we know that? How can we check the checkable parts? 

If they’re talking about personal experiences or theories, then “my experience” might be the right answer. If they’re drawing on other people’s research and learning, they should give a way for people to follow it back. There are formal academic methods here, but also a bunch of perfectly workable informal ones.

3) How current is this book? How much does that matter?

Libraries generally review their books about science, health, and technology much more frequently for currency than, say, their books about poetry, because those fields change very rapidly. A computer book from 10 years ago may not be a ton of use to anyone now. In Pagan religions, our conversations about what we’re doing and how we’re doing it, and how we can do it better have developed very rapidly over the past 10-15 years.

4) How useful is this book? 

When I teach research skills, one of the things I talk about is finding sources that give you new information, not just repeat stuff you’ve already seen. You don’t want – or need – 10 sources all saying “Wicca celebrates 8 Sabbats”. That’s not useful. However, 10 different sources talking about variations or specific details might be extremely useful to you, because that’s 10 sets of different but related information.

Usefulness is also a variable thing: what is useful to you when you’re just starting out may not be so useful when you’ve been learning for a while. (And this is a complicated thing, but it’s something many people factor into book recommendations: when someone asks me “what should I read”, I try to guide them toward books that work now, but also are of interest once someone knows more. Better use of time/money/storage.)

5) What’s the author’s goal? 

This is the most nebulous, and yet one of the most important pieces. Is this author trying to sell you something? Sell you on the idea that their way is the One True Way? Are they trying to manipulate your emotions, your intentions, your focus? Or are they laying out information and letting you choose what to do with it?

This isn’t just a distinction between ‘personal’ and ‘academic’ writing – someone can write from the heart and from personal experience, and very persuasively, but not be emotionally manipulative about it. And likewise, it’s possible to write a formal academic text that uses every emotional trick in the book to make you feel like you’re stupid or wrong or hateful to make a choice other than what the author wants you to make. It’s good to keep an eye out, especially before you act on a choice like this.

One of the basic things I use is “Does this book make me feel better about knowing this subject, or worse?” A book that makes me feel worse is probably trying to manipulate me, and I should look more closely. A book that makes me feel wonderful without any substance should also be looked at closely. On the other hand, a book that leaves me feeling like I know more, but there’s still more to learn, or that I disagree with the author in places, is probably less manipulative.

6) What context am I reading and using this thing in?

If I’m reading with an eye toward writing a ritual, I’m looking for different things than if I’m reading for general knowledge, or reading because I’m considering recommending a book to someone else, or because I’m trying to solve a particular issue or because I’m hoping someone else’s experience and stories will help me figure out something. A book may be a great resource for one of those, but not the others. (And very few books will be fabulous for all five.)

Likewise, topic matters – a resource might be excellent for someone doing religious witchcraft, but less useful for someone who doesn’t want the ‘religious’ piece. A book might be great about group work, but less useful to solitary practitioners (or the other way around.) This doesn’t make it a bad book – just not the right book for that purpose or person or moment.

There’s so much more I could say about evaluation, but fundamentally, what I want to leave you with is this: find the tools that help you do what you want, but remember that what doesn’t necessarily work for you isn’t automatically bad or wrong or worthless. Much more often, it has its useful points – they just don’t match up with yours right now.

Bookmark the permalink.