Tag Archives: methodology

thinking about ‘sentiment analysis’

I just got off the phone with a researcher this morning who is interested in looking at sentiment analysis on a corpus of fiction, specifically by having some native speakers of Japanese (I think) tag adjectives as positive or negative, then look at the overall shape of the corpus with those tags in mind.

A while back, I wrote a paper about geoparsing and sentiment analysis for a class, describing a project I worked on. Talking to this researcher made me think back to this project – which I’m actually currently trying to rewrite in Python and then make work on some Japanese, rather than Victorian English, texts – and my own definition of sentiment analysis for humanistic inquiry.*

How is my definition of sentiment analysis different? How about I start with the methodology? What I did was look for salient adjectives, which I searched for by looking at most “salient” nouns (not necessarily the most frequent, but I need to refine my heuristics) and then the adjectives that appeared next to them. I also used Wordnet to look for words related to these adjectives and nouns to expand my search beyond just those specific words to ones with similar meaning that I might have missed (in particular, I looked at hypernyms (broader terms) and synonyms of nouns, and synonyms of adjectives).

My method of sentiment analysis ends up looking more like automatic summarization than a positive-negative sentiment analysis we more frequently encounter, even in humanistic work such as Matt Jockers’s recent research. I argue, of course, that my method is somewhat more meaningful. I consider all adjectives to be sentiment words, because they carry subjective judgment (even something that’s kind of green might be described by someone else as also kind of blue). And I’m more interested in the character of subjective judgment than whether it should be able to be considered ‘objectively’ as positive or negative (something I don’t think is really possible in humanistic inquiry, and even in business applications). In other words, if we have to pick out the most representative feelings of people about what they’re experiencing, what are they feeling about that experience?

After all, can you really say that weather is good or bad, that there being a lot of farm fields is good or bad? I looked at 19th-century British women’s travel narratives of “exotic” places, and I found that their sentiment was often just observations about trains and the landscape and the people. They didn’t talk about whether they were feeling positively or negatively about those things; rather, they gave us their subjective judgment of what those things were like.

My take on sentiment analysis, then, is clearly that we need to introduce human judgment to the end of the process, perhaps gathering these representative phrases and adjectives (I lean toward phrases or even whole sentences) and then deciding what we can about them. I don’t even think a human interlocutor could put down a verdict of positive or negative on these observations and judgments – sentiments – that the women had about their experiences and environments. If not even a human could do it, and humans write and train the algorithms, how can the computer do it?

Is there even a point? Does it matter if it’s possible or not? We should be looking for something else entirely.

(I really need to get cracking on this project. Stay tuned for the revised methodology and heuristics, because I hope to write more and share code here as I go along.)

* I’m also trying to write a more extensive and revised paper on this, meant for the new incarnation of LLC.

arsenal of research: organizing citations, PDFs, notes, brainstorming, and drafts

Post title courtesy of the tyrannical Brian Vivier.

Although I post about the content of my research quite a bit (when I do post), I thought I’d take a step back and talk about the research process today. I’m going to write about a very specific aspect: the ways in which the computer helps me organize and engage in my research.

Obviously, there are things like databases and library catalogs, which are a topic for another day. Many people I talk to don’t know the first thing about WorldCat, so it needs to be addressed! But let’s pretend I already have my sources. Now what do I do?

When I read, I’m very traditional. I take notes with pen and paper when I have a book or a photocopied source. In fact, I used to print out PDFs too, and highlight and write in the margins. Well, that turned out to be a terrible idea. Your highlights and margin notes are not very accessible when you’re coming back to the document later to brainstorm, outline, or write.

My lesson learned – learned after many difficult situations – was to take notes like I’m never going to see the source again. My advisor recommended I do this with primary sources, but if you take long notes that involve mostly direct quotes from the sources, there’s no need to buy the book or really even check it out again. There’s no need to keep binders and binders of printed-out PDFs. So that’s the kind of note-taking I do with pen and paper, first.

The next step is to get them into the computer, because I want them to be 1) stored somewhere safe (I do daily external HD backups, plus sync, more later on that), and 2) searchable, and also 3) copy and paste-able. But where to keep them? How to organize?

I have gone through several pieces of software trying to figure this out, and I’ve settled on Mendeley. I first used Scrivener even for note-taking, which is a great program, but bad for citation management. I then tried Zotero, but that turned out to be bad for PDF management. What I really wanted was a good database that would save my citations, any PDFs I happened to have (I’m currently digitizing all of my sources from my dissertation so they don’t get lost or damaged, and so I can free up my filing cabinet for other things), and ideally let me take notes and even annotate or highlight the PDFs.

Well, despite Mendeley being owned by the devil (Elsevier), it’s free and it actually does everything I need with only a few minor nitpicks, and does it in a way that makes me supremely happy. (My nitpicks are no nested bulleted lists in the notes, and no shortcut keys for bold/italics in the notes.) If you have a PDF attached to your citation and it has OCR, Mendeley’s search function will search not only your citations, notes, and annotations, but also inside the PDFs. It can be overkill at times, but it’s pretty amazing.

So step two of my research organization process is the painstaking, mindless, thankless task of typing my pen-and-paper notes into Mendeley under the appropriate citation. It’s boring but worth it. As I mentioned above, it searches all my notes, and I can copy and paste them into Scrivener, which I will address next. As I type my notes, at the very least I copy and paste them into brainstorming documents as appropriate (usually full quotes), and if I’m up to it, I do some free-writing to brainstorm how the source informs my topic and what I could write about related to it. This usually brings up new ideas I didn’t know I had.

What happens after I get all the notes typed in, PDFs organized and annotated if I have them? I next move over to Scrivener. I’ve been using it for over five years, for both research and creative writing, and can’t sing its praises enough. It’s a word processor that creates a database for your project, where you can store your reference materials, brainstorming ideas, notes, and draft. And more, if you can think of other areas you need to record notes in. Unlike old Scrivener (when I first started using it), you can now add footnotes and comments that port straight to MS Word when you compile your document for it, making the transition to final draft in Word very easy. (Sadly, publishers seem to prefer things that are not Scrivener databases when reviewing.) The typical things I store are the draft itself (of course), a research diary of brainstorming that I update periodically, brainstorming specifically about sources and particular concepts or points, and also under the “Notes” section the comments and suggestions and draft corrections I receive from others. So I keep my full writing process, except for mind mapping/concept mapping (another post), all in one place. It’s amazing.

I’m extremely happy with these two pieces of software; my only complaint is that neither of them does all of what I want, and I have to use two different things complementarily. Well, the situation is still significantly better than several years ago, when I used Mendeley Alpha and it deleted my entire library of citations multiple times. Yikes. Now its syncing works perfectly and I haven’t had a library failure yet. (Fingers crossed).

Next posts will include mind mapping software, how I take notes, how to effectively find and import source citations, and how I deal with multiple languages in my citations.