On the history of evaluation in IR

I'm currently writing my PhD thesis and this paper is one of the gems I found: "On the history of evaluation in IR", S. Robertson, Journal of Information Science 34(4), 2008. You can find it here: http://jis.sagepub.com/cgi/content/long/34/4/439 (there's also a citeseer version for those without access I believe).

I think it's a must-read for any PhD student doing IR since it covers the fundamentals (and history!) of evaluation in IR. It's also written in a light and sometimes even funny style.

Enjoy :)

RSS & Co.

One of my students asked me how I keep up with conferences and journals.
Here are the resources I use:

RSS feeds
I subscribe to the blogs of a few people in my research area, and to ILPS PhD Resources of course. But I find RSS feeds the most useful for keeping up with journals. Some of the journals I subscribe to are:

• Computational Linguistics, from MIT Press (current issue | RSS feed)
• Information Processing & Management, from Elsevier (current issue | RSS feed). This is the only journal RSS feed that doesn't include article abstracts, which makes it a lot less useful than the other ones.
  
• Information Research, privately published (current issue | RSS feed)
  
• Information Retrieval, from Springer (current issue | RSS feed). I subscribe to a lot of journals from Springer, but I'm too lazy to look up all the links. Here are just the names: "Cognition, Technology & Work", "Computer Supported Cooperative Work", "Data Mining and Knowledge Discovery", "Minds and Machines", "Pattern Analysis and Applications", "User Modeling and User-Adapted Interaction", "World Wide Web".
  
• JASIS&T (Journal of the American Society for Information Science and Technology),
  from Wiley InterScience (current issue | RSS feed)
• Library Quarterly, from Chicago Journals (current issue | RSS feed)
  

mailing lists
The usefulness of these varies, some mainly have conference announcements, others have lively discussions. I'm not posting the addresses, most of these you can find on the web, or email me.

• Corpora-List
• SIG-IRList
• SIKS (I think people get subscribed to this one when they join the SIKS school)
  
• um (User Modeling, through Yahoo! Groups)
• webir (Yahoo! Groups)
  

Google Calendar
There's a Google calendar called "Information Retrieval & Web Mining Conference Dates". It includes submission deadlines and conference dates. I'm not sure how to create a link to this, but you'll find it if you search for the name in Google's public calendars.

What do you use to keep up with these things? What mailing lists and RSS feeds do you subscribe to?

Information Retrieval (study) resources

If you're looking for a way of getting to know the background of IR, there's a plethora of options (online) available. From the classic book by Keith van Rijsbergen (1979), the upcoming IR book by Manning and Schuetze (who previously wrote the classic "Foundations of Statistical Natural Language Processing") to various lecture pages of IR courses taught around the world. For example, David Carmel's course page at Haifa, James Allen's course page at UMass, Jamie Callan's course page at CMU, or ofcourse the course taught by Maarten de Rijke (which is more general and centered around "internet information").

Lectures on "Search Engines: Technology, Society, and Business"

This lecture series on search ran at UC Berkeley in fall 2005. Although some of the material may already be out of date and generally doesn't go very deep, it's still a good (broad) overview of topics related to search and provides insights from different perspectives. They got some people from industry and the videos of the lectures are good quality.

Here's the course schedule with links to individual lectures: http://www2.sims.berkeley.edu/courses/is141/f05/schedule.html

IR datasets on the web

On behalf of Manos:

Dear ilps-ers,

I found a nice post listing a lot of Large DataSets available on the web:

http://www.datawrangling.com/some-datasets-available-on-the-web.html

There is also a joint initiative to build a list of large datasets which can be found at: http://theinfo.org

You may already have heard most of the datasets but I thought this may prove handy as they are listed altogether.

Manos