Senior PC Review Procedure

EasyChair Full papers track. Two warnings: the watchlist and the senior PC member column.

Roles

Members of the Senior Program Committee (SPC) participate in the reviewing process in two roles: Primary Area Chair (1AC) and Secondary Area Chair (2AC).

See also the detailed description of the SPC responsibilities from the invitation to the SPC.

2AC assignment

As Primary AC, you will flag the papers that are neither clear accepts, nor clear rejects, which would benefit from additional input to resolve different views. In other words, you flag the papers that are considered borderline.

Because this step is newly introduced, we have to manage with a conference system that was not designed for this workflow. We found a process that should work, and request that you follow carefully the steps of the procedure for flagging 2AC reviews.

Keep in mind that 2AC reviews are assigned among the SPC, and those papers require a review in the brief time period between the 1st tier review deadline and March 13th, when the discussion phase starts. So try to keep the number of papers for additional reviews low, including only papers that really require an additional expert evaluation.

2AC review guideline

Please do not look at the other people’s reviews before entering your own.

Given that the 1AC requested your input, you know that the first tier reviews do not reach a trivial consensus to accept or reject the paper. Please feel free to weigh in your expertise as SPC member, and be firm in your evaluation - we think that this helps start an informative discussion phase that will lead to a higher quality set of reviews for the authors.

The first tier reviewers received the following guidelines on how to review for SIGIR 2017.

We are looking for an exciting conference program, and hope that our SPC members keep an open mind to allow for a wide range of topics to be discussed at SIGIR.

Discussion phase

The objective of the discussion phase is to collect sufficient information about each paper under your supervision as to reach a clear verdict. Ideally, reviewers agree on your metareview of the paper, and support the recommendation to the PC Chairs with respect to acceptance or rejection. Only in very rare cases we should see a need to discuss the paper in the PC meeting.

Initiate the discussion phase by asking PC members to clarify aspects of their reviews, to discuss deviations in evaluation between the different first tier reviews.

If the discussion leads to a different outcome than suggested in (a subset of) the first tier reviews, the metareview should reflect the process how the reviewers arrived at the final decision. PC members do not need to modify their review, unless it had shortcomings like missing references or being superficial in its evaluation; or, whenever a misunderstanding of the submission was clarified in interaction with the other reviewers.

Wrap up the discussion phase by requesting feedback on the metareview drafted by you. The deadline for metareviews and final recommendations (accept or reject) is March 27th.

PC Meeting

The optional face-to-face PC meeting will be held at Microsoft in Redmond, WA, USA on March 30 - 31 2017 (schedule .pdf). The primary focus of this optional meeting will be to finalize the list of accepted papers, and to discuss issues related to posters, workshops, etc. We will provide some level of teleconferencing to enable remote participation, but the meeting will take place during normal business hours at Redmond local time (UTC-7). SPCs who wish to attend the PC meeting in person will be expected to self-fund their travel and accommodation.

Timeline

The schedule for the review process is as follows (AC activities in Bold):

Task	Responsibility	Start	Stop
Accept invitation to SPC	ACs		September 18
Suggest referees	ACs		October 1
Abstract submissions	Authors		January 17
Bidding, COI declaration	Reviewers, ACs	January 18	January 23
Paper submissions	Authors		January 24
Assignment of papers to reviewers and ACs	PC chairs		January 31
First round reviews due	Reviewers		February 28
Flag papers needing additional review	1ACs	February 28	March 3
Additional 2AC review for 1AC flagged papers	2ACs	March 3	March 13
Discussion; starts irrespective of missing 2AC reviews	1ACs	March 13	March 27
Meta-review and recommendations (1AC)	1ACs		March 27
PC Meeting; in person in Redmond / with online option	PC chairs, ACs	March 30	March 31
Notifications	PC chairs		April 11

Modifications in 2017

SPC invitees who have had prior involvement with SIGIR reviewing will have noticed the two modifications that we have made. First, we will take charge of the reviewer assignments and balancing referee workload instead of coordinating this step through all the 1ACs, aiming to avoid a bottleneck due to depencency on all ACs availability so early in the process. Second, we involve the SPC with actual reviewing to steer the decision process on those papers where a decision is not obvious based on the initial reviewing phase. The objective of this modification is to make better use of your expertise and experience.

Act Now

If you decide to accept the invitation to join the SIGIR 2017 SPC, let us know by accessing the link included in the preamble of the invitation message.

Next, update your EasyChair account in two steps:

1. Review personal details and contact information: EasyChair/My Account
2. Select areas of interest: SIGIR 2017/My Topics

We look forward to working with you!