logo

ACM RecSys Challenge 2016

About

The RecSys Challenge 2016 is co-organized by XING, CrowdRec and MTA SZTAKI. XING is a social network for business. People use XING, for example, to find a job and recruiters use XING to find the right candidate for a job. At the moment, XING has more than 15 Million users and around 1 Million job postings on the platform. Given a user, the goal of the job recommendation system is to predict those job postings that are likely to be relevant to the user. In order to fulfill this task, various data sources can be exploited. Job recommendations are displayed on xing.com as well as in XING's mobile apps.

Challenge

Task: given a XING user, the goal is to predict those job postings that a user will positively interact with (e.g. click, bookmark).

Procedure: the key challenge is an offline evaluation. Given the training dataset, teams can train their algorithms and can then submit their solution for a sub-set of 150k target users via the submission system (requires a XING account and approval).

The evaluation measure reflects typical use cases on the XING platform.

Rules, download of the dataset, and access to the leaderboard is also handled via the submission system.

Questions

Questions and remarks about the procedure and other aspects concerning the challenge can be submitted as github issues.

Dataset

The dataset is a semi-synthetic sample of historic XING data, i.e. it is not complete and enriched with noise in order anonymize the data and abstract from real user profile data.

Prizes

Prizes are given out to the teams that achieved the highest scores (based on the entire ground truth data - notice that the scores in the leaderboard are based on ca. 1/3 of the ground truth data).

  1. First Team: 3000 €
  2. Second Team: 1500 €
  3. Third Team: 500 €

Important Dates

When? What?
Feb 26th Challenge starts / open for submissions
June 26th (23:59 Hawaiian time) Deadline for submitting solutions
June 29th Official results will be announced
July 20th Deadline for submitting accompanied papers
August 1st Notifications about paper acceptance
August 18th Deadline for submitting camera-ready papers
Sep 15th (9:00-17:30) Workshop will take place as part of the RecSys conference. Location: IBM (room: M1-2217). See: map

Paper Submissions

Each team - not only the top teams - should submit a paper that describes the algorithms that they used for solving the challenge. Those papers will be reviewed by the program committee (non-blind double review). At least one of the authors is expected to register for the RecSys Challenge workshop.

Format

Papers should not exceed 4 pages. They have to be uploaded as PDF and have to be prepared according to the standard ACM SIG proceedings format. Templates:

Upload Paper

Papers (and later also the camera-ready versions) have to be uploaded via EasyChair: submit paper via EasyChair

Publication

We aim to publish the accepted papers in a special volume of ACM Sig Proceedings dedicated for the challenge (cf. Proceedings of the last year: ACM, DBLP).

Program Committee

Workshop Program

Time Session
09:00 - 10:30 Getting started:
  • Welcome and Challenge statistics, Workshop organizers. (20min) [slides, pdf]
Short presentations (10 minutes):
  • A preliminary study on a recommender system for the Job Recommendation Challenge, Mirko Polato and Fabio Aiolli.
  • An ensemble method for job recommender systems, Chenrui Zhang.
  • Jobandtalent at RecSys Challenge 2016, Jose I. Honrado, Oscar Huarte, Cesar Jimenez, Sebastian Ortega, Jose R. Perez-Aguera, Joaquin Perez-Iglesias, Alvaro Polo and Gabriel Rodriguez. [slides]
  • A Bottom-Up Approach to Job Recommendation System, Sonu Mishra and Manoj Reddy.
  • A Scalable, High-performance Algorithm for Hybrid Job Recommendations, Toon De Pessemier, Kris Vanhecke and Luc Martens. [slides]
10:30 - 11:00 Coffe Break
11:00 - 12:30 Long presentations (20 minutes):
  • Job Recommendation Based on Factorization Machine and Topic Modelling, V. Leksin, A. Ostapets. [slides]
  • Temporal Learning and Sequence Modeling for a Job Recommender System, K. Liu, X. Shi, A. Kumar, L. Zhu and P. Natarajan [slides]
  • Multi-Stack Ensemble for Job Recommendation, T. Carpi, M. Edemanti, E. Kamberoski, E. Sacchi, P. Cremonesi, R. Pagano and M. Quadrana. [slides]
12:30 - 14:00 Lunch Break
14:00 - 15:30 Long presentations of the Top 3 (20 minutes):
  • A Combination of Simple Models by Forward Predictor Selection for Job Recommendation, D. Zibriczky. [slides]
  • RecSys Challenge 2016: job recommendations based - on preselection of offers and gradient boosting, A. Pacuk, P. Sankowski, K. Wegrzycki, A. Witkowski and P. Wygocki. [slides]
  • Job Recommendation with Hawkes Process, W. Xiao, X. Xu, K. Liang, J. Mao and J. Wang [slides]
Pictures from prize ceremony during conference banquet at Museum of Science:
15:30 - 16:00 Coffe Break
16:00 - 17:30 Pannel Discussion. Topics:
  • Dataset: what is on your wish list for the RecSys Challenge 2017 dataset?
  • Job recommendation task: not the typical recommendation problem? (evaluation metrics, prediction vs. recommendation)
  • Algorithms: what kind of algorithms work / do not work? What kind of algorithms would you like to try out (but would require a different dataset, evaluation metric, etc.)
  • On-line evaluation: how can we set up an online evaluation task in 2017?
Guests:
  • Wenming Xiao (Team Member First Place)
  • Andrzej Pacuk (Team Member Second Place)
  • Róbert Pálovics (Organizer)
  • Till Plumbaum (CrowdRec Project)

Organizers

XING AG Twitter