Participative Research labOratory for Multimedia and Multilingual Information Systems Evaluation
By rauber - Posted on April 4th, 2011
Measuring is a key to scientific progress. This is particularly true for research concerning complex systems,whether natural or human-built. Multilingual and multimedia information systems are increasingly complex: theyneed to satisfy diverse user needs and support challenging tasks. Their development calls for proper evaluationmethodologies to ensure that they meet the expected user requirements and provide the desired effectiveness.Large-scale worldwide experimental evaluations provide fundamental contributions to the advancement ofstate-of-the-art techniques through common evaluation procedures, regular and systematic evaluation cycles,comparison and benchmarking of the adopted approaches, and spreading of knowledge. In the process, vastamounts of experimental data are generated that beg for analysis tools to enable interpretation and therebyfacilitate scientific and technological progress.PROMISE will provide a virtual laboratory for conducting participative research and experimentation to carry out,advance and bring automation into the evaluation and benchmarking of such complex information systems, byfacilitating management and offering access, curation, preservation, re-use, analysis, visualization, and mining ofthe collected experimental data. PROMISE will: foster the adoption of regular experimental evaluation activities;bring automation into the experimental evaluation process; promote collaboration and re-use over the acquiredknowledge-base; stimulate knowledge transfer and uptake.Europe is unique: a powerful economic community that politically and culturally strives for equality in itslanguages and an appreciation of diversity in its citizens. New internet paradigms are continually extendingthe media and the task where multiple language based interaction must be supported. PROMISE will direct aworld-wide research community to track these changes and deliver solutions so that Europe can achieve one ofits most cherished goals.