Re: Papers are better with code: ICASSP session
- From: dbell <bellda2005@xxxxxxx>
- Date: 30 Apr 2007 13:03:14 -0700
On Apr 30, 2:29 pm, David Gelbart <for-spam-o...@xxxxxxxxxxxxxx>
wrote:
In January, Erik de Castro Lopo started a long thread here on
comp.dsp by calling for signal processing papers to include sufficient
source code for the reproduction of results.
ICASSP is one of the main conferences associated with the IEEE Signal
Processing Society. It happened two weeks ago, and there was a
special session on this topic of reproducibility. I've included the
list of papers below, along with URLs for those papers which I've
found on the public web.
I was particularly interested in the paper by Prof. Jelena Kovacevic,
a past editor-in-chief of IEEE Transactions on Image Processing.
She makes concrete proposals about changing reviewing standards in the
Signal Processing Society to place more emphasis on reproducibility.
Now is a good time for those of you who'd like to see this to make
your voices heard, because the ICASSP special session has put a
spotlight on this issue. Personally, I would be glad to sign an open
letter or petition in support of placing more emphasis on
reproducibility in the review process, and in support of using her
proposals as the starting point for discussion of how to do this.
SS-L5: Reproducible Signal Processing Research
SS-L5.1: EXPERIENCES WITH REPRODUCIBLE RESEARCH IN VARIOUS FACETS OF
SIGNAL PROCESSING RESEARCH
Patrick Vandewalle, Guillermo Barrenetxea, Ivana Jovanovic, Andrea
Ridolfi, Martin Vetterli, Ecole Polytechnique Federale de Lausanne
(EPFL), Switzerlandhttp://infoscience.epfl.ch/getfile.py?recid=97195
SS-L5.2: REPRODUCIBLE COMPUTATIONAL EXPERIMENTS USING SCONS
Sergey Fomel, University of Texas at Austin, United States; Gilles
Hennenfent, University of British Columbia, Canadahttp://slim.eos.ubc.ca/Publications/Public/Conferences/ICASSP/2007/fo...
SS-L5.3: PUTTING REPRODUCIBLE SIGNAL PROCESSING INTO PRACTICE: A CASE
STUDY IN WATERMARKING
Mauro Barni, University of Siena, Italy; Fernando Perez-Gonzalez,
Pedro Comesana, University of Vigo, Spain; Guido Bartoli, University
of Siena, Italy
SS-L5.4: REPRODUCIBLE RESEARCH: CASE STUDY ON SAMPLING SIGNALS WITH
FINITE RATE OF INNOVATION
Pina Marziliano, Nanyang Technological University, Singapore
SS-L5.5: STATE OF THE ART AND EVOLUTION IN PUBLIC DATA SETS AND
COMPETITIONS FOR SYSTEM IDENTIFICATION, TIME SERIES PREDICTION AND
PATTERN RECOGNITION
Joos Vandewalle, Johan Suykens, Bart De Moor, Katholieke Universiteit
Leuven, Belgium; Amaury Lendasse, Helsinki University of Technology,
Finlandftp://ftp.esat.kuleuven.ac.be/pub/SISTA/ida/reports/06-167.pdf
SS-L5.6: HOW TO ENCOURAGE AND PUBLISH REPRODUCIBLE RESEARCH
Jelena Kovacevic, Carnegie Mellon University, United Stateshttp://andrew.cmu.edu/user/jelenak/Repository/07_04_Icassp_Kovacevic.pdf
David,
I used to attend ICASSP. There were some real impressive papers and
demonstration, so I am not slamming ICASSP.
However, there were too many audio presentations where people were
"putting their best foot forward", presenting their best results as
typical, and there was also some methodology in some of the audio
presentations that made the performance appear much better than it
was. One method I saw was playing the same audio over and over again
while 'reducing the quality' so that by training the listeners the
quality of the final version sounded much better than if it had been
played first; in one case the final result was not actually
intelligible, but the audience "understood" it (I covered my ears for
all but the final playing).
Sometimes the results were so specific to the audio they used that the
method was of no interest, but that was not ever stated. I asked one
of the authors of such a paper about the paper during a job interview
years ago, he actually blushed bright red, admitted that it didn't
work well on audio, said it worked on what they were interested in,
and ... I didn't get an offer.
My point is, if you require supplying the code, you also need to
require people to supply a copy of any data that is processed by the
code for the paper. This requirement will probably discourage more
than a few authors. I don't think that is a bad thing.
Dirk
.
- Follow-Ups:
- Re: Papers are better with code: ICASSP session
- From: David Gelbart
- Re: Papers are better with code: ICASSP session
- References:
- Papers are better with code: ICASSP session
- From: David Gelbart
- Papers are better with code: ICASSP session
- Prev by Date: Re: Sigma-Delta A/D Converter
- Next by Date: Re: Spectral analysis
- Previous by thread: Papers are better with code: ICASSP session
- Next by thread: Re: Papers are better with code: ICASSP session
- Index(es):
Relevant Pages
|
