Meeting Summarization

Paper: Improving Meeting Summarization by Focusing on User Needs: A Task-Oriented Evaluation

Written by: Pei-Yun Hsueh and Johanna D. Moore

Comments: Brandon

Summary:

This paper conducted a study on recorded meetings and how fast they could extract the decisions made in the meetings. The goal was to make a good meeting browser so that people could see summaries of the meeting and figure out the decisions that were made in each meeting. There were 35 participants where 20 of them were female and 15 were male. They filled out a questionnaire about their prior experience with computer usage and meetings, and then they had to analyze four meetings and summarize what decisions were made in the meetings. They had to write a brief summary of the decisions in 45 minutes using the browser interface. The meeting browser had audio capability so that every participant could listen to the meeting or select portions to listen to. The browser also had four different summary displays.

1. Baseline (AE-ASR): automatic general purpose extracts with automatic speech recognition
2. AD-ASR: automatic decision focused extracts with automatic speech recognition
3. AD-REF: automatic decision-focused extracts with manual transcription
4. Topline (MD-REF): manual decision focused extracts with manual transcription

The graph above are the results from the analysis of the summaries that each participant created from the different browser options. It shows that on average the participants that were given the decision-focused extractive summaries were able to hit each decision point much better than the general purpose one. They again tested this to see why there was a difference with an analysis of variance. The decision-focused displays seemed much easier to use to the participants and they thought it was less demanding as well.

Having the display option for the user was beneficial to all participants in achieving their task. It was much easier for the participants to go through the decision summaries in order to figure out the decision points then it was to go through the extractive summaries that were usually rather long. Only a few participants actually went through the audio-video recordings that were made available in order to find a few more of the decisions. The conclusion that the paper came to was that they couldn't conclude how efficient their browser was since the baseline decisions were still of high quality. The topline decision-focused participants were able to go through the task much quicker, but since all participants concluded around the same decisions they could not pinpoint the efficiency of the browser. Even when the users encountered 30-40% inconsistency with the data from the actual meeting being translated into the extracted summary they were still able to come up with what the decisions were. So basically their browser worked well for either scenario, but they couldn't tell us how efficient it was.

Discussion:
This paper was semi-interesting and more just confusing. For some people it may be nice to have this to look back on a few things they were uncertain of or allow people that could not make the meeting see what the meeting was about, but I think that with the way businesses are being run that anyone could attend a meeting from wherever they were and this wouldn't be an issue. I think this browser could be improved and made more useful for a business to use if they found a way to highlight the decision points without someone having to go through the meeting and picking the decisions that were made. I wish I understood their methods a little bit better though and hopefully after I read it again for the presentation I can talk more about it.