a Celebration of Failure in VIS Research

Thank you to everyone who joined us for the 2020 VIS Fail Fest Workshop! The workshop took place virtually on Sunday, October 25th from noon to 3:30pm Mountain Time (MT). If you missed us, you can still watch the recording on YouTube!

Workshop Schedule

Session 3 (90 min)	Presentation & Moderated Panel
12:00p-12:10p (10 min)	Opening Remarks
12:10p-12:30p (20 min)	Workshop organizer lightning talks
12:30p-12:40p (10 min)	Leilani Battle: Walkthrough of VIS review process template
12:40p-1:20p (40 min) 1:20-1:30 (10 min Q&A)	Panel Discussion: Kristin Cook, Alark Joshi, Stephanie Evergreen, Alberto Cairo
1:30p-2:00p (30 min)	Session Break
Session 4 (90 min)	Paper Talks and Discussion
2:00p-2:10p (10 min) 2:10p-2:15p (5 min Q&A)	Sunwoo Ha, Adam Kern, Melanie Bancilhon, Alvitta Ottley: Expectation Versus Reality: The Failed Evaluation of a Mixed-Initiative Visualization System
2:15p-2:25p (10 min) 2:25p-2:30p (5 min Q&A)	Shivam Agarwal, Shahid Latif, Fabian Beck: How Visualization PhD Students Cope with Paper Rejections
2:30p-3:15p (45 min)	Nadieh Bremer: Visualizing Connections
3:15p-3:30p (10 min)	Closing remarks

Workshop Papers

Expectation Versus Reality: The Failed Evaluation of a Mixed-Initiative Visualization System Sunwoo Ha, Adam Kern, Melanie Bancilhon, Alvitta Ottley Abstract Our research aimed to present the design and evaluation of a mixed-initiative system that aids the user in handling complex datasets and dense visualization systems. We attempted to demonstrate this system with two trials of an online between-groups, two-by-two study, measuring the effects of this mixed-initiative system on user interactions and system usability. However, due to flaws in the interface design and the expectations that we put on users, we were unable to show that the adaptive system had an impact on user interactions or system usability. In this paper, we discuss the unexpected findings that we found from our "failed" experiments and examine how we can learn from our failures to improve further research.

How Visualization PhD Students Cope with Paper Rejections Shivam Agarwal, Shahid Latif, Fabian Beck Abstract We conducted a questionnaire study aimed towards PhD students in the field of visualization research to understand how they cope with paper rejections. We collected responses from 24 participants and performed a qualitative analysis of the data in relation to the provided support by collaborators, resubmission strategies, handling multiple rejects, and personal impression of the reviews. The results indicate that the PhD students in the visualization community generally cope well with the negative reviews and, with experience, learn how to act accordingly to improve and resubmit their work. Our results reveal the main coping strategies that can be applied for constructively handling rejected visualization papers. The most prominent strategies include: discussing reviews with collaborators and making a resubmission plan, doing a major revision to improve the work, shortening the work, and seeing rejection as a positive learning experience.

Featured Speakers

Template Walkthrough: Paper Submission Revision Document

Dr. Leilani Battle is an Assistant Professor at the University of Maryland, College Park, with a joint appointment in the University of Maryland Institute for Advanced Computer Studies (UMIACS). She is also affiliated with the UMD Human-Computer Interaction Laboratory (HCIL). Her research interests focus on developing interactive data-intensive systems that can aid analysts in performing complex data exploration and analysis. Her current research is anchored in the field of databases, but utilizes research methodology and techniques from HCI and visualization to integrate data processing (databases) with interactive interfaces (HCI, visualization). Prof. Battle was named one of the 35 Innovators Under 35 by the MIT Technology Review in 2020. She is also an NSF Graduate Research Fellowship Recipient (2012), and her research is currently supported by an Adobe research award, an NSF CISE CRII Award (2019-2021) and ORAU Ralph E. Powe Junior Faculty Enhancement Award (2019-2020). In 2017, she completed a postdoc in the UW Interactive Data Lab. She holds an MS (2013) and PhD (2017) in Computer Science from MIT, where she was a member of the MIT Database Group, and a BS in Computer Engineering from UW (2011), where she was a member of the UW database group. See some of Dr. Battle's projects here, and keep an eye out for A Structured Review of Data Management Technology for Interactive Visualization and Analysis at VIS!

Visualizing Connections

Nadieh Bremer is a graduated Astronomer, turned data scientist, turned data visualization designer, based near Amsterdam. She's working as a freelancer under the name "Visual Cinnamon". As 2017's "Best Individual" in the Information is Beautiful Awards, she focuses on uniquely crafted (interactive) data visualizations that both engage and enlighten its audience. She's worked with a wide range of companies, such as Google News Lab, Sony Music, the New York Times and UNESCO, to small start-ups. Creating custom visuals for print, crafting interactive online experiences, or designing promotionally focused visuals for press releases, data-driven reports, and data art. You can visit her website at VisualCinnamon.com and reach out on Twitter at @NadiehBremer or Instagram @NadiehBremer.

Panel Discussion

	Dr. Alark Joshi is an Associate Professor & Department Chair of Computer Science at the University of San Francisco. He has published many research papers in the field of data visualization (with an emphasis on medical visualization) and has been on award-winning panels at the IEEE Visualization conference. He has taught Data Visualization courses to Computer Science as well as Data Science majors. He was awarded the Distinguished Teaching Award at the University of San Francisco in 2016. He received his Ph.D in Computer Science from the University of Maryland Baltimore County. Read more about Dr. Joshi's work on his website here or follow on Twitter @alark.
	Dr. Stephanie Evergreen is a data visualization and design expert. She has trained future data nerds worldwide through keynotes and workshops, for Fortune 500 clients like MasterCard and Facebook and mission-aligned clients like the United Nations, the Boys and Girls Club, AARP, and The Alaska Native Trial Health Consortium. She has three books and a popular blog on data visualization. Check out Stephanie's website here or follow her on Twitter @evergreendata.
	Dr. Alberto Cairo is a journalist and designer with many years of experience leading graphics and visualization teams in several countries. He joined the School of Communication in January 2012. He teaches courses on infographics and data visualization. He is also director of the Center for Visualization, Data Communication & Information Design at UM’s Institute for Data Science and Computing, and a Faculty Fellow at the Abess Center for Ecosystem Science and Policy. Read more here or follow on Twitter @AlbertoCairo.
	Kristin Cook is an Advisor in the Visual Analytics Group at Pacific Northwest National Laboratory. In her more than twenty years in the field, Kris has led teams to perform innovative research and transition it into use. With Jim Thomas, she co-edited the initial research and development agenda for visual analytics, working with a diverse team of experts from industry, academia, and government research. She has also co-chaired the IEEE Visual Analytics Science and Technology (VAST) Challenge since 2011. Kris’s current research interests include human-machine teaming and the use of mixed initiative techniques to support analysis. Read more about her publications here.

Call for Participation (Closed for 2020!)

Members of the community are invited to submit a story of failure in their own visualization research that they would like to share with the world, or propose a way to better collect and use these stories as a community. Some possible causes of failure might include:

• Findings that did not generalize as expected
• Studying a non-representative population
• Methodology regrets ("I wish I'd remembered to ask...")
• Flawed data analysis pipelines
• Improper application of statistical methods
• Under- or over-estimating time, money, or other resources
• Not effectively leveraging diverse team skills
• Submitting to the wrong venue
• Not admitting failure or asking for help in a timely manner

For more information, email the organizers.

Workshop Organizers

Jane Adams is a data visualization artist at the University of Vermont Complex Systems Center. Website @artistjaneadams	R. Jordan Crouser is an assistant professor in the Computer Science Department at Smith College. Website @jordancrouser	Paul Rosen is an assistant professor of Computer Science at the University of South Florida. Website @PaulRosenPHD	Lonni Besançon is a researcher in Human Computer Interactions and Information Visualization in Norrköping. Website @lonnibesancon