Project Listen/JListen

Principal Investigator: Aditya P. Mathur
Latest update: January 20, 2006

Current Graduate Students:  None.

Current Undergraduate Students : Jack Saalweachter and John Valko

Click here to download JListen 1.0 and 1.2

Project History (Last Revision September 10, 2003 (pdf))

Introduction

As the importance of multimedia grows, we envisage increasing use of sound as an output medium. Examples of use of sound are virtual reality systems, simulations, video games, education for the visually-handicapped computer users and data analysis systems. In most of these applications sound is emitted during the execution of an application when an event occurs or during an activity. Addition of sound to such an application requires (a) identification of locations in the code that are centers of such events or activities and (b) adding suitable code responsible for emitting sound. The effectiveness of sound as a medium in an application depends, amongst other factors, on how well (a) and (b) are performed.

The Listen/JListen systems have been designed to help in performing these tasks in a friendly environment on a PC, Mac, or a workstation. The current version of Listen can be used for auralizing C programs on Sun workstations. We used  Listen to conduct experiments in understanding program behavior, testing and debugging, classroom teaching, and development of software for the blind.

The Jlisten Project: The JListen project grew out of the Listen project. The Listen Specification Language (LSL) has been adapted to Java. LSL/Java allows the specification of aspects of a Java program to be auralized. A specification written in LSL/Java is processed by the LSL/Java parser and input to the JListen parser. JListen inputs a Java program P that is to be auralized and the processed LSL/Java specifications. It then generates an instrumented version of P. The instrumented P is compiled using a traditional Java compiler. During its execution P is connected to a Media Manager. Execution of specific aspects of P causes messages to be sent to the Media Manager that in turn sends appropriate commands to an audio system allowing the generation of sounds. The Media Manager allows run-time control of sounds.

R. Jagadish Prasath and M. C. Gopinath, both graduate students at BITS, Pilani, completed two excellent MS theses on the use of auralization in testing for security. Scroll down this page to download copies of their theses. Please make appropriate citations to their thesis if your work is a follow-on to theirs...Thanks.

Audio Samples

• Hear a sample Bubble Sort .
• Hear a sample Selection Sort .
• View the LSL/C specification (plain text file) corresponding to the above Bubble and Selection sorts.

The above audio files were generated using Listen 1.0 and MIDI equipment. We used Roland's Sound Canvas SC 55 for synthetic sounds. JListen audio samples are yet to be recorded and placed on this site.

Publications

Master's Theses

Dave Boardman's Master's thesis on the Listen System, Purdue University, USA. (August 1994, 171 pages) (pdf)

Vivek Khandelwal's Master's Thesis on Listen 2.0 and its application, Purdue University, USA. (May 1995, 166 pages) (pdf)

Auralization of an intrusion detection system. M. C. Gopinath, BITS, Pilani, India. May 2004 (doc file of MS thesis, 81 pages)

Auralization of Web server using JListen. R. Jagadish Prasath, BITS, Pilani, India. May 2004 (doc file of MS thesis, 90 pages)

Refereed and other Publications

Preliminary Report on Design Rationale, Syntax, and Semantics of LSL: A Specification Language for Program Auralization. David B. Boardman and Aditya P. Mathur, September 1993, pp 51. (pdf version)

Preliminary Report on Design Rationale, Syntax, and Semantics of LSL: A Specification Language for Program Auralization. David B. Boardman and Aditya P. Mathur, September 1993, pp 51. (Original LaTeX and related files in a directory)

LSL Programming Language (ps)

D.Boardman, V.Khandelwal, and A. P. Mathur. LSL: A Specification Language for Program Auralization, Proceedings of International Conference on Auditory Display, ICAD~94, November 7-9, 1994, Santa Fe, NM, pp 257-264.

Music Composition and program behavior (pdf)

A. P. Mathur. Using computer programs as generators of compositions, Proceedings of the IV Simposio Brasiliero de Computacio e Musica, Brasilia, DF 3-7 August 1997, pp 31-41.


LISTEN system (pdf)

D. Boardman, G. Greene, and V. Khandelwal, and A. P. Mathur. LISTEN: A Tool to Investigate the Use of Sound for the Analysis of Program Behavior, Proceedings of the Nineteenth Annual International Computer Software & Applications Conference (COMPSAC'95), IEEE Computer Society Press, August~9-11, 1995, Dallas, Texas, pp 184-193.

Ed Gilmore and Bryan Strait's Listen/Java presentation on October 25, 2002 at Argonne national labs (ppt slides).

Code

• Listen/C 1.0 Code
• Listen/C 2.0 Code
• Listen/C 3.0: Contact Aditya P. Mathur.
• JListen V1.0 and V1.2

Contributors

• David Boardman (Independent Consultant):
  • Graduate student, the first one to work on Listen; developed Listen/C 1.0 single handedly. Modified the LSL language developed originally by Mathur. Also advised three undergraduate students to improve Listen/C 1.0. Continues to contribute to the Listen project.
• Vivek Khandelwal (Nortel):
  • Graduate student, added features to Listen 1.0; conducted experiments with mutation testing and sounds.
• Howard Chen:
  • Undergraduate student, added features to Listen 2.0.
• Ed Gilmore (Purdue):
  • Undergraduate student, Listen/Java grammar, LSL parser, and overall project oversight. [Fall 2002]
• Chris Kanich (Purdue):
  • Undergraduate student, Listen/Java GUI.
• Bob Liu:
  • Undergraduate student, added features to Listen 2.0.
• Jack Lowry:
  • Undergraduate student, added features to Listen 2.0.
• Geoff Greene:
  • Undergraduate student, added GUI to Listen/C 2.0.
• Nate Nystrom (Cornell):
  • Undergraduate student, designed/implemented a general purpose Listen server.
• Bryan Strait (Purdue):
  • Undergraduate student, Listen/Java instrumenter. [Fall 2002]
• Cary Supalo:
  • Undergraduate student, assisted with experiments using Listen 2.0.
• Eric Wiegman (Tellabs):
  • Undergraduate student, developed the "say" command for use with Listen/2.9.
• S. A. Ashok, Chris Kanich (Purdue), K. R. Jayaram, K E Dorai, Khemraj Sharma, Meenali Rungta, Prashanth Ganesh, Sanjay Kulhari, Shubra Pradeep, Sundaresan R, Vaibhav Ranjan
  • Undergraduate students, developed portions of JListen 1.0 during the Fall of 2003 at Purdue University and BITS, Pilani, India.
• R. Jagdish Prasath:
  • Graduate student at BITS, Pilani, developed JListen 1.2 with Gopinath during the Spring of 2004.
• M. C. Gopinath:
  • Graduate student at BITS, Pilani, developed JListen 1.2 with Jagdish during the Spring of 2004.
• Jack Saalweachter
  • Undergraduate student, Computer Science, Purdue University. Is working to instrument thread-related events in Java programs. Plans to study the usefulness of auralization in debugging thread-related errors. [Spring 2006]
• John Valko
  • Undergraduate student, Computer Science, Purdue University. Is using JListen to create innovative musical compositions. [Spring 2006]