Project Listen/JListen
Program and Data Auralization (Sonification) project
[The project is once again active! Yeah!]
Principal Investigator: Aditya P. Mathur
Latest update: Feb 6, 2012
Project History (Last Revision August 25, 2011 (pdf))
Current Graduate Students: None.
Current Undergraduate Students: Shawn Hsu and Jiangnan Shangguan [Starts Spring 2012]
Download
JListen 1.0
JListen1.2
JListen User manual [pdf]
Introduction
As the importance of multimedia grows, we envisage increasing use of sound as an output medium. Examples of use of sound are virtual reality systems, simulations, video games, education for the visually-handicapped computer users and data analysis systems. In most of these applications sound is emitted during the execution of an application when an event occurs or during an activity. Addition of sound to such an application requires (a) identification of locations in the code that are centers of such events or activities and (b) adding suitable code responsible for emitting sound. The effectiveness of sound as a medium in an application depends, amongst other factors, on how well (a) and (b) are performed.
The Listen/JListen systems have been designed to help in performing these tasks in a friendly environment on a PC, Mac, or a workstation. The current version of Listen can be used for auralizing C programs on Sun workstations. We used Listen to conduct experiments in understanding program behavior, testing and debugging, classroom teaching, and development of software for the blind.
The Jlisten Project: The JListen project grew out of the Listen project. The Listen Specification Language (LSL) has been adapted to Java. LSL/Java allows the specification of aspects of a Java program to be auralized. A specification written in LSL/Java is processed by the LSL/Java parser and input to the JListen parser. JListen inputs a Java program P that is to be auralized and the processed LSL/Java specifications. It then generates an instrumented version of P. The instrumented P is compiled using a traditional Java compiler. During its execution P is connected to a Media Manager. Execution of specific aspects of P causes messages to be sent to the Media Manager that in turn sends appropriate commands to an audio system allowing the generation of sounds. The Media Manager allows run-time control of sounds.
R. Jagadish Prasath and M. C. Gopinath, both graduate students at BITS, Pilani, completed two excellent MS theses on the use of auralization in testing for security. Scroll down this page to download copies of their theses. Please make appropriate citations to their thesis if your work is a follow-on to theirs...Thanks.
JListen is once again undergoing significant changes. Shawn is working on addining dynamic and static data sonification to JListen. Static sonification is what NASA's xSonify does by requiring the data to be sobified to be available prior to sonification. Dynamic sonification is done on data while it is being generated. In the case of JListen we assume that data is being generated inside a Java program. The data so generated is sonified. We expect dynamic sonification to be useful in rapid identification of trends in large scale simulations that take place in a variety of areas such as fluid dynamics, neural simulations, weather simulations, etc.
Audio Samples [Click the sample to listen]
C Programs sonified using Listen/C [.au files] [Sonification by: David Boardman]
The above audio files were generated using Listen 1.0 and MIDI equipment. We used Roland's Sound Canvas SC 55 for synthetic sounds.
Java programs sonified using JListen 2.x [.mp3 files] [Sonification by: Jiangnan Shanggun, Feb 1, 2012]
The above audio files were generated using JListen 2.x under development by Shawn Hsu and Jiangnan Shangguan.It is best to listen to the above samples with the code and the mapping infront of you.
Publications
- Master's Theses
- "LISTEN: An Environment for Program Auralization": Dave Boardma, s MS thesis, Purdue University, USA. (August 1994, 171 pages) (pdf)
- "ON Program Auralization": Vivek Khandelwal, MS Thesis, Purdue University, USA. (May 1995, 166 pages) (pdf)
- "Auralization of an intrusion detection syste": M. C. Gopinath, BITS, Pilani, India. May 2004 (doc file of MS thesis, 81 pages)
- "Auralization of Web server using JList": . R. Jagadish Prasath, BITS, Pilani, India. May 2004 (doc file of MS thesis, 90 pages)
- Refereed and other Publications
- Preliminary Report on Design Rationale, Syntax, and Semantics of LSL: A Specification Language for Program Auralization. David B. Boardman and Aditya P. Mathur, September 1993, pp 51. (pdf version)
- Preliminary Report on Design Rationale, Syntax, and Semantics of LSL: A Specification Language for Program Auralization. David B. Boardman and Aditya P. Mathur, September 1993, pp 51. (Original LaTeX and related files in a directory)
- LSL Programming Language (ps)
D.Boardman, V.Khandelwal, and A. P. Mathur. LSL: A Specification Language for Program Auralization, Proceedings of International Conference on Auditory Display, ICAD~94, November 7-9, 1994, Santa Fe, NM, pp 257-264.
- Music Composition and program behavior (pdf)
A. P. Mathur. Using computer programs as generators of compositions, Proceedings of the IV Simposio Brasiliero de Computacio e Musica, Brasilia, DF 3-7 August 1997, pp 31-41.
- LISTEN system (pdf)
D. Boardman, G. Greene, and V. Khandelwal, and A. P. Mathur. LISTEN: A Tool to Investigate the Use of Sound for the Analysis of Program Behavior, Proceedings of the Nineteenth Annual International Computer Software & Applications Conference (COMPSAC'95), IEEE Computer Society Press, August~9-11, 1995, Dallas, Texas, pp 184-193.
- Ed Gilmore and Bryan Strait's Listen/Java presentation on October 25, 2002 at Argonne national labs (ppt slides).
Code
Kindly send email to apm@purdue.edu when you download any code from this site. Thank you.
Contributors
- David Boardman (Independent Consultant):
- Graduate student, the first one to work on Listen; developed Listen/C 1.0 single handedly. Modified the LSL language developed originally by Mathur. Also advised three undergraduate students to improve Listen/C 1.0. Continues to contribute to the Listen project.
- Vivek Khandelwal (Nortel):
- Graduate student, added features to Listen 1.0; conducted experiments with mutation testing and sounds.
- Howard Chen:
- Undergraduate student, added features to Listen 2.0.
- Ed Gilmore (Purdue):
- Undergraduate student, Listen/Java grammar, LSL parser, and overall project oversight. [Fall 2002]
- Chris Kanich (Purdue):
- Undergraduate student, Listen/Java GUI.
- Bob Liu:
- Undergraduate student, added features to Listen 2.0.
- Jack Lowry:
- Undergraduate student, added features to Listen 2.0.
- Geoff Greene:
- Undergraduate student, added GUI to Listen/C 2.0.
- Nate Nystrom (Cornell):
- Undergraduate student, designed/implemented a general purpose Listen server.
- Bryan Strait (Purdue):
- Undergraduate student, Listen/Java instrumenter. [Fall 2002]
- Cary Supalo:
- Undergraduate student, assisted with experiments using Listen 2.0.
- Eric Wiegman (Tellabs):
- Undergraduate student, developed the "say" command for use with Listen 2.9.
- S. A. Ashok, Chris Kanich (Purdue), K. R. Jayaram, K E Dorai, Khemraj Sharma, Meenali Rungta, Prashanth Ganesh, Sanjay Kulhari, Shubra Pradeep, Sundaresan R, Vaibhav Ranjan
- Undergraduate students, developed portions of JListen 1.0 during the Fall of 2003 at Purdue University and BITS, Pilani, India.
- R. Jagdish Prasath:
- Graduate student at BITS, Pilani, developed JListen 1.2 with Gopinath during the Spring of 2004.
- M. C. Gopinath:
- Graduate student at BITS, Pilani, developed JListen 1.2 with Jagdish during the Spring of 2004.
- Jack Saalweachter
- Undergraduate student, Computer Science, Purdue University. Worked to instrument thread-related events in Java programs. Plans to study the usefulness of auralization in debugging thread-related errors. [Spring 2006]
- John Valko
- Undergraduate student, Computer Science, Purdue University. Used JListen to create innovative musical compositions. [Spring 2006] So far no significant output generated.
- Shawn Hsu
- Undergraduate student, Computer Science, Purdue University. Implementing (Fall 2011--) dynamic information (raw and anlyzed data) sonification, JListen as a service (JLAS).
- Jiangnan Shangguan
- Undergraduate student, Computer Science, Purdue University. ( Spring 2012) generating music using JListen
Similar Projects