
\documentclass[12pt]{article}
%\documentstyle[11pt]{article}

%
% Listen Specification Language (LSL) paper.
% Started writing: June 2, 1993. Aditya Mathur.
% First draft available: June 28, 1993
% Second draft (Dave's comments and
%               major LSL revision): July 13, 1993.
% Third draft: (Dave's comments and addition of event classes):
%				     July 26, 1993.

% SERC Tech report: August  30, 1993.
% Paper submitted to:
%.

% Modified for compilation using TeXshop: September 16, 2003.

\usepackage{graphicx,latexsym}

%\input{psfig-dvips}
\setlength{\textwidth}{6.5in}
\setlength{\textheight}{9in}

\raggedbottom
\begin{document}

\def\figureloc{/Users/adityamathur/Documents/www/courses/BITSC461-fall03/listen-code/listen-report/listen-figures}
\baselineskip=18pt

\renewcommand{\arraystretch}{1.2}

\def\spec/{$\cal S$}
\def\mspec/{\mbox{$\cal M$}}


\def\contentsfigtables{\clearpage
\tableofcontents
\clearpage
\listoffigures
\listoftables
\clearpage
}

\hoffset=-0.75in\voffset=-1in\hsize=6.5in\vsize=9.75in

%\input /u/u32/apm/macros/apmtex.tex

\input /Users/adityamathur/Documents/www/courses/BITSC461-fall03/listen-code/macros/apmtex.tex

\clearpage

\def\mathur{Aditya P. Mathur}
\def\boardman{David B. Boardman}
\def\emailphone/{\hbox to \smallerthanhsize{\hfill apm@cs.purdue.edu~(317) 463-7822\hfill}}

\def\serc{{\vbox{
\hbox to \smallerthanhsize{\hfill Software Engineering
Research Center and Department of Computer Science\hfill}
\vskip 0.05in
\hbox to \smallerthanhsize{\hfill Purdue University\hfill}\\
vskip 0.05in
\hbox to \smallerthanhsize{\hfill W. Lafayette, IN 47907\hf\
ill}}}
}


\def\authors{\boardman\and\mathur}


\title{\bf Preliminary Report On\\
Design Rationale, Syntax, and Semantics of LSL:\\
A Specification Language for Program
Auralization\thanks{This
work was supported in part by an educational
supplement from the National Science
Foundation  No. CCR~9102311 and 9123502-CDA. The authors are with
Software Engineering
Research Center and Department of Computer Sciences,
Purdue University, W. Lafayette, IN 47907.
Aditya P.~Mathur can be contacted at (317)494-7822
or via email at apm@cs.purdue.edu. David Boardman
can  be contacted via email at boardman@cs.purdue.edu.}
}

\author{\authors}
\date{August 30, 1993}
\maketitle




\begin{abstract}


The need for specification of sound patterns to be played
during  program execution arises in contexts where 
program auralization is useful. We present a language 
named LSL ({\bf L}isten {\bf S}pecification 
{\bf L}anguage) designed for  
specifying program auralization. Specifications
written in LSL and included in the
program to be auralized are preprocessed by 
an LSL preprocessor. The preprocessed
program  when compiled
and executed generates MIDI
or voice  data  sent
through a MIDI interface to a synthesizer module,
or via audio channels, to an audio processor,
which  transforms the notes or voice  into audible sound.
LSL has the generality to specify 
auralization of a variety of
occurrences during program execution.
It  derives its broad
applicability from its few generic elements that 
when  adapted to any procedural programming
language, such as C, C$^{++}$, or Ada, enable the writing
and use of LSL specifications for auralizing 
sequential, parallel, or object oriented programs in
that language. We view  LSL as a useful tool
for building general purpose multimedia applications
and for research in program auralization.


\end{abstract}

\contentsfigtables

\section{Introduction}


The idea of using sound to understand program
behavior and to analyze data using sound has been 
reported by several researchers. By 
program and data  auralization we refer to,
respectively,  the activity of 
mapping aspects of program execution or properties of 
data to sound\footnote{We include human voice in this context.}.
A  survey of the use of sound
in understanding program behavior or analyzing
data collected from experiments or 
generated by a program is reported by 
Francioni and Jackson\,\cite{francioni-jackson-92}.
It is believed, and has been demonstrated 
in a few cases, that the use of sound can enhance 
program understanding. Yeung\,\cite{yeung-80} proposed
the use of sound to recognize multivariate analytical data.
Francioni and 
Jackson\,\cite{francioni-jackson-92} used program auralization to
understand the runtime behavior of
parallel programs. Brown and 
Hershberger\,\cite{brown-hershberger-92} auralized some of
their animations generated using the Zeus animation
system. In his doctoral dissertation,
Edwards\,\cite{edwards-89} built and evaluated a 
word processor with an audio interface for use by
visually handicapped users. Gaver\,\cite{gaver-86,gaver-89}
proposed the use of auditory icons for use
as a part of Apple's interface on the Macintosh 
machines.


It has been noted by researchers that in most
situations program output is visual. Programming
environments and applications 
developed so far  have attempted to use 
visual media to a great extent\,\cite{ambler-burnett-89}.
Though research in the  use of audio in 
workstations\,\cite{kamel-emami-eckert-90,ludwig-pincever-cohen-90}
has been on the rise, audio remains a distant second to 
visual media. However, with the availability of 
low cost audio devices such as synthesizer modules 
and sound digitizers and the provision of sound
generators in workstations, audio is within 
reach of most PC and workstation users. 


We have designed a language  that simplifies the task
of specifying which occurrences during
program execution are to
be auralized and how. The language is 
named Listen Specification Language,
abbreviated as LSL. Listen is the name of our project 
to investigate possible uses of sound in 
programming environments and software based 
applications. The need for LSL, its syntax, 
use, and current implementation status are
described in the remaining sections.
Section~\ref{need} outlines the need for
specifications of program auralizations and LSL.
Language requirements and the
underlying rationale  are  presented in 
Section~\ref{requirements}.
Essential features of LSL are presented in 
Section~\ref{syntax}. 
Examples illustrating the use of
LSL in various applications are given in Section~\ref{examples}.
Section~\ref{environment} explains how LSL can mesh with a
programming environment.
An implementation strategy for LSL is outlined in 
Section~\ref{implementation}. Current status of LSL 
implementation is reported  in Section~\ref{summary}.
A summary and   preliminary  
conclusions regarding
the utility of LSL appear in Section~\ref{summary}.

\section{The need for LSL}
\label{need}

Our research is concerned with the investigation
of  various uses of sound in programming 
environments. Some generic questions we 
ask and seek answers to
are listed below.

\begin{enumerate}

\item How useful is sound in debugging programs\,?

\item How useful is sound in program understanding\,?

\item Can program auralization be used to improve the quality
of simulations of various kinds such as
telephone networks, mechanical systems, 
and biological systems\,?


\item How can sound be used to make programming environments
usable by, in particular,
the visually handicapped with the same ease
as by  individuals without such a handicap\,?

\end{enumerate}
As  mentioned above, researchers have attempted
to obtain answers to  similar questions.
We  believe that
significant progress remains to be made before
sound is used widely  in various 
programming related tasks.

While investigating answers to the above questions,
we encountered a need for 
a general purpose mechanism to specify
the auralization of programs. In the absence of
such a mechanism, auralization is done 
by editing the source code and adding 
calls to library procedures that
generate sound.  For example, suppose that
a parallel program to be executed on
a machine with 1000 processors is to be auralized. 
The auralization should be such that whenever 
an even numbered processor sends data,
a sound with timbre characteristic $T_1$ is  generated
and whenever an odd numbered processor
sends data a sound with timbre $T_2$ is generated.
When so auralized, a listener will be able to distinguish 
between data sends by even and odd numbered processors.
Assuming that one has access to a library routine 
{\tt gen\_sound} to generate sound with
a specific timbre, one can edit the program
and add calls with appropriate parameters to 
{\tt gen\_sound} 
at all places where a data send occurs.
This could become an inconvenient task
if communication statements are distributed 
over many procedures in a large program spread over 
several source files. The task could be simplified 
if one could formally specify the auralization mentioned above.
Such a specification and the program to be auralized could be 
preprocessed automatically
to generate a source program with code
inserted for the desired auralization.
Any change in auralization will then require a change only
to the specification and not to the source
of the parallel program.

As another example, consider the task of debugging 
a distributed system that controls various 
functions of an automobile. It is desired to auralize 
the program  so that each call   to
a control procedure, e.g. to
procedure  gear\_change, is identified by a suitable sound.
As above, one may edit the source to achieve the 
desired effect. Another alternative is to
formally specify the above auralization requirement
and use  preprocessing to add the necessary code.

A more difficult situation arises when the value of
one or more variables is
to be monitored. Suppose that we want to monitor 
the value of a variable $x$ in a 
program. Whenever this variable exceeds a predetermined
value a sound should be generated. Again, one may use an editor to
add a few lines of
code at all places in the program where $x$ has been defined.
An easier alternative appears to be
to formalize the above auralization  and preprocess it.
During further development of the software if one decides to 
monitor other variables, the specification can be 
easily modified
as compared to editing the code which might require 
careful passes through various source files.


Examples such as the ones 
above led us to consider 
designing a language for specifying auralizations.
We note the  pioneering work 
in the design of languages for 
music\,\cite{langston-90,thompson-90}. 
The main purpose of these languages was to specify 
music.  In their present form,
these  languages are not suited
to the auralization  tasks mentioned above.

\section{Basic definitions and  LSL requirements}
\label{requirements}

Based on the perceived need for a specification
language, we set forth the following idealized requirements
for LSL. 

\begin{enumerate}

\item {\em Generality\/}: It should be possible to specify
any   auralization  using LSL.

\item {\em Language independence\/}: It should be possible to
use LSL with the commonly used programming languages such
as C, C$^{++}$, Ada, Pascal, and Fortran.


\end{enumerate}
Below we define  basic terms and 
introduce concepts that help us formalize the 
above goals. Our formalization brings reality
to the above requirements. LSL satisfies the
requirements  with respect to this formalization.

\subsection{ASPECs and realizations}

To be able to design a language that
can specify all
possible auralizations, we need a quantification of
two domains. 
Let  $E$ be the domain of  all those 
occurrences  during the execution of any program
that one may wish to
auralize. The nature of such
occurrences
is discussed below. Let $S$ be the domain  of all possible sound
patterns that may be associated  with each element of 
$E$. A {\em mapping} from $E$
to $S$ is an association of  sound patterns in $S$
to occurrences in  $E$.  
Such a mapping is specified as a set of pairs ($e, s$)
where $e\in E$ and $s\in S$. 
The term {\em program auralization} for
a given program $P$ 
refers to the set \{$(e_1,s_1), (e_2, s_2), \ldots, 
(e_n, s_n)$\}, where each $(e_i, s_i), 1\le i\le n$
is an association of an occurrence to a sound pattern.
A {\em language} $L$ for program auralization
is a notation to specify any   such mapping
for any program. A mapping  specified  using $L$ is 
referred to as {\em auralization specification} 
abbreviated as ASPEC. We assume that specifications
are always written with reference to a given, though
arbitrary,  program in some programming language.
Figure~\ref{domain-mapping} illustrates 
this view of program auralization. Note
that an ASPEC is a many-to-many mapping.


\begin{figure}
\vskip-1.5in
\centerline{\includegraphics[width=5.5in,height=6in]{\figureloc/domain-view}}
\vskip-2.75in\caption{A domain-based view of program auralization.
The ASPEC in this example consists of 
four occurrence and  sound pattern pairs as shown.
$(e_i, s_i)$ are elements of the specified mapping.}
\label{domain-mapping}
\end{figure}




Let ($e, s$) be an element of
an ASPEC for  program $P$.
During the execution of $P$ if each occurrence $e$ 
is identified by a sound pattern  $s$,
we say that the pair  ($e, s$) 
has been {\em realized}. An ASPEC
for program $P$ is considered
realized if all its elements  are realized
for all executions of $P$.
We say that an implementation of $L$ 
for programs in a given programming language
$PL$ is correct if each ASPEC, written in
$L$, for any  program $P$, written in $PL$ is realized.



\subsection{Occurrence space characterization}


Ideally, we would like to be able to specify 
any auralization. To do so,
we need to  characterize   the space of
all possible occurrences that might arise
during  program execution. Towards this end we 
select a  three-dimensional
space using the orthogonal notions
of position, data, 
and time.  {\em Position} refers  to any 
identifiable point in a program. For example, in a C program,
beginning of a function call, 
end of a function return, start of a while-loop,
start of a while-loop body, and start of a condition,
are all  positions. In general,  an  identifiable
point is any point in the program at which an
executable  syntactic
entity  begins or ends. This implies that  a position
cannot be in the middle of an identifier or a constant.
In terms of a parse tree for a given program, 
any node of the parse tree denotes a position.
For example, the subscripted  dot ($\bullet_i$) denotes 
seven possible positions in  the following assignment:
$\bullet_1 X\bullet_2 = \bullet_3 X\bullet_4 +\bullet_5 3\bullet_6/
\bullet_7$ 2.

Data in a program refers to  constants
allowed in the language of the program being auralized and
the values of program variables. A 
{\em data relationship} is 
an  expression consisting of
constants,  variables, and function calls.  Time refers
to the execution time
of the program. It is measured in units
dependent on the system responsible for
the execution of the auralized  program.
In a  heterogeneous system,
time is measured in units 
agreed upon by all  elements of the system.


As shown  in Figure~\ref{occurrence-space},
a three dimensional
space is used for specifying occurrences in LSL.
Two kinds of occurrences are distinguished:  events and
activities.
LSL allows  an  arbitrary combination of 
data relationships,
positions, and time  to specify an 
event or an activity associated with 
program execution. 



\begin{figure}
\hskip6in\centerline{\includegraphics[width=4.5in,height=5.75in]{\figureloc/event-space}}
\caption{Occurrence space characterization in LSL.}
\label{occurrence-space}
\end{figure}


\subsection{Sound space characterization}


The sound space is characterized by
sound patterns comprised of  notes,  durations,
play styles, and instruments.
Notes of arbitrary durations
can be combined to form  sound patterns.
Each note can be associated with one of several play 
styles and with an arbitrary instrument.
For example, a note can be played staccato 
on a piano with a specified  volume.
Combining notes in various ways gives rise to
a domain  consisting of an infinity of sound
patterns. Digitized sound, such as human voice, 
is considered  a sound pattern.

\subsection{Programming language independence}

The second requirement stated above is significant 
as we  want LSL to be usable 
by programmers regardless of their preference for
one or the other programming language. Adherence to
this requirement has produced a language which 
in the strict sense should be considered as a meta-language.
One can therefore adapt LSL to specific 
programming languages as described later.
In all   examples below, we
use a C\,\cite{c-book} adaptation of LSL termed LSL/C.


\section{Features and syntax of LSL}
\label{syntax}

The salient features of LSL are reviewed next.
Details of LSL  syntax  and semantics 
appear in the Appendix.
An LSL program is known as a specification.
Each specification is composed of one or more
specification modules.  
Each specification module is composed of 
zero or more  specification definitions
and one main specification.  A specification module,
a specification definition, and 
a main specification 
are  analogous to, respectively, 
a module, a procedure, and a module body in a 
Modula-2\,\cite{sethi-89}
program. 
As an example of LSL specification structure consider the
specification listed in Figure~\ref{lsl-spec-structure}.
It begins with  \lslabegin/ and ends with \lslaend/.
Each module begins with a header identified
by the  \lslspecmod/ keyword followed by the module name
such as spec\_module\_1, spec\_module\_2, and so on in this example.
A module header  is followed by applicability constraints which
specify  parts of the program to which
the specifications  are to be applied.
Then come declarations of variables used in this module 
followed by zero or more specification definitions 
such as spec\_def\_1, spec\_def\_2, and so on.
Global variables are shared between 
various modules. Variables and specification
definitions to be exported (imported) are listed in
the \lslexport/ (\lslimport/) declaration.
Variables declared in  the program
being auralized can also
be used inside  LSL specifications. These are known as
{\em external} variables.


\begin{figure}
\vskip0.001in
\begin{tabbing}
\programtab\kill
\lslabegin/\\
\>\lslspecmod/  myprog\_auralize\\[0.1in]
\>/* \parbox[t]{5in}{
          This module contains specifications to auralize
          myprog procedure.} */\\[0.1in]
\> /* \parbox[t]{5in}{
          Applicability constraints, if any, come here.} */\\[0.1in]
\> /* \parbox[t]{5in}{Declarations for variables
      global  and external to this module .} */\\[0.1in]
\> \lslspecdef/  specdef\_1 (parameters);\\
\>\> /* Declarations of parameters, local variables, and 
functions. */\\
\>\>\lslbegin/~specdef\_1\\
\> \>\vdots\\
\>\> \lslsend/~specdef\_1;\\[0.1in]
\> {\tt  spec-def} spec\_def\_2 (parameters);\\
\>\> /* Declarations of parameters, local variables,
and functions. */\\
\>\>\lslbegin/~specdef\_2\\
\>\> \vdots\\
\>\> \lslend/~specdef\_2;\\[0.1in]
\>\> \vdots\\
\> \lslspecdef/ specdef\_n (parameters);\\
\>\> /* Declarations of parameters, local variables, and
functions. */\\
\>\>\lslbegin/~specdef\_n\\
\>\> \vdots\\
\>\> \lslend/~specdef\_n;\\[0.1in]
\> \lslbegin/ myprog\_auralize;\\
\> /* Specifications for module myprog\_auralize. */\\
\>\> \vdots\\
\> \lslend/ myprog\_auralize;\\[0.1in]
\> /* Other module specifications. */\\
\>\vdots\\
\lslaend/.
\end{tabbing}
\caption{Structure of an LSL specification containing 
one module..}
\label{lsl-spec-structure}
\end{figure}


\subsection{Constants, variables, and types}
\label{constants}

LSL is a typed language. It contains constants,
variables, and types just as several other 
languages do. An identifier name is a sequence 
of one or more characters
consisting of upper or lower case letters, digits,  
and 
the underscore (\_). The first character
in an identifier must be a letter or an underscore.
Upper and lower case letters are treated as being 
different.  Variables and constants can be assigned
arbitrary names.
Values likely to
arise during program auralization are grouped together
into primitive  types. Table~\ref{primitive-types} 
lists the primitive types available in LSL.
Values of type \lslnote/
and  \lslpattern/ are enclosed in quotes
to distinguish them from variable names.
A note is specified by indicating its pitch
e.g. ``E4b'' indicates E-flat  above the middle C on a piano
keyboard. Attributes listed in Table~\ref{attributes}
 can be added to a note
separated by a colon (:).  
A pattern is a sequence of notes  and 
voices\footnote{Throughout this work data of type  ``voice'' 
refers to digitized sound. Thus, for example, 
both  digitized voice and  digitized
guitar sound  are characterized as  
voice data.} played in the specified sequence. 
A sequence of notes within a pattern can be 
enclosed in parentheses to indicate a blocked chord
also referred to as a chord pattern. 
A  variable name
can be  used within a pattern by preceding it with a dot.
For example, if the identifier {\em cmajor} denotes
a chord pattern, 
then  p:= ``.cmajor~E5'' denotes a pattern consisting of
the value of {\em cmajor} followed by the note E5. 
Just as  values could
be printed or displayed in computer programs,
we say that values 
of type \lslnote/ and  \lslpattern/ could
be {\em played back} during the execution of an auralized
program.


The set of key signatures constitutes
the type  \lslksig/. Pre- or user-defined functions
are used to manipulate values  of type \lslksig/.
Constants of type \lslksig/ are enclosed inside
double quotes and can be assigned to
variables of the same type. A key signature could
be predefined or user defined. 
A predefined key signature consists
of two parts: a key name and a modifier.
Examples of key names are Eb (denoting E flat) and
C\# (denoting C sharp). Modifiers could be
major, minor (same as harmonic minor),
lydian, ionian (same as major), mixolydian,
dorian, aeolian,   phrygian,  and locrian. 
Thus, for example, ``C\#:minor'' and ``E:phrygian''
are valid key signatures. A user defined key signature
is any enumeration of notes. For
example, ``C D Eb G A'' is a  key
signature of a  pentatonic scale.


The set of time signatures constitutes the type 
\lsltsig/. Constants of type \lsltsig/ are 
enclosed within parentheses. A time signature consists
of two parts: the beat structure and the note that
takes one beat. For example, (4:4) is a simple time signature
indicating 4 beats to a measure with a quarter note
of one beat in duration. A more complex
time signature is (3+2+2:8) which indicates a beat
structure of 3+2+2 with  an eigth note
taking one beat. A beat structure 
such as 3+2+2 indicates that
the first measure is of 3 beats in duration,
followed by two   measures each of 2 beats duration,
followed by a measure of 3 beats and so on.
Time signatures can be assigned to variables
of the same type and manipulated
by functions.


Type \lslfile/ is the set of file names.
A filename is specified by enclosing the name within
double quotes. Thus, ``your\_name\_please.v'' can serve as
a file name. The use of file names
is illustrated through LSL examples below.  Note that we use 
a string of characters enclosed within
double quotes in
a variety of contexts. It is the context that
unambiguously determines the type of a string. 

\begin{table}
\centering
\caption{Primitive types in LSL.}
\label{primitive-types}
\begin{tabular}{|l|l|p{3.7in}|}
\mc{3}{l}{\tablestrut}\\
\hline
\mc{1}{|c|}{\bf Keyword}&
\mc{1}{c|}{\bf Sample values}&\mc{1}{p{4in}|}{\bf Description}\\
\hline
\tablestrut&&\\
\lslint/&$-20$ or 76&Set of integers.\\
\lslnote/& ``E4b''&Set of notes; not all of
these may be played back in a particular implementation.
A subset of the notes is labelled  starting at A0 and 
going up to  C8 as 
found on an 88-key piano keyboard. These 88 notes
correspond to integer values of
0 to 87. Other notes values  may be obtained
using the predefined function {\em inttonote}.
Other predefined functions are listed in Table~\ref{predef-functions}.\\
&R&A rest is treated as a silent note with duration
specified by a duration attribute.\\
\lsltsig/&(3:8) or (3+2+2:4)&Set of pairs of values
denoting a time signature.
The first element
in the pair specifies the beat structure i.e. the
number of
beats per measure. The second element
is the note value that corresponds to
one beat. The beat structure could
be complex as explained in the text.\\
\lslksig/&\vtop{\hbox{``Eb:minor''}\vskip0.1in
                \hbox{``(C D E F\# G A B)''}}&Set of k-tuples of 
pitch values. The set may be specified 
using abbreviations such as Eb:minor to indicate the key
of Eb minor or by enumerating all  pitches regardless of their
specific position on a keyboard as  in the example.\\
\lslpattern/& ``G3E3C4''&Set of note and/or chord
patterns  consisting of zero or more notes or chords.\\
\lslvoice/&$^{\dag}$&Set of digitized voice patterns. A variable of
this type can be set to point to a memory or disk file
containing a digitized voice pattern.\\
\lslfile/& ``done-voice.v''&Set of file names.
File extensions are interpreted. .c is for C program files,
.v for digitized voice files.\\
\hline
\mc{3}{l}{$^{\dag}$ Any digitized sound in a suitable format, e.g.
AIFF\,\cite{audio-formats}.}
\end{tabular}
\end{table}




A special type \lslvoice/ has been included to 
play  digitized voice during program execution.
We have assumed that voice will be
digitized using  a suitable digitizer, e.g. the
Audiomedia~II\,\cite{audiomedia} card from Digidesign, and 
stored as a sample
in a file. It is this sample that  becomes a constant and
can be assigned to a variable of
type \lslvoice/. Voice can  be used in 
note patterns by specifying variables of type \lslvoice/.


\begin{table}
\centering
\caption{Attributes in LSL.}
\label{attributes}
\begin{tabular}{|l|l|p{4in}|}
\mc{3}{l}{\tablestrut}\\
\hline
\mc{1}{|c|}{\bf Code}&\mc{1}{c|}{\bf Applicability}&
\mc{1}{p{4in}|}{\bf Description}\\
\hline
\tablestrut&&\\
{\tt f}&Note&Indicates  a  full note .\\
{\tt h}&Note&Indicates  a half note.\\
{\tt q}&Note&Indicates  quarter note.\\
{\tt e}&Note&Indicates  eight note.\\
{\tt s}&Note&Indicates  sixteenth note.\\
{\tt chan}&Note, pattern& Specifies the 
MIDI$^{\dag}$ channel on 
which to play.\\
\lslplay/& Note&Indicates one or more play styles.\\
\lslinst/&Note,  pattern&Specifies which instrument is
to play.\\
\lslmm/&Pattern&Metronome setting. This is applicable 
only to patterns. Notes  not
part of a pattern are 
played  for a duration determined by global 
metronome setting. A metronome setting specified
for a pattern takes priority over any global setting
only while this pattern is played.\\
\lslptime/&Note, pattern& Specifies the exact time in
seconds to play the note or a pattern. \\
\hline
\mc{3}{l}{\parbox[t]{4.5in}{$^{\dag}$ MIDI is
an acronym for Musical Instrument Digital Interface.}}
\end{tabular}
\end{table}



Variables must be  declared before use. 
The following declaration declares 
body\_begin and body\_end   to be of type \lslnote/, 
loop\_begin, loop\_end,
and measure to be  of type \lslpattern/.


\begin{tabbing}
\programtab
\>\lslvar/\\
\>\> body\_begin, body\_end: \lslnote/;\\
\>\>loop\_begin, loop\_end, measure: \lslpattern/;\\
\end{tabbing}


\subsection*{Note and rest values}

Attributes aid in specifying various properties of notes
and patterns. Perhaps the most common 
attribute of a note or a chord sequence is its duration. 
For example, ``E4:q'' denotes a quarter note
whose duration  will be determined by 
the time signature and the metronome value.
The duration attributes can be multiplied or added
to get dotted quarter note, and other fractions of 
note values. For example, (hq) read as {\em half
of quarter}  denotes an eight note, 
(hhq)  read as {\em half of half of a quarter}
denotes a sixteenth note. Table~\ref{note-values}
lists sample note values and the
corresponding attribute combinations. 
Various rests could be obtained using
the attribute combinations shown
in Table~\ref{note-values} with the letter {\tt R}.
For example, ``R:(hq+hhq)'' denotes a dotted 
eight rest.



Duration can be specified for 
a chord by a single duration attribute.
For example, ``(C4E4G4):q'' denotes a chord consisting of
three quarter notes.
Notes and chords for which the duration
is not specified explicitly, as in ``E4'',
are played for a duration determined by implementation
dependent default durations (See Table~\ref{default-values}
for various defaults.). 

\begin{table}
\centering
\caption{Sample  note values 
using LSL duration attributes.}
\label{note-values}
\begin{tabular}{|l|l|}
\mc{2}{l}{\tablestrut}\\
\hline
\mc{1}{|c|}{\bf Note value}&
\mc{1}{c|}{\hbox{\vtop{\hbox{\bf Attribute}\vskip0.05in
                       \hbox{\bf  combinations}\vskip0.05in}}}\\
\hline
\tablestrut&\\
Quarter note&{\tt q} or {\tt hh}\\
Eight note&{\tt hq}\\
Sixteenth note&{\tt hhq} or {\tt qq}\\
Thirty second note&{\tt hhhq}\\
Sixtyfourth note&{\tt hhhhq} or {\tt ss}\\
Dotted half note&{\tt h+q}\\
Dotted quarter note&{\tt q+hq}\\
Dotted eigth note&{\tt hq+hhq}\\
\hline
\end{tabular}
\end{table}


\begin{table}
\centering
\caption{Default values of run time parameters.}
\label{default-values}
\begin{tabular}{|l|l|}
\mc{2}{c}{\tablestrut}\\
\hline
\mc{1}{|c}{\bf Item}& \mc{1}{|c|}{\bf Default value}\\
\hline
\tablestrut&\\
Metronome& \lslq/=120\\
Key signature& C major\\
Time signature&(4:4)\\
Channel&1\\
Instrument code&1\\
Note duration&\lslq/\\
Play mode&\lsldiscrete/ for  \lslnotify/\\
         &\lsldiscrete/ for  \lsltrack/\\
         &\lslcontinuous/ for \lslatrack/\\
Pitch& ``C4''\\
\hline
\end{tabular}
\end{table}


\subsection*{Type constructor}

Values of primitive  types  can be combined together into
an array. The following sequence
declares an array of measures, each measure being
a pattern. Elements of an array can be accessed by subscripting.
Thus tclef\_staff\,[k+1] refers to the
(k+1)th element of tclef\_staff which is of type 
pattern. 




\begin{tabbing}
\programtab
\>\lslconst/\\
\>\> scoresize = 25;\\[0.1in]
\>\lslvar/\\
\>\> tclef\_staff: \lslarray/ 
[1..scoresize]~\lslof/~\lslpattern/;
\end{tabbing}



\subsection{Sound pattern specification}

The \lslplay/ command is used to specify
what  sounds are to be generated when some part
of a program is executed. The general syntax\footnote{Syntactic
entities are enclosed in $<$ and $>$.
Optional entities are enclosed in \{ and \}.
For a complete syntax of LSL see  Appendix.} of \lslplay/ is:

\lslplay/ $<$playlist$>$ 

\noindent where $<$playlist$>$ is a list consisting of
one or more notes 
and  patterns specified using constants, variables,
and function calls. Key and time  signatures are some
of the parameters that may
be specified.
Elements of $<$playlist$>$ can be separated by a comma (,)
or a parallel ($\parallel$) sign. An example of
\lslplay/  command  appears below.

\lslplay/ (loop\_background~$\parallel$~(func\_call, 
no\_parameters))
 \lslwith/ \lslmm/ \lslq/ =120, \lslinst/ = ``piano'';

\noindent The above \lslplay/ when executed will
play the sound associated with 
the variable loop\_background together with a sequence
of sounds denoted by 
the variables func\_call and no\_parameters. 
Default  key and time signatures will be used.
The  metronome will be set to play 120 quarter notes
per minute and the notes will be played 
using a piano sound.



\subsection{Event notification}

A  useful
characteristic of LSL is its  ability to 
specify  events to be auralized. A programmer  may formulate an
event to be auralized  in terms of the application. 
However, such a specification is 
translated in terms of program position,
data, and time as described earlier.  For example, in an 
automobile simulator,  events such as 
{\em gear change, speed set, resume cruise},
and {\em  oil check} may be candidates for auralization. 
Suppose that the  occurrence of
these events is indicated by calls to procedures 
that correspond to the simulation of
an activity such as {\em gear change}. It is
these procedure calls that serve as event indicators to
LSL. Thus, for example, such a call to the 
{\em gear\_change} procedure could be mapped to sound
using an LSL specification. 

Event specification is  achieved
by the \lslnotify/ command. \lslnotify/ is a generic
command and can be adapted to a variety of
procedural languages. In examples below
we assume that programs being auralized have been coded in C.
The syntax of \lslnotify/ appears  below:

\vskip0.1in\indent \lslnotify/ \{$<$all-selective$>$\} 
\{$<$label-parameter$>$\}
$<$event-specifier$>$ \{$<$sound\_specifier$>$\}\\
\indent\{$<$scope-specifier$>$\}
\vskip0.1in


\noindent $<$all-selective$>$ specifies which subset of
events selected
by a \lslnotify/ are to be  auralized. Possible event codes
are \lslall/ and \lslselective/. If \lslselective/ is
used, one or more labels  must be specified to
indicate which events are to be selected. 
$<$event-specifier$>$   specifies  one or more
events  to be notified aurally.

There are five ways to specify an event.
One may specify a general syntactic entity,
a special syntactic entity, an assertion,
a relative timed event,
and any  combination of the above four. 
Relative timed events are discussed in 
Section~\ref{timed-events};  other  methods are described
below.
Table~\ref{event-specifiers} lists all event codes 
in LSL/C.
For example,  {\em while-statement-enter}  is
an event  specifier; the  corresponding event
occurs once each time a  while statement is executed.
The start and termination of
program execution serve as events.

The expression ($x<y$) serves as a special syntactic entity.
The associated event occurs whenever
the expression
(x$<$y) is executed. An assertion such as $(x+y)>(p+q)$
also specifies an event which occurs whenever the assertion 
evaluates to false.
If $e_1$ and $e_2$ are two events specified using any of
the above approaches, then ($e_1$~\lsland/~$e_2$)
and ($e_1$~\lslor/~$e_2$) are also events.

The scope of a \lslnotify/ may be
restricted using the $<$scope-specifier$>$. 
In LSL/C, the scope can be restricted to one
or more  functions or  files. For example,
if an assertion is to be checked only
inside function {\em sort}, one may 
suitably restrict the scope to that function.
Labels can be used in conjunction with scope restrictions
to specify arbitrarily small regions in a program. 


The sound specifier is a variable name, constant,
or a function call   that
specifies the intended auralization
of the selected events. Sample \lslnotify/ 
commands appear below.


\begin{enumerate}

\item\label{all-rule}
\lslnotify/ \lslall/ \lslrule/=while\_loop\_body\_begin 
\lslusing/  body\_begin;

\lslnotify/ \lslall/ \lslrule/= while\_loop\_body\_end 
\lslusing/   body\_end;

\item\label{select-rule}
\lslnotify/ \lslselective/ \lsllabel/~=~special\_loop
\lslrule/=while\_loop\_body\_begin 
\lslusing/  body\_begin;

\lslnotify/ \lslselective/ \lsllabel/~=~special\_loop 
\lslrule/=while\_loop\_body\_end \lslusing/  body\_end;


\item \label{instance-rule}
\lslnotify/ \lslall/ \lslinstance/= ``++count'' \lsland/ 
``search(x)'' \lslusing/  count\_or\_search \lslin/ 
\lslfunc/ = ``search'',  ``report'';


\item \label{all-cond}
\lslnotify/ \lslall/ \lslassertion/~=~(x$<$y~$\parallel$~p$\ge$q)
\lslusing/  assertion\_failed;

\item \label{compound-event}
\lslnotify/ \lslall/ \lslrule/ = conditional\_expression
\lsland/ \lslassertion/ = odd(x)
\lslusing/  cond\_sound  \lslin/ 
\lslfilename/ = ``myfile.c'';






\end{enumerate}

Example~\ref{all-rule} above specifies two event types,
namely the beginning and end of a while-loop body
using two general purpose syntactic specifiers.
It also indicates that all positions in
the program  where such events could
occur are to be auralized. Thus, a C program auralized 
using the above \lslnotify/ will
generate the sound corresponding to the variables
body\_begin and
body\_end, respectively,
whenever the beginning and end of  a while-loop body
are executed. 

Example~\ref{select-rule}
is the same as Example~\ref{all-rule} except that 
the event selection is selective.  Thus, any  loop
body  labelled  by {\em special\_loop} will be auralized.
Any syntactic entity can be labelled in
the program being auralized by placing an LSL 
\lsllabel/ command in front of that entity
as described in Section~\ref{embedding}.

Example~\ref{instance-rule} specifies the execution
of the statements {\em ++count} and {\em search(x)} as the events.
When any of these two events occur,  {\em count\_or\_search}
is played. However, these events are to be
recognized only
inside functions {\em search} and {\em report}.

Example~\ref{all-cond}	 above specifies an event
which occurs whenever the condition
(x$<$y~$\parallel$~p$\ge$q) is not satisfied.
Note that this condition is based on variables
in the program being auralized. When this condition is
not satisfied, {\em assertion\_failed} is to be played.
Example~\ref{compound-event}
shows how to specify the auralization of
all conditional expressions that occur in
file {\em myfile.c} only when condition {\em odd(x)}  is
not  satisfied.


The \lslall/ and \lslselective/ tags can 
restrict any event selection. Multiple labels are used 
within one \lslnotify/ command as in the following.

\vskip0.1in
\lslnotify/ \lslselective/ \lsllabel/~=~loop\_1, loop\_2 
\lslrule/=while\_loop\_body\_begin \lslusing/ body\_begin;

\lslnotify/ \lslselective/ \lsllabel/~=~special\_loop
\lslrule/= while\_loop\_body\_end \lslusing/  body\_end;
\vskip0.1in

\noindent The above \lslnotify/ commands specify the same type
of events as in Example~\ref{select-rule} except that
loop body begins and ends that contain any one of the two
labels {\em loop\_1}  and {\em loop\_2} will be selected for
auralization.

\begin{table}
\centering
\caption{Keywords and codes for LSL 
event specifiers in C.}
\label{event-specifiers}
\begin{tabular}{|l|l|l|l|l|}
\mc{5}{l}{\tablestrut}\\
\hline
\mc{1}{|c}{\bf Category}&\mc{1}{|c}{\bf Event specifier}&
\mc{1}{|c|}{\bf Code$^{\dag}$}&
{\bf Event specifier}&
\mc{1}{|c|}{\bf Code$^{\dag}$}\\
\hline
\tablestrut&&&&\\
Program&	start&                  start			
&               end&end\\
Expression&	variable&		var
&		assignment\_expression&	aex\\
&		conditional\_expression&	cex&&\\
Iteration&	iteration\_statement&	ist
&		iteration\_body\_begin&	ibb\\
&		iteration\_body\_end&	ibe
&		while\_statement\_enter&	wse\\
&		while\_statement\_exit&	wsx
&		do\_while&		dow\\
&		for\_statement\_enter&	fre
&		for\_statement\_exit&	frx\\
&		while\_body\_begin&	wbb
&		while\_body\_end&		wbe\\
&		for\_body\_begin&		fbb
&		for\_body\_end&		fbe\\
&		do\_while\_body\_begin&	dbb
&		do\_while\_body\_end&	dbe\\
Jump&		jump\_statement&		jmp
&		continue\_statement&	cst\\
&		break\_statement&	bst	
&		return\_statement&	rst\\
&		goto\_statement&		gst&&\\	
Selection&	selection\_statement&	sst
&		if\_statement&		ist\\
&		if\_then\_part&		itp
&		if\_else\_part&		iep\\
&		switch\_statement&	sst
&		switch\_body\_begin&	sbb\\
&		switch\_body\_end&	sbe&&\\
Functions&	function\_call&		fnc
&		function\_entry&		fne\\
&		function\_return&	fnr&&\\
\hline
\mc{5}{p{5in}}{$^{\dag}$ Event specifiers and their
abbreviated codes 
can be used interchangeably to specify a rule 
in a \lslnotify/ statement.}
\end{tabular}
\end{table}


\begin{table}
\centering
\caption{Predefined functions  in LSL.}
\label{predef-functions}
\begin{tabular}{|l|l|p{3.5in}|}
\mc{3}{c}{\tablestrut}\\
\hline
\mc{1}{|c}{{\bf Function name}}&\mc{1}{|c}{{\bf Mapping}}&
\mc{1}{|p{3.5in}|}{{\bf Description}}\\
\tablestrut&&\\
\hline
intton&\mymap/{int}{note}&Converts an integer 
to a note. Integers in the inclusive range 0
to 87 get converted to notes A0 to C8.\\
ntoint&\mymap/{note}{int}&Converts a note 
to an integer. \\
nabove&\mymap/{\mytimes/{note}{ksig}}{note}&Returns
the note above the input in the given scale.  \\
nbelow&\mymap/{\mytimes/{note}{ksig}}{note}&Returns
the  note below  the input in the given scale.  \\
naboveh&\mymap/{note}{note}&Returns
the  note one half step above the input.\\
nbelowh&\mymap/{note}{note}&Returns
the  note one half step below the input.\\
circlen&\mymap/{ksig}{ksig}&Returns
the next key signature in the circle
of fifths. Valid only for
predefined key signatures.\\
circlep&\mymap/{ksig}{ksig}&
Returns the previous key signature in the circle
of fifths. Valid only for
predefined key signatures.\\
sectotick&\mymap/{int}{int}&Converts seconds to system dependent
ticks.\\
\hline
\end{tabular}
\end{table}


\subsection{Data tracking}

Event notification 
consists of specifying one or more events 
and  reporting them
aurally during program execution.
There are applications wherein  changes to  values 
of variables need to be monitored.  It is certainly possible
to specify assignments to such variables as events
and then report the execution of
these assignments aurally. Such reporting is, however, 
independent of
the data  being assigned. To obtain data
dependent auralization,  LSL provides the  \lsltrack/
command. The syntax of \lsltrack/
appears  below.


\vskip0.1in\indent\lsltrack/ $<$track-id-list$>$ 
$<$sound-specifier$>$
\{$<$mode-specifier$>$\} \\
\indent \hskip0.1in \{$<$start-event-spec$>$\} 
\{$<$term-event-spec$>$\}
\vskip0.1in

\noindent Using \lsltrack/, one or more variables
can be tracked.   For the variable to be tracked,
an initial value   can optionally
be specified using the \lslinitval/ keyword.
The type of the initial value must match that
of the variable to be tracked.
The initial value may also be captured immediately after 
the execution of an assignment labelled using an LSL
label.  

As in \lslnotify/, a 
$<$sound-specifier$>$ specifies the sound 
to be used while tracking the
variables.  Here we introduce another
method for specifying sounds which is particularly useful
in conjunction with the \lsltrack/ command.
A sound pattern  whose characteristics depend 
on program generated  data will be referred to
as a {\em Value Dependent Aural Pattern}
and abbreviated as VDAP.  The \lslusing/ clause
in the $<$sound-specifier$>$
specifies the name of the function,
say $f$,
that emits a VDAP based on  variables being
tracked.  $f$ is a language dependent function
containing  LSL commands for
auralization. Thus,  in LSL/C,  $f$ is 
a valid C function interspersed with
LSL commands. $f$  is 
executed after each assignment to the variable
being tracked.

Tracking may  be carried out in continuous or 
discrete mode.
 In continuous mode, 
tracking begins at the start of program
execution, unless specified otherwise. A note pattern
is emitted continuously until 
there is a change in the value of the variable being monitored.
When the value changes, a newly computed note pattern
is emitted continuously.
In discrete mode, a note pattern is emitted once whenever
the tracked variable changes its value.
In discrete mode tracking begins 
the first time the tracked variable changes its value 
after program execution. 


Tracking can also be controlled using 
$<$start-event$>$ and $<$term-event$>$. 
Start and terminating  events
are  specified, respectively,  using the  \lslwhen/ 
and \lsluntil/ clauses. 
A few examples of \lsltrack/ use appear below.

\begin{enumerate}

\item \label{track-simple}
\lsltrack/ $speed$; will track variable {\em speed}
 using an initial value
of 0 and default sound parameters such as note pitch
and volume.

\item \label{track-init}
\lsltrack/ {\em crash} \lslinitval/=\lslfalse/; 
will track  {\em crash} assuming an
initial value of \lslfalse/.

\item \label{track-capture}
\lsltrack/ $x$ \lslcapture/=x\_reset; will track 
$x$ after capturing its initial value at the
assignment
labelled by the LSL  label {\em x\_reset}


\item \label{track-vdap}
\lsltrack/ {\em mouse} \lsland/  {\em color} \lslusing/ 
{\em color\_mouse\_melody} (\&mouseval, \&colorval) ;
will track variables {\em mouse} and {\em color}
using a user defined
function named {\em color\_mouse\_melody} with two parameters.



\item \label{track-after}
\lsltrack/ {\em speed\/}  \lslwhen/ speed$>$65 
\lsluntil/ x$\le$65
\lslmode/=\lslcontinuous/;
will begin tracking  {\em speed}  whenever its
value exceeds 65 and will stop tracking it
immediately after its
value becomes equal to or less than 65.
Tracking will resume if  the start event occurs
again. The discreet mode can be used
to avoid resumption of tracking of {\em speed}.



\end{enumerate}

\subsection{Activity monitoring}
\label{activity-monitoring}

An activity is a sequence of actions
between two events. An activity begins 
at the occurrence of  an event and ends at 
occurrence of a later event.
As mentioned earlier, 
start and termination  of program execution
are considered as events. LSL allows specification of
tracking arbitrary activities using  the \lslatrack/ command
given below.

\vskip0.1in\indent\lslatrack/ \{ \lslwhen/ $<$event-specifier$>$ \}
\{ \lsluntil/ $<$event-specifier$>$ \}\\
\indent \hskip0.1in  $<$sound-specifier$>$
\{$<$mode-specifier$>$\}
\vskip0.1in

\noindent $<$event-specifier$>$, $<$sound-specifier$>$,
and $<$mode-specifier$>$
have the same meaning as in the \lsltrack/
command.  Tracking
begins when the event specified immediately
following \lslwhen/ occurs (start event)
and  stops when
the event specified following
\lsluntil/ occurs (terminating event). 
If the start event is omitted, 
tracking begins at the start of program execution.
Tracking ends at program termination if
the terminating event is  omitted.
If both the start and terminating events are omitted then
the entire program execution is tracked.
In \lslcontinuous/ mode, an activity begins whenever 
the starting
event occurs and terminates at the terminating event.
In the \lsldiscrete/ mode, an activity occurs 
as above but does not resume.
Using the start and terminating
events one may specify a variety of
activity tracking patterns as shown in 
Figure~\ref{activity-patterns}.


\begin{figure}
\centerline{\includegraphics[width=4.5in,height=6.0in]{\figureloc/activity-patterns}}
\vskip-2.5in\caption{Sample activity patterns specifiable in LSL.
E1, E2, and E3 denote events. Start and End
denote the start and end of
program execution. }
\label{activity-patterns}
\end{figure}




\subsection{Timed events}
\label{timed-events}


LSL provides a powerful mechanism to
auralize  timed events. \lsltime/ is a special variable
in LSL which denotes
the time spent from the start of
program execution. Time is measured in system dependent 
{\em ticks};  
each tick being the smallest unit by which \lsltime/
could be incremented.
Thus, any expression using \lsltime/ can be used as
a timed event.
As an example, suppose that the {\em gear\_change }
function must be invoked in a program in less than 60 seconds
after the program execution begins. It is desired
to playback variable {\em bad\_program} if this condition is
not satisfied. The following \lslnotify/ illustrates how
to write this specification in LSL.


\vskip0.1in\indent \lslnotify/ \lslrule/ = function\_call: 
gear\_change 
\lsland/ \lslassertion/=\lsltime/ $\le$ sectotick(60) \\
\indent \hskip0.1in\lslusing/ bad\_program \lslmode/ = \lsldiscrete/;
\vskip0.1in

\noindent In the above example, {\em sectotick}
is an LSL predefined function  to convert seconds
to ticks. Notice that the expression \lsltime/ $>$ sectotick(60)
is a valid way to specify an event as described earlier
while discussing the syntax of \lslnotify/. 

It is often  required to specify time relative to the
occurrence of some event. This can be done in LSL
using  relative  timed events as shown below.


\vskip0.1in\indent \lslrtime/ = $<$expression$>$ \lslafter/ 
$<$event-specifier$>$
\vskip0.1in

\noindent Consider the use of this mechanism in the following
example for tracking an event.

\vskip0.1in
\lsltrack/ \lslwhen/  (\lslrtime/\,=\,sectotick(30)) \lslafter/
\lslrule/ = function\_call: missile\_launch\\
\indent  \hskip0.1in\lsluntil/ 
\lslrule/= function\_return: target\_hit \lslusing/ 
missile\_in\_motion;
\vskip0.1in

\noindent The above \lsltrack/ can be read as ``Begin
tracking 30 seconds after the function 
{\em missile\_launch} has been called and terminate 
tracking  when the function {\em target\_hit} returns.
The tracking sound is defined by the LSL variable
{\em missile\_in\_motion}. Thus, using a combination 
of \lsltime/ and \lslrtime/, one may specify a variety
of timed events for auralization.



\subsection{Playback synchronization}

Synchronization mode
controls the playback of notes during program execution.
There are two such modes: {\em program} or
{\em metronome}.
In the program mode, 
playback is synchronized to the program. In
the metronome mode it is synchronized to 
a global metronome. The \lslsync/ command
is used for setting the synchronization mode. The syntax of
\lslsync/ is:

\indent \lslsync/  $<$sync-to$>$

\noindent The $<$sync-to$>$ parameter can be 
\lslprogram/ or \lslmm/  for synchronization with,
respectively, program execution or a global metronome.
Multiple \lslsync/ commands may be placed 
in an LSL specification to alter the synchronization
mode.

In the metronome mode, a buffer holds the notes generated by
the executing program. When this buffer is full
and  the program attempts to
send a note for playback, the playback routine does not return
control to the program until the 
received note can be buffered. This may slow down program
execution. To avoid this situation in metronome mode,
one may use the \lslnoslow/ parameter 
such as in the command 
\lslsync/ \lslmm/ \lslq/=120,  \lslnoslow/.  
When the \lslnoslow/
parameter has been specified,
playback routine discards notes that are received when
the buffer is full. This could cause some events 
or data tracking to pass by unauralized.
The size of the playback buffer can be controlled by
setting the \lslbufsize/ parameter such as in
\lslsync/ \lslmm/=120, \lslbufsize/=1000 which
specifies a buffer size that will hold at least 
1000 notes.\footnote{Each note belonging to a chord
counts as one note.}


\subsection{Assignments, loops, and conditionals}


An assignment command
has the general syntax  shown below.

\vskip0.1in
identifier \{$<$subscript\_list$>$\}:= $<$expression$>$;
\vskip0.1in

\noindent where identifier is the name of a 
variable. Expression is any valid expression that
evaluates to the
type of the identifier on the left of the assignment.
$<$subscript\_list$>$ is a list of subscripts used
for selecting array elements if the identifier
denotes an array.
Loops can be formulated in an LSL specification
using the \lslfor/ and \lslwhile/ constructs.
Syntax of these two constructs is given below.

\vskip0.1in\indent\lslfor/ $<$for\_index$>$ := 
$<$init\_expression$>$ \lslto/ 
$<$final\_expression$>$ 
\{\lslstep/ $<$step\_expression$>$\}\\
\indent \hskip0.1in$<$spec\_sequence$>$


\indent\lslwhile/ $<$condition$>$ 
\lsldo/ $<$spec\_sequence$>$;
\vskip0.1in

\noindent The semantics of each of the
above commands are similar to
that of the \lslfor/ and \lslwhile/ statements
in Pascal. All expressions in a \lslfor/ command
must evaluate to integers. A $<$spec\_sequence$>$
is a sequence of zero  or more LSL specification
commands.


Conditional commands are  provided in LSL 
for selectively specifying  an auralization.
The syntax of a conditional command appears  below.
Its semantics are similar to that
of the \myif/ statement in Pascal. 

\lslif/  $<$condition$>$ \lslthen/ $<$spec\_sequence$>$
\{\lslelse/ $<$spec\_sequence$>$\}




\subsection{Controlling  auralization state}


During execution, an auralized program can be in one
of two auralization states: ON or OFF.
In the ON state any sound data resulting from
the occurrence of an auralized event is sent to
the sound processor. In the OFF state 
any such sound data is suppressed. LSL provides two
commands to dynamically 
alter the auralization state.
These are the \lslturn/ and the \lsltoggle/ commands.
These   commands have no effect when placed inside 
an LSL specification. They may affect the 
auralization state 
when placed inside the auralized program.


Using \lslturn/ is one  way to switch sounds
on or off. \lslturn/~\lslon/ 
switches the sound on and \lslturn/~\lsloff/ 
switches  it off. The command may be placed anywhere 
inside the auralized program. Upon the start of
program execution, the auralization state is ON.
The \lslturn/ command takes effect immediately
after it is executed. Sound channels can be switched off 
selectively
by specifying the channel number as in
\lslturn/ \lsloff/ \lslchan/=4; switches off
any sound on channel 4. 


Another way to turn the sound on or off
is with the \lsltoggle/ command.
The syntax of \lsltoggle/  is
given below.

\vskip0.1in
\lsltoggle/ \{\lslid/\}  $<$toggle-source$>$ = constant
\vskip0.1in

\noindent where $<$toggle-source$>$ could be 
the MIDI or  computer keyboard
indicated, respectively,
by the keywords \lslmidi/ and \lslkey/.
The constant is a string containing
the toggle note from the MIDI keyboard and  the toggle key
from the computer keyboard. 
When specified, \lslid/ denotes the name of a class
(defined below) of
events, activities, and data items to be affected
by this command. 


During program execution,
the auralization state can be toggled using the
source specified in the command. For example,
if the middle C on a MIDI keyboard is the toggle source,
tapping the middle C once, after program execution begins,
turns the sound off. Tapping it again 
turns it on. Input from the toggle source
is processed only when an auralized event occurs.
When such an event occurs, an LSL library routine is
invoked to check for a pending toggle request.
If a request is pending,
the auralization state is switched to OFF if
it is ON,  or to ON if it is OFF. 



A program may contain both \lslturn/ and \lsltoggle/
commands. A \lslturn/ might change the
auralization state to off only to be switched
back to on by a toggle. This is certainly one useful
scenario. Note that whereas \lslturn/ commands are placed 
into the code prior to compilation and do not provide the
user any control {\em after} compilation, 
the \lsltoggle/ command permits dynamic 
changes to the  auralization state.  The toggle default
in LSL is the space bar on the computer keyboard.
Thus, even when no \lsltoggle/
is specified in a program, auralization state
may be toggled using the space bar.


Regardless of the auralization state,
note values are generated and sent to
the library routine responsible
for playback. It is this library routine
that decides, based on
the current auralization state, if the received 
notes are to
be played or not.  In the metronome sync mode, all
notes emitted are buffered in a special
playback buffer maintained by the library routine.
The buffered notes are removed  from
the buffer when their turn comes for playback.
This is determined by the current metronome setting.
When playback resumes due to a
\lsltoggle/ or a \lslturn/ changing the auralization
state to \lslon/, the 
notes are  played back 
in accordance with the metronome setting.
In program sync mode, notes received by the library
routine are discarded if playback is turned off.



\subsection{Event, data,  and activity classes}
\label{event-classes}


An event class\footnote{Classes defined in
this section have no intentional relationship with the notion 
of classes in C$++$ and object oriented programming literature.}
consists of one or more events.
A \lslnotify/ command specifies one or
more  events  which may occur at several
positions inside a program and several times
during program execution. Events   specified in 
one or more \lslnotify/  commands constitute an
event  class.
Similarly, a {\em data class} is a collection
of one or more variables. A \lsltrack/ command
specifies one or more variables to
be tracked. Variables  specified in
one or more \lsltrack/ commands
constitute a data  class. An {\em activity class} is defined
similarly with respect to activities 
specified in one or more \lslatrack/ commands.
A class that consists of
at least two elements  of different types,
e.g. event and activity, or event and data,
or data and activity,  is known as a
{\em mixed class}. 


It is possible for a user to define   each of the above classes
in an LSL specification.
This is done by  naming one or more \lslnotify/,
\lsltrack/, and \lslatrack/ commands. Any
of these three commands can
be named using the following syntax:

\vskip0.1in
\indent id$_1$::id$_2$::\ldots::id$_n$::command
\vskip0.1in

\noindent where each  subscripted {\em id} above  
denotes a name and 
{\em command} denotes any
event, data, or activity specification command.
Multiple commands can share a name.
Each {\em id}, when used as the name of a command,
is treated as the name of
a class. The class so named  consists of
events, data, or activities   specified in the
commands named by {\em id}.  One command can be assigned multiple
names. This makes it easy to
define classes that are not disjoint.
Consider the following example.


\vskip0.1in
\indent function\_related::\lslnotify/ \lslrule/=function\_call;\\
\indent function\_related::\lslnotify/ \lslrule/=function\_return;\\
\indent data\_related::\lsltrack/ a \lsland/ b \lsland/ c;\\
\indent special::data\_related::\lsltrack/ p \lsland/ q;
\vskip0.1in

\noindent The above three commands have
been named to identify three classes. Class {\em function\_related}
consists of events that correspond to function calls and return.
Another class named 
{\em data\_related} consists of data items {\em a, b,c, p},
and {\em q}. Yet another class named {\em special} consists of
data items {\em p} and {\em q}. 

The notion of a class can be used to
model abstraction during program auralization.
For example, consider the auralization of
tractor control software. The  programmer  may like to
group all the events into two classes.
One  class consists of events that correspond to engine
control. Another class consists of events that correspond to
the control of
paraphernalia attached to the tractor,  e.g. a seeding device.
By simply using the event specification mechanism
of LSL there is no way to explicitly incorporate these
classes into an LSL specification.  The mechanism of naming
a command, as described above, however, does
provide a convenient
means for defining  classes.

Once defined, classes of events can be accessed
at an abstract level using their
names. For example, during the execution
of an auralized program, it is possible to
interact with the LSL run-time system and
turn off the auralization of all
events within a class. It is also possible to
request LSLed (described in Section~\ref{lsled} below)  to
provide a comprehensive list of 
classes and their individual elements. 
Thus the use of classes enables a user to
interact with an auralized program in terms of
``high level''  occurrences, e.g. events,
instead of  dealing with syntax based definitions. 



\subsection{Embedding LSL commands}
\label{embedding}


LSL commands can be embedded in C programs inside
comments. The LSL preprocessor
recognizes an LSL command embedding if
the first token beginning with a letter
immediately following
the comment begin delimiter (/*) is {\tt LSL:}. 
Immediately following the delimiter,
a sequence of LSL commands can be placed 
enclosed within the \lslbegin/ and \lslend/ delimiters.
For an example of such an embedding see 
Example~\ref{track-vdap-example} 
on page~\pageref{'track-vdap-example'}.
The LSL commands so embedded are
translated to C code by the LSL preprocessor.
LSL commands such as \lslplay/ and  \lslnotify/
get translated into calls to library functions.
Other LSL commands, such as assignments and \lsltrack/
commands get translated into more complex C 
code. 

It is possible to identify specific 
constructs of a C program by labelling.
A label is placed inside a comment 
by using the keyword \lsllabel/ as the first keyword starting with
a letter  immediately following the comment
start delimiter. Thus, for example, 
/* \lsllabel/=here, onemore */
provides two labels {\em here} and {\em onemore}
for possible use by the LSL
preprocessor.
The following example shows how to label 
the beginning and end of a loop. 


\begin{tabbing}
\programtab
\>\>\vdots\\
\>\lslwhile/ (c~=~getchar()!=eof)\\
\>\>\{\\
\>\>/*\lsllabel/=special\_loop   This is an LSL label for
the beginning of loop body. */ \\
\>\>\>++nc;\\
\>\>\vdots\\
\>\>/*\lsllabel/=special\_loop   This is an LSL label for 
the end of loop body. */ \\
\>\>\}
\end{tabbing}





\section{Examples of  LSL/C use }
\label{examples}


We now present a few  examples 
illustrating the use of LSL. Each example consists of
a problem statement and a solution using LSL/C.
In each example  we
assume that, unless specified otherwise,
default values  are used for various sound
related parameters such as MIDI channel,  timbre, 
volume, and metronome value.





\subsection*{\myexample{loop-auralize}}

It is desired to auralize all loops in a C program. 
Loops will be identified by the \lslwhile/-\lsldo/ constructs.
On entry to a loop,  note C4 is to be played 
for half a measure duration. Each time 
control reaches 
the start and end of a loop body,  notes
E4 and G4 are to be played, 
respectively. On loop 
exit, note C5 is to be played for a full measure.
The following LSL/C specification meets the above
auralization requirements.


\begin{tabbing}
\programtab
\lslabegin/\\
\lslspecmod/ loop\_auralize\\
\lslvar/\\
\>l\_begin, l\_end, b\_begin, b\_end:~\lslnote/;\\
\lslbegin/ loop\_auralize\\
\>l\_begin:= ``C4:h''; l\_end:= ``C5:f'';\\
\>b\_begin:= ``E4:q''; b\_end:= ``G4:q'';\\
\>\lslnotify/ \lslall/  \lslrule/ = while\_statement\_enter \lslusing/ l\_begin;\\
\>\lslnotify/ \lslall/  \lslrule/ = while\_statement\_exit \lslusing/ l\_end;\\
\>\lslnotify/ \lslall/  \lslrule/ = while\_body\_begin \lslusing/ b\_begin;\\
\>\lslnotify/ \lslall/  \lslrule/ = while\_body\_end \lslusing/ b\_end;\\
\lslend/ loop\_auralize;\\
\lslaend/.

\end{tabbing}

\bend



\subsection*{\myexample{call-auralize}}

An automobile contains a distributed microcontroller
network. The software to control this
network and other automobile 
functions  is to be tested and debugged. All calls
to functions  {\em gear\_change}, {\em oil\_check}, and
{\em weak\_battery} are to be auralized by playing suitable
sound patterns. The following LSL/C specification meets
this auralization requirement.

\begin{tabbing}
\programtab
\lslabegin/\\
\lslspecmod/ call\_auralize\\
\lslvar/\\
\>gear\_change\_pattern, oil\_check\_pattern,
battery\_weak\_pattern: \lslpattern/;\\
\lslbegin/ call\_auralize\\
\>gear\_change\_pattern:= ``F2G2F2G2F2G2C1:qq''+ ``C1:f'';\\
\>oil\_check\_pattern:= ``F6G6:h'';\\
\>battery\_weak\_pattern:= ``A2C2A2C2'';\\
\>\lslnotify/ \lslall/ \lslrule/ = 
function\_call: ``gear\_change'' \lslusing/
gear\_change\_pattern;\\
\>\lslnotify/ \lslall/ \lslrule/ = 
function\_call: ``oil\_check'' \lslusing/
oil\_check\_pattern;\\
\>\lslnotify/ \lslall/ \lslrule/ = 
function\_call: ``battery\_weak'' \lslusing/
battery\_weak\_pattern;\\
\lslend/ call\_auralize;\\
\lslaend/.
\end{tabbing}

\bend

\subsection*{\myexample{parallel-prog-auralize}}


A parallel program consists of a procedure MAIN which,
after initialization, loads each of N processors
with a copy of program SOLVE.
On each processor, SOLVE aids in the solution of
a partial differential equation using a discretization
of a multidimensional  input domain.
During execution SOLVE communicates with its neighboring
processors by placing calls to procedures
{\em nread} (for read from a processor) and 
{\em nwrite} (for
write to a processor). It is
desired to auralize this program  so that calls to
nread and nwrite are reported by 
suitable sound patterns. Only calls
that originate from
even numbered processors are to be auralized.
The following LSL/C specification meets the above requirements.

\begin{tabbing}
\programtab
\lslabegin/\\
\lslspecmod/ par\_processing\\
\lslexternal/ even, processor\_num;\\
\lslvar/\\
\>read\_pattern, write\_pattern: \lslpattern/;\\
\lslbegin/ par\_processing\\
\>read\_pattern:= ``G5'';\\
\>write\_pattern:=  ``C4'';\\
\>\lslnotify/ \lslall/ \lslrule/ = 
function\_call: ``nread'' \lsland/ 
even(processor\_num) \lslusing/ 
read\_pattern ;\\
\>\lslnotify/ \lslall/ \lslrule/ = 
function\_call: ``nwrite'' \lsland/ 
even(processor\_num)\lslusing/ 
write\_pattern ;\\
\lslend/ par\_processing;\\
\lslaend/.
\end{tabbing}
\bend

\subsection*{\myexample{editor}}

An editor accepts  commands for
text editing. After having executed a  command,
a voice message saying ``Done'' is to be generated.
Assume that after receiving an edit command that
is to be voiced as above, 
the editor invokes  a function named 
{\em process\_command} to
process the input command.
The following LSL/C specification meets the above requirement.

\begin{tabbing}
\programtab
\lslabegin/\\
\lslspecmod/ editor\_auralize\\
\lslvar/\\
\>done\_voice: \lslvoice/;\\
\>myfilename: \lslfile/;\\
\lslbegin/ editor\_auralize\\
\>filename := ``done-container.v'';\\
\>done\_voice:= filename;\\
\>\lslnotify/ \lslall/ \lslrule/ =function\_return: 
``process\_command''
\lslusing / done\_voice;\\
\lslend/ editor\_auralize;\\
\lslaend/.
\end{tabbing}

\bend


\subsection*{\myexample{for-loop}}


It is desired to auralize all labelled assignments
in 10 files named file-1.c, file-2.c, and so on through
file-10.c. Assignments in
the first of these files are to be notified by
playing a C4, the next by playing a D4, then by E4,
and so on using successive notes in
the C-major scale. The following LSL/C specification
meets this requirement.

\begin{tabbing}
\programtab
\lslabegin/\\
\lslspecmod/ for\_loop\\
\lslconst/\\
\>num\_of\_files = 10;\\[0.1in]
\lslvar/\\
\>first\_note, next\_note: \lslnote/;\\
\>next\_file: \myint/;\\
\>filenames: \lslarray/  [1..num\_of\_files]
\lslof/ \lslfile/;\\
\lslspecdef/ init\_file\_names();\\
\>\lslbegin/~init\_file\_names\\
\>\>filenames[1] := ``file-1.c''; filenames[2] := ``file-2.c'';\\
\>\>filenames[3] := ``file-3.c''; filenames[4] := ``file-4.c'';\\
\>\>filenames[5] := ``file-5.c''; filenames[6] := ``file-6.c'';\\
\>\>filenames[7] := ``file-7.c''; filenames[8] := ``file-8.c'';\\
\>\>filenames[9] := ``file-9.c''; filenames[10] := ``file-10.c'';\\
\>\lslend/~init\_file\_names;\\
\lslbegin/ for\_loop\\
\>next\_file:=1; next\_note:=first\_note; \\
\>init\_file\_names();\\
\>\lslfor/ next\_file:=1 \myto/ num\_of\_files \lsldo/\\
\>\lslbegin/\\
\>\>\lslnotify/ \lslselective/ \lsllabel/~=~this\_assign
\lslrule/=assignment\_expression  \lslusing/ next\_note\\
\>\>\>\>\lslin/ \lslfilename/ = filenames[next\_file];\\
\>\>next\_note := nabove(next\_note, ``C:major'');\\
\>\lslend/\\
\lslend/ for\_loop;\\
\lslaend/.
\end{tabbing}

The above LSL/C specification uses an LSL function
to insert  all file names into an array.
The \lslfor/ loop then steps through each file
name to specify the desired auralization.
Successive notes are obtained using a predefined function
named {\em nabove} which takes a note as an argument
and returns the next note a full step above
on a given scale. 
\bend


\subsection*{\myexample{track-vdap-example}}
\label{'track-vdap-example'}

This example illustrates 
how \lsltrack/ can be used
with VDAP (Value Dependent Aural Pattern)
 to track arbitrary  functions
of program variables. Suppose that 
it is desired to track 
the dynamic relationship between two
variables named {\em rock\_remain} and 
{\em dist\_remain}. {\em rock\_remain}
represents the number of rockets remaining
to be fired and {\em dist\_remain}
the remaining distance to be travelled.
Each time a rocket is fired the count of remaining rockets
reduces by 1. The distance remaining to
be travelled is updated by some process in
the program. 

It is desired to emit a continuous audible sound only 
while 
the remaining distance ({\em dist\_remain})
 is more than the critical
distance ({\em crit\_dist}) and
the number of remaining rockets 
{\em rock\_remain} is less than 
the critical rocket count ({\em crit\_rock}).
An LSL specification to 
achieve the above auralization consists of two parts.
One part  is an LSL specification module
containing a \lsltrack/ command. This command
specifies the use of a VDAP function
named {\em crit\_dist\_track}
for tracking {\em dist\_remain}. The second part
is the definition of a C function  named {\em crit\_dist\_track}
which has embedded LSL commands.
The entire LSL specification module appears below.


\begin{tabbing}
\programtab
\lslabegin/\\

\>\lslvdapbegin/\\
\> crit\_dist\_track ( \lslint/ *distval, *rockval);\\
\>\{\> /* This is a C function. {\em distval} is a pointer to \\
\>\>{\em dist\_remain} and {\em rockval} is a pointer
to {\em rock\_remain}. */\\
\>\>\myif/ ((*distval $>$ crit\_distance) \myand/ 
(*rockval $<$ crit\_rock))\\
\>\>\{~/* LSL: \\
\>\>\>\lslbegin/\\
\>\>\>\> \lslplay/ grave\_note \lslwith/ \lslinst/=bass,
\lslmode/=\lslcontinuous/, \lslchan/=4;\\
\>\>\>\lslend/\\
\>\> */~\}\\
\>\>\myelse/ \\
\>\>\{ /* LSL: \\
\>\>\>\lslbegin/\\
\>\>\>\> \lslturn/ \lsloff/ \lslchan/=4;\\
\>\>\>\lslend/\\
\>\> */~\}\\
\> \}\\
\>\lslvdapend/;\\
\>\lslspecmod/ emergency\_sound\\
\>\lslapply/ \lslfilename/ =  dist\_compute.c;\\
\>\lslconst/\\
\>\>grave\_note = ``G2'';\\
\>\lslbegin/ emergency\_sound\\
\>\>\lsltrack/ {\em dist\_remain} \lsland/  {\em rock\_remain}
\lslusing/  {\em crit\_dist\_track}
\,(\&dist\_remain, \&rock\_remain\,);\\
\>\lslend/ emergency\_sound;\\
\lslaend/.
\end{tabbing}

While processing the above LSL module,
the LSL preprocessor
adds calls to the C function {\em crit\_dist\_track},
together with the specified
parameters, immediately after each program statement that
could possibly alter the value of {\em dist\_remain} or 
{\em rock\_remain}. The preprocessor also translates
the LSL commands
inside a VDAP to C code. The function so obtained
is placed in suitable file for
compilation by the C compiler. The preprocessor
does not check for the correctness of the VDAP.
\bend


\subsection*{\myexample{factorial-dap-example}}
\label{'factorial-vdap-example'}



A recursive function named {\em factorial} is to be auralized
in such a way that each call to {\em factorial} generates
a note with  pitch proportional to the depth of recursion.
The first call to factorial should
generate C4; successive calls should generate
one note higher in the C-major scale. 
The  return sequence from {\em factorial} should 
play  the notes back in
the opposite order.
The following LSL specification meets the above requirements.


\begin{tabbing}
\programtab
\lslabegin/ \\
\>\lslvdapbegin/\\
\> fact\_enter (~)\\
\>\>\{~/* LSL: \\
\>\>\>\lslbegin/\\
\>\>\>\>note\_to\_play = nabove (note\_to\_play, ``C:major'');\\
\>\>\>\> \lslplay/ note\_to\_play\\
\>\>\>\lslend/;\\
\>\> */~\}\\
\> fact\_exit (~)\\
\>\>\{~/* LSL: \\
\>\>\>\lslbegin/\\
\>\>\>\>note\_to\_play = nbelow (note\_to\_play,``C:major'');\\
\>\>\>\> \lslplay/ note\_to\_play;\\
\>\>\>\lslend/\\
\>\> */~\}\\
\> \}\\
\>\lslvdapend/;\\
\>\lslspecmod/ fact\_auralize\\
\>\lslvar/\\
\>\>note\_to\_play: \lslnote/;\\
\>\lslbegin/ fact\_auralize\\
\>\>note\_to\_play:= ``B3''; /* The next note is C4.*/\\
\>\>\lslnotify/ \lslall/ \lslrule/ = function\_entry: 
``factorial'' \lslusing/ fact\_enter();\\
\>\>\lslnotify/ \lslall/ \lslrule/ = function\_return: 
``factorial'' \lslusing/ fact\_exit();\\
\>\lslend/ fact\_auralize;\\
\lslaend/.
\end{tabbing}

The above specification contains two VDAP functions,
namely {\em fact\_enter} and {\em fact\_exit}.
Each call to and return from
{\rm factorial}  results
in, respectively,  the execution of {\em fact\_enter} and 
{\em fact\_exit} resulting in the desired playback.
\bend


\section{LSL in a programming environment}
\label{environment}

% Discuss LSL editor design. The need for special editor.


LSL is designed to fulfill complex demands for
program auralization. In a programming environment,
one may use LSL to auralize parts of a program 
during the testing and debugging phase or 
auralize an application to
meet auralization requirements stated at the
beginning of the development cycle. 
As  shown in Figure~\ref{lsl-use},
there are at  least three different ways
a software developer could use LSL. An expert in the use
of LSL could   write
an LSL specification or add LSL commands to the program itself
and preprocess the program as
shown in Figures~\ref{lsl-use}(a) and (b). Alternately,
as in Figure~\ref{lsl-use}(c),
a graphical interface could be used to
ease the task of specifying auralization. 
This  interface  accepts and translates user 
commands 
to LSL specifications.
Specifications so
generated are internal to the interface 
and  hidden from a novice  user.


\begin{figure}[htb]
\vskip-1.5in
\centerline{\includegraphics[width=4.5in,height=6in]{\figureloc/lsl-uses}}
\vskip-1.5in\caption{Use of LSL in a programming environment.}
\label{lsl-use}
\end{figure}


The result of using any of the above methods is 
an object program instrumented with calls
to procedures from the LSL sound library.
When the instrumented program is executed,
these calls generate sound data that is
directed to a sound module\,\footnote{The LISTEN system
currently uses Proteus/3 World from
E-mu Systems  as the sound module. The Audiomedia~II 
card from Digidesign
is used for voice sampling and 
playback. }
via   MIDI. The sound
module output is sent through an amplifier 
to a speaker. Voice
is sent via an audio processor and amplifier to the
speakers. Non-voice and voice data are mixed
at the amplifier and played through the
same speakers.  A user may organize the audio 
playback hardware differently.


\subsection{LSL editor}
\label{lsled}

In principle it is
possible to use 
any of the well known text editors, such as 
{\tt gnuemacs} or {\tt vi} to edit an LSL specification.
However, in several cases such editing may be
inconvenient; specially so for one not familiar with
the piano keyboard or music terminology.
For this reason we propose a special editor,
named LSLed, for preparing LSL specifications.


LSLed has features similar to those found
in  syntax-directed editors on personal computers
such as the Think~C editor on the Macintosh\,\cite{think-c}
computer\footnote{Macintosh is a trademark of 
Apple Computer, Inc..}.
LSLed is most useful when editing sound related 
commands. For example, suppose that variable 
named {\em drag-icon} is declared
to be of type \lslpattern/. We wish to
assign a sound pattern to {\em drag-icon}.
One way to do so would be to 
type in the pattern using the notation
described earlier in Section~\ref{constants}.
Thus, one may type in the following text:

drag-icon := ``C2D2E2F2G2A2B2C3'';

\noindent We identify two problems with this approach.
First, an individual may not be  familiar with the 
notation (e.g. one
who asks the question: {\em What is C2\/}?)
Second, one may be familiar with the notation
but would like to experiment
with various sounds and just play
the sound pattern on an electronic  keyboard 
instead of converting it to the textual 
notation. LSLed helps overcome
these problems.

During editing, LSLed could be in  one of two modes:
{\em text} and  {\em aural}. When in text mode,
all input to LSLed is from
the computer keyboard. When in aural mode,
the source of input could be  MIDI via 
an electronic keyboard
or the computer keyboard. When the above assignment to
variable {\em drag\_icon} is  typed, LSLed expects 
input from the computer keyboard until all
characters up to and including the
assignment (:=) operator
have been typed. It then adds a double quote 
to indicate the starting of 
a string and prompts the user to
play the pattern on the electronic keyboard.
At this point the editor screen would 
display, among other characters:

drag\_icon := ``

\noindent As the user plays the notes on a 
keyboard, the textual version of each note
appears following
the double quotes.  On completion,
the user hits the return key
indicating that the pattern has terminated.
To change any previously entered pattern,
the user merely places the cursor at the beginning
of the pattern immediately
following the double quotes,
and repeats the above process. 
Textual version may also be edited 
by switching LSLed to textual mode. This
may be useful for
adding or altering note parameters such as,
for example,   duration or play style.
The above process can be used to enter other
types of sound related constants too. The advantage
of this approach is clear when one considers the
ease of data entry. The duration of notes 
played on the keyboard is determined using various
LSLed parameter settings not described here.

LSLed also interprets commands 
in an LSL specification. For example, 
a play command  in an LSL specification
can be interpreted to hear how the patterns specified 
in the command will sound. Interpretation
of  more complex commands such as \lslnotify/
and \lsltrack/ are  also possible via the graphic interface
as described in Section~\ref{implementation}.


As LSL is a typed language, LSLed checks for
any type mismatch. For example,  if $x$ is of type 
\lslnote/ and one types:

x:= 4;

\noindent LSLed complains about the invalid type on the 
right side of the assignment.

Figure~\ref{lsl-ed} shows how LSLed fits in a programming
environment. It takes as input an existing 
LSL specification file and creates an updated
or a new file. LSLed obtains its  input 
from an electronic or a computer keyboard.
LSL library routines are used for the input and output of
sound patterns. The program to be auralized
is also an input to LSLed. This enables LSLed
to check for any interaction errors
between the LSL specification and the
program. For example, if a
\lslnotify/ command  indicates that
the events  to
be selected are  within function {\em  icon-operation},
LSLed checks if such a function indeed exists
in the program. If not, then a warning is
issued. In any case, checks such as these
are also made during LSL specification preprocessing. 




\begin{figure}[htb]
\vskip-1.5in
\centerline{\includegraphics[width=4.5in,height=6in]{\figureloc/lsl-ed}}
\vskip-2.5in\caption{LSLed  in a programming environment.}
\label{lsl-ed}
\end{figure}





\section{An LSL implementation outline}
\label{implementation}

An LSL implementation consists of four
software components: (i)~an LSL preprocessor,
(ii)~an LSL editor, (iii)~an LSL sound library,
and (iv)~an LSL graphic interface. 
Below we outline the elements of the preprocessor,
the library, and the graphic interface.
The editor has already been described above.

\subsection{LSL preprocessor}

The task of the preprocessor is to 
instrument the program to be auralized with
calls to sound library procedures. The instrumentation
is guided by the LSL specification. In the absence of
any  specifications, the preprocessor may be required to
process LSL commands embedded within the
program.  

The preprocessor takes an LSL specification and a
program $P$
as input and produces the instrumented version
$P_L$ of $P$. $P_L$ is then compiled and linked
to the library routines  using the 
traditional C compiler and linker. This generates 
an object program with auralization commands compiled in.
The following steps provide a high level
view of  the sequence of actions
taken by the preprocessor
to transform $P$ into $P_L$.

\begin{enumerate}


\item \label{parse} Parse each source  file $F$  
that is input to the preprocessor.
This results in a parse tree $T$ for source
file $F$,   whose nodes are typed.
Using the  type information,
 it is possible to examine  a  node of $T$  and
determine what syntactic construct
it represents. 



During the construction of $T$, identify any LSL commands.
Replace each command by an appropriate call to 
a sound library procedure. This call  now becomes a 
part of $T$.




\item \label{parse-lsl} Begin parsing 
the LSL specification if one has
been supplied. If there is no LSL
specification then go to
step~\ref{deparse}.  Otherwise, for each
\lslnotify/ or \lsltrack/
command encountered, say $C$,  do the following:

\begin{enumerate}


\item \label{process-lsl} Traverse $T$ and 
identify  syntactic constructs
that match the  event specification in $C$.
To each syntactic construct $S$ so
identified, add  an appropriate call to
the sound library procedure that will
generate the sound
specified in $C$.
This step updates $T$.


\end{enumerate}


\item \label{deparse} The updated parse tree, 
say $T'$ contains
new nodes that represent calls to
sound library function calls.
Deparse  $T'$ by traversing it and
generating a source file $F'$.

\item \label{compile} Invoke the C compiler available 
in the environment with input $F'$.
This will result in the object version of
$F'$. 


\item \label{clean} Delete $F'$.

\end{enumerate}


In a UNIX environment, the above sequence of steps is
initiated by the following  command:

\vskip0.1in
lp -spec aspec.l -file icons.c
\vskip0.1in


\noindent Assuming that icons.c contains a
function named {\em main}, the above command will result in 
a file named {\em a.out} containing the 
executable auralized version of {\em icons.c.}
The auralization will be done using the LSL
specifications contained in the file aspec.l.
For programs  spread over several
files,  makefiles\,\cite{unix-guide},
and the make command can be used.
 The {\tt lp} command can
be embedded in the makefile. For programs 
that need to be auralized, calls to the
C compiler may be replaced by  calls to {\tt lp}.
Recall from Section~\ref{syntax} that 
applicability constraints may be
used  to restrict the scope of LSL specifications
within a source file.


The above algorithm may perform  multiple passes over the
parse tree. The number of
passes over the program parse  tree $T$
is equal to the number of 
times \lslnotify/ and \lsltrack/ commands
are encountered  while
interpreting an LSL specification. 
Development of a more efficient single pass algorithm 
for auralization based on a given LSL specification
is an open problem. Further, the preprocessor
may not always succeed in performing
correct auralization due to
pointers and aliases. The problem can be resolved using
labels or scope specifiers in the LSL specification
to inform the preprocessor where 
to look for an event  or data item.


\subsection{Graphic interface}
\label{graphics}


A graphic interface is an integral
part of an LSL environment. The interface 
enables one to specify auralization requirements
without having to learn LSL. When using this interface,
a user uses pull-down and pop-up  menus that guide
the development of specifications. These specifications
are transformed into LSL syntax by
the interface and saved in a specification file just
as a user would do if LSL was used directly.
Once the specification is complete, the interface
can be instructed to invoke the LSL preprocessor and 
the C compiler to instrument and auralize a program.
The interface also provides an environment for
executing an auralized program. Thus, after
having compiled the program,
it may be  instructed to execute the
program under programmer's control. 
This feature may be used
to debug the program and alter auralizations
when necessary.


\subsection{The LSL sound library}
\label{sound-library}

This is a library consisting of procedures 
that are responsible for
sending appropriate data to MIDI or
other sound generation devices during program execution.
The entire package to manage
the processing
of notes received during  program execution 
is a part of the library.
In addition, the library consists of 
several predefined functions callable from within
an LSL specification and a C program.




\section{Summary}
\label{summary}

Research efforts in program auralization appear to be
on the increase. We are not aware of any
general purpose method 
to specify program auralizations and a  supporting tool
to auralize a program in accordance with the
specifications. We have presented the syntax and semantics
of a language named LSL  that provides a notation to
specify a variety of
program auralizations. LSL is 
generic and needs to be
adapted to the programming language
of an environment in which programs 
are expected to be auralized. 
A language specific  implementation of
LSL serves as a tool to auralize programs.
We have presented the design of LSL/C, a 
C adaptation of LSL. 

Writing LSL  specifications could be a tedious task.
To simplify the process, we have proposed a
syntax directed editor for LSL,
named LSLed, and a graphic interface.
LSLed is useful while writing LSL specifications
directly using LSL. The graphic interface 
is useful for a novice to
LSL. Whether one uses LSL
directly or the graphic interface to
specify an auralization, the
result is an LSL specification. This specification
serves as the basis for instrumenting the program to
be auralized. The instrumentation consists of 
calls to procedures from LSL sound library.

The implementation of various parts of the LSL environment
is currently underway as  part of the LISTEN project.
Once different components of this  environment
are available and integrated  we will be  
sufficiently equipped to
conduct experiments
with the use of 
auralization in various program development 
activities. The tools are also expected to help
other researchers in the area of program auralization.





\section*{Acknowledgements}

Ronnie Martin spent many painful hours  
reviewing the first draft
of this report.  The idea of using time in LSL is due to 
the guitarist, humorist, and computer scientist, 
Professor Vernon~Rego. The most wonderful
teachers Verna Abe and Helen Brown  taught 
the second author the rudiments
of music notation and theory that shaped parts 
of LSL design. Will~Montgomery 
and Neil~Herzinger answered endless MIDI related
questions. Our thanks go to all these individuals.

\addcontentsline{toc}{section}{\protect\numberline{ }{Acknowledgements}}


%\bibliography{/u/u32/apm/research/sound-research/music-bibliography/music}

\bibliography{/Users/adityamathur/Documents/www/courses/BITSC461-fall03/listen-code/listen-report/vk-thesis.bib}
\bibliographystyle{plain}




\clearpage

\addcontentsline{toc}{section}{\protect\numberline{ }{Appendix\,: 
Syntax of LSL}}
\input lsl-grammar.tex






\end{document}


\item \label{func-play}
\lslnotify/ \lslall/ \lslinstance/=(shuttle\_speed=* \lslor/
\lslpar/=shuttle\_speed) 
\lslusing/ \lslfplay/ variable\_track(shuttle\_speed);


% fplay command has been removed from
% the text. June 28, 1993.


A variation of \lslplay/ is the \lslfplay/ command
with a simpler syntax:

\lslfplay/ $<$playfunc$>$ \{ \lslwith/ $<$keysig$>$, $<$timesig$>$
\lslat/ mm=$<$integer$>$ \}

\noindent In the above command,
$<$playfunc$>$ is a language dependent function
which is called during execution. LSL does 
not parse  $<$playfunc$>$. It is intended to be used
in the program to be auralized. However, it
is placed in the program by an LSL  preprocessor
as described in the examples given below.



% Removed on July 12, 1993. Replaced by event-space idea.


so that works of great composers could be specified 
using LSL syntax. Such a specification when interpreted
will auralize the work specified. 


On first thought our
strategy might appear to be an overkill 
and irrational. However, we used the following logic 
to justify our design decisions. First, we 
do not want to purposefully limit in any way 
the creativity of a programmer who desires to
auralize a program. Thus, for example, a programmer
might want to use the execution of
a program as the basis for a wild composition
that might be classified as belonging to
a completely new style of  {\em modern program music}.
Second, a parallel program can be compared with an
orchestral work.  For example, individual processes,
possibly executing on different processors, are like 
the different  staves in a symphonic score.
Execution of each process is like the rendition of
the score on one of the staves. Multiple processes work
in harmony to produce the desired results.
