System for Distribution of Synchronous Transcription in Real-time
Site index:
- Introduction
- History
- Purpose of Polygraf
- Description of the System
- Uniqueness of Polygraf
- Features of Polygraf
- Components of Polygraf
- Past occasions for presenting Polygraf v2
- Past occasions for presenting Polygraf v1
- Appendices to Download
Introduction
The Support Centre for Students with Special Needs at Masaryk University, Czech Republic, provides students with disabilities with services in educational setup.
Besides providing translation into sign language, one of the major services for students with hearing impairment is the speech-to-text reporting service, when a reporter records the speech (a lecture or a seminar in a university environment or even any similar event outside of a university environment) as a text on a computer by means of a simultaneous typing, and several students reading the report on one common screen at the same time.
As students with visual impairment are concerned, the primary service of Masaryk University is to provide accessible documents to the group of students. Considering the fact that the documents to be accessed include also visual materials presented by a lecturer via a presentation screen, we have to deal with the problem that distance to the presentation screen as well as light conditions or the presentation layout creates readability barriers and obstructs the students to perceive presented information.
As the number of students who are dependent on those services has been increasing, Masaryk University tried to find a satisfactory solution of:
situations when groups of three or more hearing impaired students needed to follow a text report at once and provide them with a comfortable access to the report
1. Visualisation of the synchronous transcription (speech-to-text reports)
Initially, all students would watch a single screen – that of a reporter‘s laptop – but this solution turned out to be unbearable when there were more than two students. Requirements for a new solution were the following:
- simple and quick technical preparation;
- independence on a local technical infrastructure;
- speech-to-text report must be synchronized with a reporter‘s typing;
- basic parameters of a reported text should be customizable;
- a possibility of a basic two-way communication (a user must be allowed to send text messages).
and
access to presentation documents in enlarged form displayed on a handheld device by visually impaired.
2. Presentation screen mirroring enhancing the readability for users with visual impairment
Observing the students in lectures at the university and participants at presentations on conferences we gathered a list of key features required to such a system:
- simple and quick technical preparation;
- independence on presentation technologies;
- speech-to-text report must be possible to be added;
- magnification of the content must be possible;
- compatible with handheld devices of users;
- nearly real-time update of the mirroring without consuming too much battery power.
Polygraf, version 2 has been developed at Masaryk University to solve technically both the issues. And as environment of lectures, seminars and presentation events naturally creates other more barriers for persons with other kinds of disabilities, we plan to continue developing Polygraf to implement more features resolving other issues.
top
History
The development on Polygraf system started in 2010 as a reaction on the increasing number of hearing impaired students. Till that time, the visualisation of transcripts was realised by individual solutions. In few cases where there were more than two hearing impaired students at once, we provisionally solved the issues in these ways:
-
We would connect a second monitor to a reporter's laptop so that some
users could follow the report directly on a laptop screen and some on
a monitor.
This solution was dependent on providing the connection between a laptop and a monitor and a power line to the monitor. This was not ideal for reasons of organization, time and, after all, even aesthetically. -
We would set up a wireless network for a distribution of a video
output from a reporter's laptop (using a VNC application) to small
laptops available to users.
As the distributed image was an exact copy of a reporter's laptop screen (this is not always appropriate or desirable), the system did not allow any customization on the part of user of the shared report.
Furthermore, the initial procedure – interconnection of all system components – was not simple and its success was quite dependent on users' technical skills.
The development on Polygraf version 2 started at the end of 2012 and it was required by visually impaired students as well as by other participants who follow presentations within university activities. Improvised solutions upon the release of Polygraf version 2 were not very effective and costed a lot of preparation:
-
Actually, there was no systematic solution to guarantee overall an access to
presentation documents by visually impaired. In individual cases students were
provided with presentation files beforehand obtained from a teacher, or
the files are available for the students in the central information system of
the university. The students could follow the presentation document on their
computer during a lecture or seminar.
Even though it assures an basic access to the presentations, it could not provide to view it synchronously with currently projected part of the presentation.
top
Purpose of Polygraf
Initially, the main aim to develop Polygraf was to provide an access to spoken language for hard of hearing students of university, so Polygraf version 1 solved the issue of visualizing speech-to-text reports for larger group of users with that disability.
As more and more students with visual impairment have encountered barriers to perceive information displayed on presentation screens, Polygraf has been developed further on (version 2) to resolve the issue.
Polygraf as a whole is typically used at lectures and seminars but, however, given the fact that requirements for a technical equipment on a location are minimal, it can also be used outside of school facilities (such as at presentations and business meetings) when participation of persons with the disabilities is expected.
top
Description of the System
A text reporter records a transcript of spoken language using a text
editor – Microsoft Word is commonly used among Czech speech-to-text reporters.
MS Word is extended with Polygraf add-in installed alongside and it “shares”
the text report using a wireless network to the user’s mobile devices. At
the same time, an hardware or alternatively software component of Polygraf (see below)
captures the video signal of the presenter’s computer and provides it to the user’s
mobile devices as constantly refreshed images. The transmission of the text report as
well as the presentation screen to user’s mobile devices is continuously made over
a Wi-Fi network. This network connects all devices together and it may be
established with any portable WiFi access point, for instance. There is no need of
Internet connectivity for the network.
Devices of users, such as mobile phones, tablets or laptops (with iOS, Android or Windows operating system) are installed with Polygraf and they receive the content – both the text report and the “mirror” of the presentation screen. The content can be displayed on mobile devices in the ways as follows:
- To display the speech-to-text record only – the record is continuously updated but a user can still browse earlier parts of the text; view of the report is customizable.
- To display the “mirror” of the presentation screen only – the mirrored screen is continuously updated and the view can be customized (magnified and color-inverted).
- To display the mirrored screen in combination with the speech-to-text report as closed captions; the format of both can be customized.
top
Uniqueness of Polygraf
- It does NOT require Internet connection nor its own server.
-
It uses mobile devices what secures a basic user comfort:
- a touch-screen user interface,
- long battery life,
- portability comfort,
- quick and simple start-up.
-
It enables customizable view of the content:
- text reports and closed captions: font family, size and colour, line-spacing;
- presentation screen mirroring: magnification, color-inverting.
-
It is compatible with commonly used soft- and hardware
- MS Word environment (in Windows) used often by speech-to-text reporters;
- any device (as a source of presentation) connected through VGA or DVI interface.
- The software solution also enables to show a report as dynamic closed captions on a speaker‘s computer and thus immediately supplement the speaker‘s presentation when there is one.
- The content can be displayed for individual purpose in hand and for public purposes on a large screen using the handheld device.
Polygraf is based on a technology which can be used independently of commonly considered technical parameters of an environment such as wireless network signal coverage. It only requires access to electricity. All this makes it quite a universal system and as experience showed not only students with disabilities can take an advantage of the service but any users can do, as well.
top
Features of Polygraf
-
Customizable view of a text report
A user may select custom parameters for viewing (font family, size and colour, line-spacing and so on) on a handheld display for a better comfort. -
Communication interface for a transfer of text messages between a reporter and users
It is possible to send text messages from handheld displays to a reporter’s laptop, such as technical comments, further questions about contents of a report (the reporter is the only receiver of the message), or the reporter‘s own contributions to a current discussion (here, the reporter‘s role is a mediator among debating partners). -
Visualization of a speech-to-text report as subtitles
The report is transformed to continuously updated subtitles of up to four lines displayed as closed captions, which can be attached to a speaker‘s presentation. The advantage is that a speech-to-text report and a speaker‘s visual presentation are shown together on one screen. -
Customizable view of mirrored presentation screen
A user may select his preferred ratio of magnification as well as turn on color-inverting to enhance the readability of the mirrored content. -
Optional closed captions added to the mirrored presentation screen
The software enables to display a mirrored screen in combination with a speech-to-text report as closed captions in format which can be customizable.
top
Components of Polygraf
Hardware:
- A laptop or PC with the Microsoft Windows operating system and Microsoft Word editor (version 2003 and later). This computer is for the text reporter.
- A screen capturing device for the presenting speaker – a broadcasting device capturing the video signal and providing it, apart from the output for data projectors, through network interface. (Optionally, this hardware can be substituted by the software – Polygraf Broadcaster.)
- WiFi access point to create a wireless network connecting all devices together and transfer the content among them. There is no need of Internet connectivity for this wireless network.
- Any handheld device with iOS, Android or Windows operating system (typically, a smart phone, tablet or laptop). This device works as a handheld display for users to follow a report and/or mirrored screen.
Software
-
Polygraf Word AddIn (for the reporter’s computer)
This is installed separately alongside the Word, and its purpose is to “share” the text report to the users’ devices. -
Polygraf (for the users’ handheld devices)
It handles the reception of the text report as well as the mirrored screen and it continuously updates and displays according to the user’s custom setting. -
Polygraf Broadcaster (for the speaker’s computer)
The application captures the screen of speaker’s computer and provides it through network to the users’ devices. (Optionally, this software can be substituted by the hardware broadcasting device) -
Polygraf Captions (for the speaker’s computer)
An optional application to display the reported text in the form of continuously updated subtitles displayed as closed captions, which can be attached to a speaker's presentation.
top
Past occasions for presenting Polygraf v2
INSPO 2013, Prague, Czech Republic16 March 2013 |
|
|
The conference INSPO – Internet and Information Systems for Persons
with Special Needs is focused on the use of the Internet and information
technologies in favour of persons with disabilities. Practical use of Polygraf during the conferenceThe system Polygraf was used at the conference for visualizing spoken language of presentations in the plenary hall. For this reason it was necessary to provide a public display of the text transcript on one large projection screen, which displayed an output of an iPod with installed Polygraf system. Polygraf was further used for visualizing the presentation screen in the plenary hall, as well as in two additional conference rooms where parallel sessions were held, using the new Polygraf function of screen mirroring. Participants also had a chance to borrow an iPad device for an individual viewing of the text transcriptions and presentation screen mirroring. Presentation of the system PolygrafThe system Polygraf was introduced in a presentation given by Svatoslav Ondra and Jiøí Tu¾il within the conference plenary session. Further, Polygraf was presented in the form of an exhibition stand in the lounge where the participants had a chance to try it out and to borrow iPad handheld displays for individual viewing of text transcrips of conference presentations and presentation screen mirroring. |
|
AHEAD 2013 – Is Universal Design in Education Any of My Business, Dublin, Ireland12–13 March 2013 |
|
|
AHEAD (Association for Higher Education Access & Disability) is the Irish National Centre of Expertise on inclusive education and works to promote full access to and participation in further and higher education for students with disabilities.
The AHEAD conference organized in 2013 aimed at HEI staff, academics, employers,
students and policy makers with an interest in universal design.
Presentation of the system PolygrafThe system Polygraf was introduced to an international audience within a poster session with a practical demonstration presented by Christoph Damm. |
|
CSUN – 28th Annual International Technology and Persons with Disabilities Conference, San Diego, United States25 February – 2 March 2013 |
|
|
www.csun.edu/cod/conference/2013/sessions/index.php Presentation slides for the CSUN conference, 2013
|
|
3rd International conference Universal Learning Design 2013, Brno, Czech Republic11–15 February 2013 |
|
|
The international conference Universal Learning Design, organized by Masaryk
University, was a collaborative meeting of all who in their practice, i.e.
technologically, organizationally or legally, linguistically, psychologically or
pedagogically, provide for the universal learning design at the university level
for persons with special needs. The conference took place for the 3rd time after
it was first held in 2011, and continued as a special topical section at
ICCHP conference, Linz, Austria. Practical use of Polygraf during the conferenceThe system Polygraf was used at the conference for visualizing spoken language of presentations in two rooms at the same time, in two different languages in one of the rooms – in Czech and English, as the whole conference was simultaneously interpreted from/to English - and in Czech in the other room where simultaneous interpretation from English was visualised. For this reason it was necessary to provide a public display of text transcripts on four large screens that displayed output of four iPod devices with installed Polygraf system. Polygraf was further was used to provide easy access to the presentation screen for sign interpreters, verbal translators and speech-to-text reporters on individual handheld devices using the new Polygraf function of screen mirroring. Participants also had a chance to borrow an iPad device for an individual viewing of the text transcriptions in the language of their preference and presentation screen mirroring. Presentation of the system PolygrafThe system Polygraf version 2 was introduced to an international audience for the first time as a conference paper given by Christoph Damm within the section Linguistic Competence of the Hearing Impaired and the Role of Sign Languages in Tertiary Education. Polygraf was also presented in the form of an exhibition stand in the vestibule where the participants had a chance to try it out and to borrow iPad handheld displays for an individual viewing of text transcriptions of conference presentations and presentation screen mirroring. |
|
top
Past occasions for presenting Polygraf v1
Universal Learning Design @ ICCHP 2012, Linz, Austria11–13 July 2012 |
|
|
ICCHP is a series of biannual scientific conferences in the field of ICT/AT for people with disabilities run in co-operation with the Johannes Kepler University, Linz and the Austrian Computer Society. ICCHP provides a unique platform for end users, researchers, developers and practitioners including amongst others:
Practical use of Polygraf during the conferenceThe system Polygraf was used at the conference for visualizing text transcriptions of presenters' spoken contributions. Polygraf provided a public display of text transcriptions on a large LCD via a single iPod device having Polygraf installed during the topical section ULD and all keynotes of the main ICCHP program, Conference participants had a possibility to borrow an iPad for an individual viewing of the text transcriptions. Presenting the system PolygrafPolygraf was presented in the form of an exhibition stand in the vestibule where the participants had a chance to practically examine it and to borrow iPad hand-held displays for an individual viewing of text transcriptions of conference presentations. |
|
ECOS 2012 – European Congress of Speech-to-text-reporters, Ermelo, Netherlands24–26 August 2012 |
|
|
ECOS is a congress organised by members of the NSV (the Dutch speech-to-text-association)
for speech-to-text-reporters from as many countries as possible. The ECOS congress
took place on 24–26 August 2012 in Ermelo. Presenting the system PolygrafPolygraf was presented in the form of a workshop presented by Petr Peòáz and Christoph Damm. |
|
12th conference INSPO 2012, Prague, Czech Republic17 March 2012 |
|
|
The conference INSPO – Internet and Information Systems for Persons with
Special Needs is focused on the use of the Internet and information technologies
in favour of persons with disabilities. Practical use of Polygraf during the conferenceThe system Polygraf was used at the conference for visualizing text transcriptions of presenters' spoken contributions. Polygraf provided a public display of text transcriptions on a projection screen via a single iPod device, and conference participants had a possibility to borrow an iPad for an individual viewing of the text transcriptions. Presenting the system PolygrafPolygraf was presented in the form of an exhibition stand in the vestibule where the participants had a chance to practically examine it and to borrow iPad hand-held displays for an individual viewing of text transcriptions of conference presentations. |
|
11th conference INSPO 2011, Prague, Czech Republic19 March 2011 |
|
  
The conference INSPO – Internet and Information Systems for Persons with
Special Needs is focused on the use of the Internet and information technologies
in favour of persons with a health impairment. The conference INSPO is organized
by BMI, AISO
and Køi¾ovatka.cz. These associations have continued to organize the conference after
the project March – the month of the Internet ended in 2008. The first eight
years of the conference INSPO took place as a part of the project. Practical use of Polygraf during the conferenceThe system Polygraf was used at the conference for visualizing text transcriptions of presenters' spoken contributions. Polygraf provided a public display of text transcriptions on a projection screen via a single iPod device, and conference participants had a possibility to borrow an iPad for an individual viewing of the text transcriptions. Presenting the system PolygrafThe system Polygraf was introduced in a presentation given by Jiøí Tu¾il within the conference section Information and Communication Technologies for the Deaf. Further, Polygraf was presented in the form of an exhibition stand in the vestibule where the participants had a chance to practically examine it and to borrow iPad hand-held displays for an individual viewing of text transcriptions of conference presentations. |
|
International conference Universal Learning Design 2011, Brno, Czech Republic8–11 February 2011 |
|
  
The international conference Universal Learning Design, organized by Masaryk
University on the occasion of the 10th anniversary of the establishment of
the Support Centre for Students with Special Needs, was a collaborative
meeting of all who in their practice, i.e. technologically, organizationally
or legally, linguistically, psychologically or pedagogically, provide for the
universal learning design at the university level for persons with special
needs. Practical use of Polygraf during the conferenceThe system Polygraf was used at the conference for visualizing spoken presentations in two rooms at the same time, in two different languages in each of the rooms – in Czech and English (the whole conference was simultaneously translated from/to English). For this reason it was necessary to provide a public display of text transcriptions on four large screens via four iPod devices. Participants also had a chance to borrow an iPad device for an individual viewing of the text transcriptions in the language of their preference. Presentation of the system PolygrafThe system Polygraf was introduced to an international audience for the first time as a conference presentation given by Jiøí Tu¾il within the section Linguistic Competence of the Hearing Impaired and the Role of Sign Languages in Tertiary Education. Besides, it was presented individually to several speech-to-text reporters and other conference participants. |
|
top
Appendices to Download
- Polygraf Word AddIn (a component for the reporter’s computer)
- Polygraf (a component for the users’ handheld devices)
-
Polygraf Captions (for the speaker’s computer)
- for Windows XP/Vista/7 operating system (e.g. laptops with these operating systems) – betaversion
- Polygraf Broadcaster (for the speaker’s computer)
- Documentation
top
Èesky