Weston Ruter

Web application developer in Portland, Oregon

HTML5 Audio Read-Along

Published December 22nd, 2009. Updated December 23rd, 2009.
By

Jump to demo below to listen while reading along to the nativity story from the Gospel of Luke.

I guess I’ve been on an HTML5 Audio kick lately, with this and my previous post on using it to play Google’s Text-To-Speech (TTS) service on Google Translate. I wrote that post while attempting to build a text-to-speech interface which audibly reads the text on the page while highlighting the currently-spoken word (like my old favorite ReadPlease), but a bug in Google Chrome for Mac discouraged me from finishing it. However, when I saw the inestimable Paul Irish’s post about the jquery-singalong plugin which times HTML5 Audio with text and a bouncing ball cue, I was inspired to do the same but for a (non-TTS) read-along.

ESVThe text I chose was the nativity story from the Gospel of Luke in the English Standard Version (ESV), not only in keeping with the (true) Christmas spirit but also because the ESV Online has an excellent API which allows both a passage’s text and its audio to be queried. With the text and audio in hand, each of the words in the text needs to be time-indexed for its begin time and duration in the corresponding audio. In the past, audio Bibles were divided into chapter segments only and that was as granular as you could go; the ESV team did the innovation of taking this granularity down to the verse-level. Unfortunately, however, the granularity is not available at the word-level. Therefore, in order to make this exciting (IMHO) read-along demo work, I manually traversed the audio to find each word’s begin time and duration (spending more time than I care to say), and I added these time indicies to the word markup as data-begin and data-dur attributes, akin to SMIL’s begin and dur attributes (which I suppose I could use). I also spruced up the markup for the passage by incorporating some HTML5 constructs as well as adding OSIS verse elements as I’ve been talking about over at Open Scriptures.

The read-along demo has been tested in Firefox 3.5, Chrome 4, and Safari 4 (as expected, it will not work in Internet Explorer). Chrome plays the MP3 as served from the ESV API. Firefox doesn’t support MP3 so I include an OGG Vorbis source as well. Safari seems to have trouble playing the MP3 and can’t play OGG, so I also include an 8kHz WAV source as well.

What it does: Upon playing the audio, the word corresponding to the one currently being spoken is highlighted (via DOM Range/Selection); this eliminates the need to re-find your place if you momentarily look away. Likewise, when manually adjusting the seek position, the words which correspond to each position sought will be highlighted; and conversely, clicking a word causes the audio to seek to its corresponding position (and double-clicking will then cause it to start playing). Thus the text itself serves as an interface for navigating the audio. See the inline JavaScript source code for all the magic.

Without further adieu, please enjoy listening while reading along to the Christmas story!

Luke 2:1-20 (ESV)

In those days a decree went out from Caesar Augustus that all the world should be registered. This was the first registration when Quirinius was governor of Syria. And all went to be registered, each to his own town. And Joseph also went up from Galilee, from the town of Nazareth, to Judea, to the city of David, which is called Bethlehem, because he was of the house and lineage of David, to be registered with Mary, his betrothed, who was with child. And while they were there, the time came for her to give birth. And she gave birth to her firstborn son and wrapped him in swaddling cloths and laid him in a manger, because there was no place for them in the inn.

And in the same region there were shepherds out in the field, keeping watch over their flock by night. And an angel of the Lord appeared to them, and the glory of the Lord shone around them, and they were filled with fear. And the angel said to them, Fear not, for behold, I bring you good news of great joy that will be for all the people. For unto you is born this day in the city of David a Savior, who is Christ the Lord. And this will be a sign for you: you will find a baby wrapped in swaddling cloths and lying in a manger. And suddenly there was with the angel a multitude of the heavenly host praising God and saying,

Glory to God in the highest,
and on earth peace among those with whom he is pleased!

When the angels went away from them into heaven, the shepherds said to one another, Let us go over to Bethlehem and see this thing that has happened, which the Lord has made known to us. And they went with haste and found Mary and Joseph, and the baby lying in a manger. And when they saw it, they made known the saying that had been told them concerning this child. And all who heard it wondered at what the shepherds told them. But Mary treasured up all these things, pondering them in her heart. And the shepherds returned, glorifying and praising God for all they had heard and seen, as it had been told them.

Scripture taken from The Holy Bible, English Standard Version. Copyright ©2001 by Crossway Bibles, a publishing ministry of Good News Publishers. Used by permission. All rights reserved. Text provided by the Crossway Bibles Web Service.

Comments

  1. Rui Lopes

    Interesting demo, but I do have to say that we did this at my HCI research group 3-4 years ago, but with HTML+TIME (IE only, though). Check the “Rich Content Books for All” project at http://hcim.di.fc.ul.pt/wiki/RiCoBA for more details.

    Cheers

  2. Jesse Griffin

    Very cool Weston! I especially like the seek-via-word-click feature.

  3. Joshua Clark

    I wanted to build something like this for sermon transcripts. Upload a sermon mp3, and receive a time-coded transcript, instantly searchable. I guess this would be STT, not TTS. But once transcribed, this would be the perfect front-end.

Leave a Comment