Personalized Object-Based Audio for Hearing Impaired TV Viewers: new paper published (open access)

 

For a long time now Ofcom and broadcasters have received complaints that speech on TV can be difficult, or impossible, to understand. The problem is of course much worse for viewers who with even quite mild hearing loss. Reasons are varied and well described in a recent Conversation article from one of our researchers, Lauren Ward. Causes include unfamiliar accents, unclear speaking from actors, excessive background sound effects or music and even occasionally badly recorded location audio.

Recent highly publicised complaints about Poldark, Happy Valley and SS-GB, and earlier, Jamaica Inn and Wonders of the Universe have generated something of a media storm over the issue which shows no sign of abating and was debated recently in the House of Lords.

At the Acoustics Research Centre at University of Salford we’ve been working on solutions to these problems for a long time, currently a lot of research is looking at how new object-based audio formats can solve some of these problems and make TV sound more accessible. Object-based audio has the potential to allow individual personalisation of TV sound based the viewer’s preferences or needs. Our most recent work, carried out using the DTS MDA object-based audio format can be found in the Journal of the Audio Engineering Society article here. It is freely available as an open access publication – please feel free to read and comment. You can find more details of our accessible audio work on our blog here.

IBC 2015: TV Sound for Hearing Impaired People

IBC 2015 Demonstration of Object Based Clean Audio

The problems of hearing impaired people watching TV have been well documented of late. Loud music, background noise and other factors can ruin the enjoyment of TV for many people with hearing loss – around 10 million people in the UK according to Action on Hearing Loss.

In previous research funded by the ITC and Ofcom I looked at solutions that took advantage of the (then) recent introduction of 5.1 surround sound broadcast. Some of this ended up in broadcast standards and is being used by broadcasters. Now emerging new audio standards are opening the door to improving TV sound much more for hearing impaired people, and also for many others.

I’ve written about some of this work before, a recent blog post described our journal article in the Journal of the Audio Engineering Society where my colleague Rob Oldfield and I picked up where my PhD left off and looked at how we could improve TV sound for hearing impaired people by using features of emerging object-based audio formats. In object-based audio all component parts of a sound scene are broadcast separate and are combined at the set top box based on metadata contained in the broadcast transmission. This means that speech, and other elements important to understanding narrative, can be treated differently compared to background sound (such as music, noise etc).

I’ve just returned from IBC in Amsterdam where we’ve been demonstrating some University of Salford research outputs on object-based clean audio with DTS, a key player in object-based audio developments.

IBC 2015: The largest global electronic media and entertainment show in Amsterdam last week.

Object-based Clean Audio at IBC 2015

Last week we spent a week showing the results of our recent collaboration with DTS – presenting personalised TV audio and

Trust me, I’M A DOCTOR!

Well I finally did it. PhD viva is now over. Passed. Subject to some minor corrections I’m a doctor!

After a lot of years of work, and three years of interruption in the middle to work on our MediaCityUK campus development, I finally submitted my thesis on ‘Improving Television Sound for People with Hearing Impairments’ just before Christmas and defended it successfully in a viva last Wednesday. Hence the lack of posts here, if I was going to write it had to be proper work! It’s more than a nice feeling, very very good indeed. Definitely a relief. I’ve spent all weekend celebrating and enjoying the congratulations and good wishes of family, friends and colleagues.

To cap the weekend off a parcel arrived in the post today from Wiley. Complementary copies of Media Production, Delivery and Interaction for Platform Independent Systems: Format-Agnostic Media, the book I co-edited and co-authored with colleagues on the FascinatE project last year. Ha! 2014 is shaping up very nicely.

So what comes next? I’m fortunate enough to

‘The future of television?’ FascinatE: The Final Demonstration

This week has seen the final demonstration of research developed over three and a half years of  the FascinatE EU FP7 research project. The project has developed a complete end-to-end future broadcast system which combines ultra high definition panoramic video, 3D ambisonic and object based audio, new methods for delivery of interactive AV content and new interfaces and methods to interact with the AV media at the user end. It’s been my pleasure to lead University of Salford’s part of the project and this week, to host the final demonstration of the project.

We hosted the final demonstration event at our MediaCityUK building – it’s one of the few places that could actually support what we were trying to do, the infrastructure of the building was actually designed for this kind of thing but we pushed it pretty hard this week. FascinatE partners worked through

Through the Looking Glass: return to the ETC

Over the last couple of years I have written a little about my previous visits to the Entertainment Technology Center at Carnegie Mellon University here and here, a few things have changed since then so this visit was an interesting one in a number of ways. First of all Don Marinelli has now retired. As Don was one of the two founders of the ETC this leaves something of a hole in the place: Don is something of a superstar – inspirational speaker, larger than life character and a truly gifted teacher with a tight grasp on how learning works. A good friend too. Nevertheless, the ETC is bigger than any one man and Don and Randy Pausch provided enough momentum that I suspect their ongoing project is pretty much unstoppable now.

Also, last semester we had the pleasure of hosting one of the ETC’s project teams here at MediaCityUK, (I’ll document that in another post soon – complications around a parallel project meant that I’ve been obliged to keep quite about it till now). This was my first return visit since hosting the project at MediaCityUK and I was here to do more workshops on 3D audio and how ETC projects could use it in game development. The brilliant Dave Purta, the ETC tech support, made sure all the hardware was set up for me and ready to roll so I arrived with a couple of laptops on the audio evangelist tour pt II.

A nice welcome when I arrived

Last year I showed

FascinatE project 2nd review

I’ve just returned from an intense week at the FascinatE EU FP7 project review  at Fraunhofer HHI, Berlin, you can find some detail of the project here. This is our second annual review by the European Commission review team, I blogged about the first one at the time. As part of the review we’ve demonstrated a bunch of components from the project, many of which are now integrated after a great deal of development work over the last year and some very long working days and nights over the last couple of months.

This time we’ve been pretty ambitious and ran the review in parallel with a first public demonstration of the system combined with a test shoot at the Arena concert hall in Berlin. The test shoot is

Live football broadcast – OB at Eastlands

After a chance meeting at BVE recently I was invited by Ian Rosam to join Sky’s outside broadcast audio crew to see how they cover live football so found myself at Manchester City vs Blackburn at Eastlands on Saturday afternoon. Really useful visit from a couple of angles for me: one is for the Fascinate Project, the other is my current teaching in digital control in audio. Also always good to develop links with the industry we are sending our graduates out into.

The FascinatE project I’ve talked about once or twice before – we’re developing an end to end ‘future broadcast’ system for live events and as part of this we covered a Premier League game with 180 degree hi resolution video and 3D sound at Stamford Bridge thanks to the good people at Chelsea Football Club and friends at SISLive, the outside broadcaster. Saturday was a bonus as Sky have slightly different methods and equipment compared to SIS and it was another opportunity to get first hand knowledge of how live sports broadcast is done. Also

Audio Flipboard anyone?

Listen: Sign on BBC Building at MediaCityUK

“ I think 2012 will be the year that audio in social really explodes.“

When I first got my hands on an iPad Flipboard was a revelation – previously I dug into Twitter and Facebook chronologically and Flipboard’s magazine layout and easy-to-browse nature made it a much more random and fun process. I’d spot things I hadn’t before but more interestingly, I was reading social media streams like I’d read a newspaper and not at my desk with a computer.

While reading Katie Moffat’s predictions for 2012 I was struck by, “ I think 2012 will be the year that audio in social really explodes.”

I commented on the post that, “ it really needs a platform that will manage audio well. If we had something that would do to podcasts and AudioBoo what FlipBoard does for text you’d have yourself a custom radio magazine show, it’d be awesome! I’d listen every morning on the daily commute. ”

The post made me think, there must be a way of aggregating audio content, esp social content in a similar vein to FlipBoard. I’d be interested in knowing if anyone has done this effectively yet BECAUSE I WANT IT! And if it’s not being done, looking further into developing ideas around it. Would make an interesting project for some of our bright students perhaps. Shuffler.fm are doing it for music but I haven’t yet found an equivalent for spoken word….

Games People Play: a post from the ETC

The Last Message - A Kinect platform-game that tells a story of the journey of a waveform. The player controls the the amplitude and frequency of the waveform using both arms, and must guide it to the broadcasting tower — with Brian Lee, Ruijie Liu, Shibli Mansuri and Scott Chen. From the Fall 2011 BVW Show

As promised, this post comes from Pittsburgh where I’m working for a week as a guest of Carnegie Mellon University’s Entertainment Technology Center (ETC).

The inimitable Don Marinelli, Executive Producer of the ETC

The ETC was created by the inimitable Don Marinelli and the late Randy Pausch combining skills in drama and computer science to produce game designers and “entertainment technologists”. It is a unique place that I have written about before. Students here study on a largely project based masters programme that aims to produce graduates that thrive on cross disciplinary team based project work. 

I’ve been invited here to do some workshops on 3D audio and to inspire students to incorporate some of the work we do at the Acoustics Research Centre at Salford into their projects and

Social Media and learning: why?

Twitter: Come to the dark side (pic Dot D)

I’ve just been reading through some of our BSc Professional Sound and Video Technology student blogs and having read @JustPressPlay_‘s post A few thoughts.. « JDTWerk I started commenting on it and the comment got long enough for a new post on here. So here it is, gets me back on track anyway as I’ve been getting busy again and the blogging is the first thing to drop off the to do list.

Firstly it’s a nice post and resonates with me as I didn’t ‘get’ Twitter for a long time. Until it starts being useful to you it’s just so much noise and there’s plenty of mainstream media belittling the platform as pointless. I have a lot of sympathy for the students who just don’t want to engage with it. It doesn’t necessarily seem core to what they feel they should be learning. It’s worth persisting though.

In the end I was forced onto the platform by my students who were