TapVolume Levelling and Replay Gain

4 Feb 2011 13:38 by Rick

Ever since recording began there has always been the problem of varying volume levels—with a big horn gramophone you sat closer, with record players you kept getting up to adjust the volume. Sometimes this was the fault of lazy recording engineers but often it was inherent in the media; the peaks had to be limited to avoid clipping (or worse on vinyl) whilst still maintaining a respectable amplitude bandwidth. With digital media there should have been a solution at source. The bandwidth stretches below the threshold of hearing but recordings are usually made with the peak just a few dB off the maximum. The impact of this is that recordings with a wide dynamic range such as acoustic and classical music sound very quiet when compared to highly compressed pop music (compressing pop is an issue for another time).

DJs learn to “ride the fader” to keep the apparent volume steady over a session. At home we have to resort to the remote, but there is a solution with media that you have control over—or at least there should be. Digital formats such as MP3, WMA, Flac and AAC have developed mechanisms to adjust the decoder output to a user defined level. A good description of the technique is described on the MediaMonkey FAQ pages (scroll down to the last chapter).

The three different methods are

1. Recode the audio to the level you require; this is potentially destructive as information (and hence quality) is lost each time you do it and it is not reversible so few systems employ this method.

2. Code the reference level on each audio frame so that standard decoders can interpret them. This is the method used by MP3gain and MediaMonkey “Level Playback Volume” and is the most universally successful. However there is some doubt over whether it is truly reversible without loss. It is also not possible with files protected by rights management (DRM).

3. Code the adjustment required in the metadata without touching the audio stream. This truly does not lose any information but requires support from the player to interpret the tags. It is supported, amongst others by WMP “Volume Levelling”, MediaMonkey “Analyze Volume” and iTunes “Sound Check” and by players to varying degrees.

The way they all calculate what adjustment to make goes by the grand name of Psychoacoustic Analysis to judge how loud the listener perceives the music to be. The two ways to do this are by track, what used to be called “Radio Levelling” and is what you would want if you were a DJ or playing a party mix; and by album which used to be called “Audiophile Levelling.” This preserves the relative volume of the tracks on an album to respect the artist/engineer’s requirements and is what you would want if listening to a symphony with each movement as a separate track.

The problem

The definition of many of the metadata tags for method 3 are not standardised, particularly for the most common format—MP3. Different encoders do it in different ways which means that players have to decide which, if any they support. This is partly as a consequence of multiple independent tagging systems.

Replay Gain

The first format to adopt what it called Replay Gain was Flac (Ogg Vorbis) and that is (nearly) standardised using the tags REPLAYGAIN_TRACK_PEAK, REPLAYGAIN_TRACK_GAIN & REPLAYGAIN_ALBUM_GAIN. There is also a REPLAYGAIN_ALBUM_PEAK which most encoders ignore. The loose point in the standard is that it specifies a reference volume of 83dB above the threshold of hearing whereas everyone now accepts (and implements) 89dB as a better level to avoid stretching the analogue amplification too far. The technical description for the standard proposed is

…the ReplayGain tags stored in the files are 6dB above the gain adjustments required to make the files “sound as loud” as a -20dB RMS pink noise signal when replayed in an SMPTE RP 200 calibrated system. The -20dB RMS pink noise signal will measure 83dB [89dB] SPL at the listener’s seat in such a system.

http://www.hydrogenaudio.org/forums/lofiversion/index.php/t83397.html

Encoder/Taggers such as MediaMonkey also use these as sub-tags of the TXXX {user defined text information} ID3v2 tag in MP3 files. They have the advantage that they are easy to read.

Windows Media Player

It is predictable that the proprietary organisations should do it differently. “Volume Levelling” has a system for its own WMA format using PeakValue and AverageLevel tags (these are “track gain” only) which it also uses to code MP3 files as sub-tags of the PRIV {Private} ID3v2 tag. It is coded in binary. I have seen reported elsewhere on the internet that WMP uses WM/WMADRCAverageReference, WM/WMADRCPeakReference, WM/WMADRCAverageTarget, and WM/WMADRCPeakTarget tags but I can’t find evidence for this in my files. What ever it does, WMP does it very slowly just like its collection of other meta data.

Apple iTunes

“Sound Check” is different again. I can’t analyse AAC files as I can’t find a structure definition document but for MP3 files it writes an iTunNORM sub-tag of the COMM {Comment} ID3v2 tag. There are 40 bytes of binary data in there but what they mean I haven’t discovered.

LAME

Surprisingly, this very popular public domain encoder also uses a unique system called the MP3 INFO tag. Replay Gain uses bytes 167-174 (not 175 as the documentation says) of the tag coded in binary—Track Peak Amplitude (4 bytes floating point), Track Gain (2 bytes), Album Gain (2 bytes). The format of the latter two is as follows—3 bits; type code, 000=Not Set, 001=Track, 010=Album. 3 bits originator code; 000=Unspecified, 001=set by producer, 010=set by user, 011=calculated automatically. 1 bit: sign. 9 bits; value * 10.

RGAD

As well as the Vorbis type tags, MediaMonkey also writes an MP3 ID3v2 tag called RGAD {Replay gain adjustment} with 8 bytes of data supporting both track and album gain and I think there was some intention to get this standardised but I see no sign of it. The format (inside the tag) is the same as the LAME data described above.

What now

My immediate requirement is for my Sonos system to play at the correct volume. Sonos supports WMP tags for WMA & MP3, iTunes tags for AAC & MP3 and the standard tags in Flac files. It only supports “track gain” (and, as I have discovered by experiment, only supports negative values, so it will lower the volume but not raise it). What I require is “album gain” on Flac and MP3 files not written by the proprietary systems. What I need is a method to write either iTunes or WMP type tags based on the MediaMonkey ones. As a start I am working on a MediaMonkey plugin that first saves the “track gain” in a custom field for safety then copies the “volume gain” to the “track gain” field to fool the player into supporting audiophile mode. To do the rest of the job I will need to discover what the binary means in the WMP or iTunes tags.

[Edited 4 Jan 2012] to add information obtained from ReplayGain legacy metadata formats (with thanks).

TapThe Word of the Lord

1 Feb 2011 23:31 by Rick

Lyrics by Stephen Schwartz, Music by Leonard Bernstein

You can hear (and see) it sung here—the song is in English despite the narrative being in German.

You can lock up the bold men.
Go and lock up your bold men and hold them in tow.
You can stifle all adventure for a century or so.

Smother hope before it is risen,
watch it wizen like a gourd,
but you cannot imprison the Word of the Lord.
No, you cannot imprison the Word of the Lord.

For the Word,
for the Word was at the birth of the beginning,
it made the heavens and the earth and set them spinning.
And for several million years,
it withstood all our quorums and fine ideas.
It’s been rough,
it’s been rough but it appears to be winning.

There are people who doubt it.
There are people who doubt it and shout it out loud.
There are local, vocal, yokels who we know collect a crowd.

They can fashion a rebuttal that’s a subtle as a sword,
but they’re never going to scuttle the Word of the Lord.
No, they’re never going to scuttle the Word of the Lord.

All you big men of merit,
all you big men of merit who ferret out flaws,
you rely on our compliance with your science and your laws.

Find a freedom to demolish while you polish some award,
but you cannot abolish the Word of the Lord.
No you cannot abolish the Word of the Lord.

For the Word,
for the Word created mud and got it going.
It filled our empty brains with blood and set it flowing.
And for thousands of regimes,
it endured all our follies and fancy schemes.
It’s been tough,
it’s been tough and yet it seems to be growing.

Oh you people of power,
oh you people of power, your power is now.
You may plan to go forever but you never do somehow.

So you wait in silent treason until reason is restored,
and we wait for the season of the Word of the Lord.
We await the season of the Word of the Lord.

We wait,
we wait for the Word of the Lord.

I hope that is right, I wasn’t sure about a couple of words. I was working from the 1971 recording, I hear that the newer one on the video is slightly different.

TapAn’ damn’d Excise-men in a bussle

18 Jan 2011 17:48 by Rick

All the talk this morning is about the cost of alcohol and I wondered how the tax is calculated—it is complicated. The quote in the title is to reflect that it is close to Burns Night and we are not able to get to the dinner :(

The general principle is that the tax (excise duty) on beer and spirits is based on the alcohol content—which makes sense. The duty on all other drinks is charged at various different flat rates—which doesn’t. The reasons are largely political. Here is a simplified table with the current rates…

Beer 17.32p/Ltr/%Alc
Still cider 1.2%–7.5% 33.46p/Ltr
Still cider 7.5%–8.5% 54.04p/Ltr
Sparkling cider 1.2%–5.5% 33.46p/Ltr
Wine & alcopops 1.2%–4% 69.32p/Ltr
Wine & alcopops 4%–5.5% 95.33p/Ltr
Still wine & alcopops 5.5%–15% 225.00p/Ltr
Sparkling wine, cider & alcopops 5.5%–8.5% 217.83p/Ltr
Sparkling Wine & alcopops 8.5%–15% 288.20p/Ltr
Wine & alcopops 15%–22% 299.97p/Ltr
Wine & alcopops >22%, spirits, all spirit based alcopops 23.80p/Ltr/%Alc

This duty is charged at source to the manufacturer. On top of this VAT, currently 20%, is charged on the retail sale price.

So lets do some examples (the minimum price assumes that it costs nothing to manufacture & ship and the end customer pays all the tax).

3.5% beer—duty 34.5p per pint—Minimum over the counter price 42p
Flandres 2.6% gnats—duty 11p per 250ml bottle—Minimum price 14p
Special Brew 9% fighting juice—duty 78p per 500ml can ==> Minimum price 94p
A nice 14.5% Rioja—duty £1.69 per 75cl bottle—Minimum price £2.03
Budget 30% vodka—duty £5 per 70cl bottle—Minimum price £6.00
Bacardi 151 death wish—duty £17.97 per litre—Minimum price £21.56

The news today was that there should be a minimum retail price for beer set at 21p per unit of alcohol (1% in 1 litre) and spirits at 28p per unit. With the same examples as above we get

3.5% beer—pint—Minimum over the counter price 42p
Flandres 2.6% gnats—250ml bottle—Minimum price 14p (costs 28p in Sainsbury’s)
Special Brew 9% fighting juice—500ml can ==> Minimum price 95p (£1.57 in Sainsbury’s)
A nice 14.5% Rioja—75cl bottle—Minimum price £2.28 (assuming the beer rate)
Budget 30% vodka—70cl bottle—Minimum price £5.88
Bacardi 151 death wish—litre—Minimum price £21.14

i.e. identical for beer and close for spirits. When there was a big fuss about this last summer I did see basic lager on sale for less than 50p per litre. 3% lager would now be at least 63p.

TapSkype Newsletter?

12 Jan 2011 12:47 by Rick

If you see an email announcing itself as from “Skype Newsletter”, check very carefully. Most, if not all of these are bogus. They are spammers trying to get your login details and maybe get you to pay for an upgrade to a free product or install malware on your system.

TapMac App Store

10 Jan 2011 12:45 by Rick

At a rough count I have 32 third party applications installed on my Mac (not including stuff that came with my accessories like camera, GPS and printer). Two of those are paid-for applications, a few are donation-ware. Just one is now supported by the new Mac App Store, TextWrangler, but it doesn’t say so on the product web site.

The only benefit that I have seen from the store is that I may buy Pages now that is is available as a separate download (but curiously it doesn’t say that on the product web site either!)

SuperDuper, Springy, Crossover, CyberDuck, Sbooth Max, Firefox, SourceSource, SoundFlower, SoundApp Reborn, LineIn, LAME, Growl, KeePassX, Perian, Flip4Mac, XNJB, TextWrangler, NeoOffice, Gimp, PaintBrush, Adium, PopFile, Skype, Sonos, Thunderbird, uTorrent, VMware Fusion, Adobe Air, BBC iPlayer, Adobe Flash, Audacity, SilverLight

TapThe Battle of Gloucester Cathedral

14 Dec 2010 14:46 by Rick

The Spectator reports

Annabel Hayter, chairwoman of Gloucester Cathedral Flower Guild, received an email saying that she and her 60 fellow flower arrangers would have to undergo a CRB check. CRB stands for Criminal Records Bureau, and a CRB check is a time-consuming, sometimes expensive, pretty much always pointless vetting procedure that you must go through if you work with children or ‘vulnerable adults’. Everybody else had been checked: the ‘welcomers’ at the cathedral door; the cathedral guides; the whole of the cathedral office (though they rarely left their room). The flower guild was all that remained.

The cathedral authorities expected no resistance. Though the increasing demand for ever tighter safety regulation has become one of the biggest blights on Britain today, we are all strangely supine: frightened not to comply. Not so Annabel Hayter. ‘I am not going to do it,’ she said. And her act of rebellion sparked a mini-revolution among the other cathedral flower ladies. In total she received 30 letters from guild members who judged vetting to be either an invasion of privacy (which it certainly is) insecure (the CRB has a frightening tendency to return the wrong results) or unnecessary (they are the least likely paedophiles in the country). Several threatened to resign if forced to undergo it.

Follow the rest of the story for other examples of the CRB cancer. Thanks to Schneier on Security for the pointer.

TapGawker Media hacked

10:11 by Rick

I have heard today that this prominent publishing house, which includes Lifehacker, Gizmodo (and, ahem, Fleshbot), has had its database of accounts compromised. There is not much point in changing your password there yet, but, if you use the same one elsewhere, it would be a good idea to change them NOW.

Thanks to SANS ISC for the information. Lots more information here.

TapIs this the future of Home Computing?

8 Dec 2010 15:15 by Rick

Disclaimer: Although I work for HP, I have no contact with the division responsible for this product. All views expressed are my own.

I have always thought that home computing has never got out of the hobbyist mindset of the Sinclair Spectrum and BBC home Micro. They require deep knowledge to get the best out of them, require regular maintenance and are forever going wrong in little (how do I do that) or big (BSOD) ways. This puts a lot of people off. To really take off as a universal resource, it needs to be a commodity like your TV; you just plug it in and go. Is this possible for a computer—it is if you stop thinking of it as a computer, that was out of date at the turn of the century. People use these devices to create and read documents, do research, consume entertainment in the form of pictures, video, music and games—only the odd spread sheet could be called computing. So, look at this…

HP recently launched a product called the DreamScreen 400 but you may not have heard of it because it was developed in India for the Indian market. It is well worth taking a look at because it is quite revolutionary and yet derivative at the same time.

It is very targeted to those homes that do not have computer technology already. It is very low cost—less than £300 and does not assume any prior knowledge of computers or even the internet, that is all hidden away. I have no information about the storage mechanism, but I would hope that it is transparent like the iPad, not requiring any understanding of folders.

It is uses standard components which is how the cost is kept down—18.5″ TouchSmart screen, probably an Intel processor, internet connection, webcam & microphone, SD card slot, DVD drive, 4 x USB sockets and probably a Linux operating system with a custom UI bolted on top. I have seen no mention of a wireless connection, DVD burner or what storage capacity but the specification is pretty comprehensive. There is multiple account capability for each member of the family to be kept separate and, presumably, some sort of parental control system. The applications are clearly targeted at the growing middle class of India with emphasis on education, entertainment and running a home. They include eMail, video chat, downloaded and DVD video, music, games, documents, spreadsheets, presentations (I don’t know what formats), news, finance, travel and educational material geared to the Indian curriculum. For the more adventurous there is generic web browsing using Firefox with custom extensions.

The interface is very iPad like, including touch scrolling, and you could say that it is a large screen, permanently docked iPad. There is a mouse, and I think arm-ache would make this almost essential, and a USB keyboard. The latter would only be really required if you wanted to create documents, there is a multi-language on-screen keyboard for simple things like search terms. You can also attach certain (HP only?) USB printers.

This has clearly been engineered to a price with a limited development teams but with a rich array of “Apps”, including third-party developers, I could see this really fulfilling the requirements of any family even in the western world—Will HP pull it off or someone else like Apple with a iMac/iPad hybrid? That remains to be seen.

Update: [14 Dec] I was walking around Maplin and spotted on the discount shelf an MSI Wind Top AE1900. This is so similar to the DreamScreen 400 that the latter is surely just re-badged and with custom software. The MSI came with Wireless-N and is/was almost twice the price.

TapAdvent

30 Nov 2010 17:23 by Rick

I haven’t really noticed the NT reading for Advent Sunday before so don’t know if it is always the same but this seems very appropriate for the weeks leading up to Christmas.

Romans 13:13-14

let us live honourably as in the day, not in revelling and drunkenness, not in debauchery and licentiousness, not in quarrelling and jealousy. Instead, put on the Lord Jesus Christ, and make no provision for the flesh, to gratify its desires.

TapThe Portishead Traffic Light Experiment

1 Nov 2010 11:43 by Rick

As you can see in this video, the experiment to switch off a notorious set of traffic lights in Portishead was an outstanding success. I missed this at the time but, although it clearly needs a bit of work, it was good enough for the experiment to be made permanent. Since then two zebra crossings have been introduced making it easier for disabled people, but in an unfortunate retrograde step, they have been put on raised “traffic calming” platforms. Now we just need the experiment to be repeated in Bristol itself—at sensible and appropriate locations.