Archive for the ‘Technical’ Category

TapVolume Levelling and Replay Gain

4 Feb 2011 13:38 by Rick

Ever since recording began there has always been the problem of varying volume levels—with a big horn gramophone you sat closer, with record players you kept getting up to adjust the volume. Sometimes this was the fault of lazy recording engineers but often it was inherent in the media; the peaks had to be limited to avoid clipping (or worse on vinyl) whilst still maintaining a respectable amplitude bandwidth. With digital media there should have been a solution at source. The bandwidth stretches below the threshold of hearing but recordings are usually made with the peak just a few dB off the maximum. The impact of this is that recordings with a wide dynamic range such as acoustic and classical music sound very quiet when compared to highly compressed pop music (compressing pop is an issue for another time).

DJs learn to “ride the fader” to keep the apparent volume steady over a session. At home we have to resort to the remote, but there is a solution with media that you have control over—or at least there should be. Digital formats such as MP3, WMA, Flac and AAC have developed mechanisms to adjust the decoder output to a user defined level. A good description of the technique is described on the MediaMonkey FAQ pages (scroll down to the last chapter).

The three different methods are

1. Recode the audio to the level you require; this is potentially destructive as information (and hence quality) is lost each time you do it and it is not reversible so few systems employ this method.

2. Code the reference level on each audio frame so that standard decoders can interpret them. This is the method used by MP3gain and MediaMonkey “Level Playback Volume” and is the most universally successful. However there is some doubt over whether it is truly reversible without loss. It is also not possible with files protected by rights management (DRM).

3. Code the adjustment required in the metadata without touching the audio stream. This truly does not lose any information but requires support from the player to interpret the tags. It is supported, amongst others by WMP “Volume Levelling”, MediaMonkey “Analyze Volume” and iTunes “Sound Check” and by players to varying degrees.

The way they all calculate what adjustment to make goes by the grand name of Psychoacoustic Analysis to judge how loud the listener perceives the music to be. The two ways to do this are by track, what used to be called “Radio Levelling” and is what you would want if you were a DJ or playing a party mix; and by album which used to be called “Audiophile Levelling.” This preserves the relative volume of the tracks on an album to respect the artist/engineer’s requirements and is what you would want if listening to a symphony with each movement as a separate track.

The problem

The definition of many of the metadata tags for method 3 are not standardised, particularly for the most common format—MP3. Different encoders do it in different ways which means that players have to decide which, if any they support. This is partly as a consequence of multiple independent tagging systems.

Replay Gain

The first format to adopt what it called Replay Gain was Flac (Ogg Vorbis) and that is (nearly) standardised using the tags REPLAYGAIN_TRACK_PEAK, REPLAYGAIN_TRACK_GAIN & REPLAYGAIN_ALBUM_GAIN. There is also a REPLAYGAIN_ALBUM_PEAK which most encoders ignore. The loose point in the standard is that it specifies a reference volume of 83dB above the threshold of hearing whereas everyone now accepts (and implements) 89dB as a better level to avoid stretching the analogue amplification too far. The technical description for the standard proposed is

…the ReplayGain tags stored in the files are 6dB above the gain adjustments required to make the files “sound as loud” as a -20dB RMS pink noise signal when replayed in an SMPTE RP 200 calibrated system. The -20dB RMS pink noise signal will measure 83dB [89dB] SPL at the listener’s seat in such a system.

http://www.hydrogenaudio.org/forums/lofiversion/index.php/t83397.html

Encoder/Taggers such as MediaMonkey also use these as sub-tags of the TXXX {user defined text information} ID3v2 tag in MP3 files. They have the advantage that they are easy to read.

Windows Media Player

It is predictable that the proprietary organisations should do it differently. “Volume Levelling” has a system for its own WMA format using PeakValue and AverageLevel tags (these are “track gain” only) which it also uses to code MP3 files as sub-tags of the PRIV {Private} ID3v2 tag. It is coded in binary. I have seen reported elsewhere on the internet that WMP uses WM/WMADRCAverageReference, WM/WMADRCPeakReference, WM/WMADRCAverageTarget, and WM/WMADRCPeakTarget tags but I can’t find evidence for this in my files. What ever it does, WMP does it very slowly just like its collection of other meta data.

Apple iTunes

“Sound Check” is different again. I can’t analyse AAC files as I can’t find a structure definition document but for MP3 files it writes an iTunNORM sub-tag of the COMM {Comment} ID3v2 tag. There are 40 bytes of binary data in there but what they mean I haven’t discovered.

LAME

Surprisingly, this very popular public domain encoder also uses a unique system called the MP3 INFO tag. Replay Gain uses bytes 167-174 (not 175 as the documentation says) of the tag coded in binary—Track Peak Amplitude (4 bytes floating point), Track Gain (2 bytes), Album Gain (2 bytes). The format of the latter two is as follows—3 bits; type code, 000=Not Set, 001=Track, 010=Album. 3 bits originator code; 000=Unspecified, 001=set by producer, 010=set by user, 011=calculated automatically. 1 bit: sign. 9 bits; value * 10.

RGAD

As well as the Vorbis type tags, MediaMonkey also writes an MP3 ID3v2 tag called RGAD {Replay gain adjustment} with 8 bytes of data supporting both track and album gain and I think there was some intention to get this standardised but I see no sign of it. The format (inside the tag) is the same as the LAME data described above.

What now

My immediate requirement is for my Sonos system to play at the correct volume. Sonos supports WMP tags for WMA & MP3, iTunes tags for AAC & MP3 and the standard tags in Flac files. It only supports “track gain” (and, as I have discovered by experiment, only supports negative values, so it will lower the volume but not raise it). What I require is “album gain” on Flac and MP3 files not written by the proprietary systems. What I need is a method to write either iTunes or WMP type tags based on the MediaMonkey ones. As a start I am working on a MediaMonkey plugin that first saves the “track gain” in a custom field for safety then copies the “volume gain” to the “track gain” field to fool the player into supporting audiophile mode. To do the rest of the job I will need to discover what the binary means in the WMP or iTunes tags.

[Edited 4 Jan 2012] to add information obtained from ReplayGain legacy metadata formats (with thanks).

TapSkype Newsletter?

12 Jan 2011 12:47 by Rick

If you see an email announcing itself as from “Skype Newsletter”, check very carefully. Most, if not all of these are bogus. They are spammers trying to get your login details and maybe get you to pay for an upgrade to a free product or install malware on your system.

TapMac App Store

10 Jan 2011 12:45 by Rick

At a rough count I have 32 third party applications installed on my Mac (not including stuff that came with my accessories like camera, GPS and printer). Two of those are paid-for applications, a few are donation-ware. Just one is now supported by the new Mac App Store, TextWrangler, but it doesn’t say so on the product web site.

The only benefit that I have seen from the store is that I may buy Pages now that is is available as a separate download (but curiously it doesn’t say that on the product web site either!)

SuperDuper, Springy, Crossover, CyberDuck, Sbooth Max, Firefox, SourceSource, SoundFlower, SoundApp Reborn, LineIn, LAME, Growl, KeePassX, Perian, Flip4Mac, XNJB, TextWrangler, NeoOffice, Gimp, PaintBrush, Adium, PopFile, Skype, Sonos, Thunderbird, uTorrent, VMware Fusion, Adobe Air, BBC iPlayer, Adobe Flash, Audacity, SilverLight

TapThe Battle of Gloucester Cathedral

14 Dec 2010 14:46 by Rick

The Spectator reports

Annabel Hayter, chairwoman of Gloucester Cathedral Flower Guild, received an email saying that she and her 60 fellow flower arrangers would have to undergo a CRB check. CRB stands for Criminal Records Bureau, and a CRB check is a time-consuming, sometimes expensive, pretty much always pointless vetting procedure that you must go through if you work with children or ‘vulnerable adults’. Everybody else had been checked: the ‘welcomers’ at the cathedral door; the cathedral guides; the whole of the cathedral office (though they rarely left their room). The flower guild was all that remained.

The cathedral authorities expected no resistance. Though the increasing demand for ever tighter safety regulation has become one of the biggest blights on Britain today, we are all strangely supine: frightened not to comply. Not so Annabel Hayter. ‘I am not going to do it,’ she said. And her act of rebellion sparked a mini-revolution among the other cathedral flower ladies. In total she received 30 letters from guild members who judged vetting to be either an invasion of privacy (which it certainly is) insecure (the CRB has a frightening tendency to return the wrong results) or unnecessary (they are the least likely paedophiles in the country). Several threatened to resign if forced to undergo it.

Follow the rest of the story for other examples of the CRB cancer. Thanks to Schneier on Security for the pointer.

TapGawker Media hacked

10:11 by Rick

I have heard today that this prominent publishing house, which includes Lifehacker, Gizmodo (and, ahem, Fleshbot), has had its database of accounts compromised. There is not much point in changing your password there yet, but, if you use the same one elsewhere, it would be a good idea to change them NOW.

Thanks to SANS ISC for the information. Lots more information here.

TapIs this the future of Home Computing?

8 Dec 2010 15:15 by Rick

Disclaimer: Although I work for HP, I have no contact with the division responsible for this product. All views expressed are my own.

I have always thought that home computing has never got out of the hobbyist mindset of the Sinclair Spectrum and BBC home Micro. They require deep knowledge to get the best out of them, require regular maintenance and are forever going wrong in little (how do I do that) or big (BSOD) ways. This puts a lot of people off. To really take off as a universal resource, it needs to be a commodity like your TV; you just plug it in and go. Is this possible for a computer—it is if you stop thinking of it as a computer, that was out of date at the turn of the century. People use these devices to create and read documents, do research, consume entertainment in the form of pictures, video, music and games—only the odd spread sheet could be called computing. So, look at this…

HP recently launched a product called the DreamScreen 400 but you may not have heard of it because it was developed in India for the Indian market. It is well worth taking a look at because it is quite revolutionary and yet derivative at the same time.

It is very targeted to those homes that do not have computer technology already. It is very low cost—less than £300 and does not assume any prior knowledge of computers or even the internet, that is all hidden away. I have no information about the storage mechanism, but I would hope that it is transparent like the iPad, not requiring any understanding of folders.

It is uses standard components which is how the cost is kept down—18.5″ TouchSmart screen, probably an Intel processor, internet connection, webcam & microphone, SD card slot, DVD drive, 4 x USB sockets and probably a Linux operating system with a custom UI bolted on top. I have seen no mention of a wireless connection, DVD burner or what storage capacity but the specification is pretty comprehensive. There is multiple account capability for each member of the family to be kept separate and, presumably, some sort of parental control system. The applications are clearly targeted at the growing middle class of India with emphasis on education, entertainment and running a home. They include eMail, video chat, downloaded and DVD video, music, games, documents, spreadsheets, presentations (I don’t know what formats), news, finance, travel and educational material geared to the Indian curriculum. For the more adventurous there is generic web browsing using Firefox with custom extensions.

The interface is very iPad like, including touch scrolling, and you could say that it is a large screen, permanently docked iPad. There is a mouse, and I think arm-ache would make this almost essential, and a USB keyboard. The latter would only be really required if you wanted to create documents, there is a multi-language on-screen keyboard for simple things like search terms. You can also attach certain (HP only?) USB printers.

This has clearly been engineered to a price with a limited development teams but with a rich array of “Apps”, including third-party developers, I could see this really fulfilling the requirements of any family even in the western world—Will HP pull it off or someone else like Apple with a iMac/iPad hybrid? That remains to be seen.

Update: [14 Dec] I was walking around Maplin and spotted on the discount shelf an MSI Wind Top AE1900. This is so similar to the DreamScreen 400 that the latter is surely just re-badged and with custom software. The MSI came with Wireless-N and is/was almost twice the price.

TapBroadband Speed

27 Oct 2010 10:27 by Rick

This is not going to be a very scientific post being based on a sample of just one, but it may well be of relevance for other people who find that their broadband internet is not as quick as it should be. Before blaming the supplier and accepting their explanation about old infrastructure and distance from exchanges, take a look at what is on your side of the network. In particular look at your modem/router. Is it yellowing with age? If so, that may be the problem. I recently swapped out one that was six years old for a new one and it made a huge difference. I don’t think some early models are capable of supporting current network speeds.

If you look into the detailed status reports available on most models you may find that among the incomprehensible figures there are ones for inbound and outbound connection speeds. The one I was looking at reports only 2Mbps inbound. Changing it for a new model instantly made that jump to 8Mbps. And this was not just figures, the whole browsing experience became sprightly. The replacement wasn’t particularly sophisticated and cost just £55 in a retail shop (we were desperate); if yours comes as part of the contract you could try pestering them.

The same may also be true if you are on a cable network but there you have to convince your supplier because they provide the modem. When I look into this I will report what information is available and how I got on.

As a footnote I will also remark on how peculiar the router market is at the moment.

  • It is very difficult to by a router without wireless and if you can find one they are not a lot cheaper.
  • It is also very difficult to buy a router where the built in switch runs at gigabit speeds and virtually impossible if you want ADSL rather than cable. You may ask why you want that when the broadband is running between 8 and 20Mbps but it is needed for communication between devices internally—like network storage and video streamers. There are even some that boast 300Mbps wireless but only provide 100Mbps wired connections—daft!
  • Beware of wireless systems advertised as “Dual Band N.” The better ones are simultaneous 2.4GHz and 5GHz but others are switch-able so you have one or the other. As you are almost bound to have some devices on the older standard (even if only guests) then the 5GHz option is wasted.
  • Finally, the decision on which one to buy in the exercise above was determined by a trivial factor; wall mount. For the sake of a couple of keyhole slots on the back, competitors were discarded. Even then it wasn’t advertised on the box, I had to get a salesman to open it up to see. Our location (for power and telephone connection) was in a busy narrow gangway so a shelf was not an option but even in better environments why should we have to give up valuable surface space to a utilitarian device like this.

TapProxy Spam

6 Oct 2010 08:51 by Rick

I had a strange comment that leaked through the spam filter today. The content was just an IP address—211.138.124.211:80. That turns out to be a proxy machine in Hangzhou in Zhejiang, China owned by China Mobile Communications Corporation. The spam link was also interesting. It appeared to point to content that was taken from a US local paper (from Mount Vernon) but the language was strange. Here is an extract…

A name from a pointy-eyed neighbor culminated in the arrest of three suspected burglars and the restoration of nearly half of the almost $1,200 in valuables stolen.

The decision came in just after 12:40 Thursday afternoon with a precise description of the three suspects, the automotive they have been in and the direction of travel. The descriptions matched an earlier name from another deal with on Mount Vernon on suspicious persons.

I think it has been machine translated into another language and back again to disguise the source.

TapWallpaper

10 Sep 2010 10:05 by Rick

Apparently Windows 7 Starter Edition, typically installed on new Netbooks, is very crippled. You can’t even change the background wallpaper. Illiad has a solution—and it has the advantage that you no longer see what Windows is doing to your data either.

TapAnother Flash Shock(wave)

26 Aug 2010 11:20 by Rick

Adobe announced yesterday that there is a security patch for Shockwave Player which users should install. However, it would be worth first checking that you have the product in the first place and, if not, then you probably don’t need it at all. Very little content on the web uses it.

Just to confuse matters, the Firefox plugin that Adobe installs for their Flash product is called “Shockwave Flash.” This is one you probably do want as it makes surfing the web a bit easier.

^ Top