This month - more legal shenanigans, Winamp makes a comeback, Discogs turns 20 and more personal music hosting articles.
The youtube-dl saga
Over the past month we’ve added a new chapter in the large-companies-don’t-understand-the-Internet book… this time, as the RIAA attempt to force GitHub to take down a useful open source project.
The attempted #youtubedl takedown has shades of the streisand effect. Do lawyers and corporations just not understand how the internet renders them powerless? Maybe there's an obvious answer to that. https://t.co/C2tnKZaFMMpic.twitter.com/YQeiwmTwHt
Since then, GitHub decided to back the developers and re-instated the project. So, all’s well for now…
Winamp for Windows 10 on the way
The much loved music player has been updated for Windows 10, with better native support so stuff like CD playback/ripping, AAC and H.264 playback all work better.
Last month I mentioned that personal music hosting seems to be gathering pace. Personal music hosting is the practice of self storing music libraries, including on the Internet to take advantage of what the cloud can offer.
This month there were more developments. Manuel Vonau wrote up a list of music lockers:
An important change in behaviour in this release: automatic remediation is no longer performed when an individual manual fix is clicked in the UI.
Before, there could be quite confusing behaviour where an automatic fix kicked in after the some other fix is clicked. Sometimes this is useful, but sometimes not. For example, if a new artwork was uploaded that was too small for the minimum resolution rule, configured as automatic, it would be automatically replaced.
We decided that the cost of potential confusion was not worth the few times this was useful; after all, your rules will still assess the music that has been chaned by the manual fix and will offer a one-click fix. In the example above, the minimum resolution rule would offer a shrink button.
So now automatic remediation is never done as a result of an individual manual fix being executed. “Fix all” and “Fix album” are unaffected, as is a full library rescan (the type you get when you click Refresh or change settings and click Apply rules).
There’s more…
We’ve made some more improvements:
Handle non-Latin characters in upper/lower case rules.
Expose image corruption errors.
When switching to listening mode, enable the file watcher.
Don’t re-initialise music libraries when the library folders haven’t changed.
Fix the show more button.
The show more overlay should pass down button presses to the buttons below.
Cognitive biases are a fascinating explanation of why we sometimes make irrational decisions.
I was wondering: if the majority of people are making a given decision, and I have a different opinion, does that mean I must be wrong-headed and I have some bias? If the world is switching to music streaming, but I’m here with my Squeezeboxlike some caveman does that mean I’m not thinking clearly?
I recently finished Thinking Fast and Slow by Daniel Kahneman- a great book about (amongst other things) cognitive biases. It describes how our thinking can be biased in various ways, with the general idea that “System 1” thinking is more intuitive and, in some ways, emotional, with “System 2” being more rational and logical. System 1 is where, in particular, the cognitive biases creep in.
Away from Kahneman, the broad concept of cognitive biases has been one attracting more discussion. It’s pretty common to read about confirmation bias in the current discussion of social media filter bubbles and unconscious bias when considering gender, race or class.
It struck me that if we are all susceptible to cognitive biases (and we are) we should all probably make a conscious effort to review our thoughts, plans and strategies to ensure we make optimal decisions. The first area I thought of this, with regard to music collecting, was around streaming. I’m a self-storer and yet the world has pretty much already moved toward a rented, open access streaming model. What am I missing that everyone else understands? I wondered if a cognitive bias (or biases) were responsible for wrong-headedness.
The streaming / storing quandry
As I said, not so much a quandry for most; it’s streaming all the way. But for me, I’m the opposite, I’m happy to self store. It’s not so much that I don’t perceive a benefit of streaming, it’s more that I can’t see how my current approach to listening fits in with that.
There are also individuals that fall between the stools; some stream for discovering music, but then purchase releases for various reasons (e.g. wanting the vinyl or simply some intangible benefit of ‘owning’ the music).
Sunk cost fallacy
This is a common one - it explains (amongst other things) why people often say they want to “finish what they started”, regardless of the cost of finishing it.
Applied to music libraries, you may say you want to continue to add to and maintain a music library because you’ve already put in a lot of effort and thought as to how it currently works.
I think this thinking is wrong-headed, personally. It can be very difficult to step-back and consider how much effort you are spending on something, but if you can, and the costs outweigh the benefits, I think you should give up and make a switch.
For me, however, this doesn’t explain why I continue to eschew streaming. I recognise that some time is taken purchasing and integrating music, but with automation like using bliss, this sort of thing just doesn’t cost me what I consider a lot of time. We can all think of activities that you could save time doing, but it doesn’t mean doing them is entirely counter-productive, nor what you’d do otherwise is more productive (e.g. just flicking from stream to stream).
If you have put time into building a music collection, it maybe you want to demonstrate your knowledge and exquisite taste by some extant creation; your curated music library. In addition, similar to the sunk cost fallacy you might attribute higher value to the library, simply because of the time you spent building it.
That said, there may be reasons that your curated library is of more value to you, if not generally. You may have genuinely difficult-to-find releases not available on streaming services. Streaming services might not be able to deliver music in a level of audio quality that you desire, and so on.
In my case, I don’t feel I do this. The library I run is, at most, only for my household. I don’t think it has any intrinsic value to anyone else that couldn’t be derived from a streaming service, assuming that service has access to the same content.
Technical constraints like storage space, or CPU time? No, I think these are getting cheaper all the time.
My own time? Possibly. That said, I already have two small children, a house, a dog and a couple of old cars. My time is probably as stretched as it will ever be, and I’m still managing.
Maybe it’s a question of priorities. If you’re not interested in music collecting, why do it? I am - I like the feeling of builing and maintaining a library, so I make time for it.
Status quo bias
Maybe clinging to a self stored music library is emotional? Maybe it’s nostalgic, memoralising a time when I used to buy the latest CD release from a shop and actually use a CD player? Maybe I just don’t like change? The status quo bias is a preference to current state.
I think there’s a bit of this in me. I like the idea of ‘consuming’ (yuk!) music in the ways I used to, and I see streaming as being, if not completely in opposition, structured and incentivised to pervert that. For example, playlist culture is not something I would immediately subscribe to - I’m an album guy.
Applied to streaming this means: if my entire music library was stolen (which seems pretty impossible as it’s backed up in many ways, but anyway…) what would I do from the next day.
I think the honest answer is I would end up using a streaming service. The cost of starting over seems too high for the benefits.
But that ignores the important consideration that losing the library seems very unlikely; it’s backed up physically and digitally. Someone would really have to put effort into this!
Am I biased?
Given my musical listening preferences and alround philosophies on life, I don’t think it’s cognitive biases that are influencing my current approach to music.
What I’d like to think is it’s a rational… a System 2… consideration of the costs and benefits of the different approaches to music, given the status quo, and I am continuing to class myself as a “music hoarder”.
YouTube Music is a popular service amongst digital music collectors. Like its predecessor did, Google Play Music, it allows file uploads, so your existing, lovingly curated library could be uploaded to the cloud, listened to anywhere, and it would benefit from being backed up by Google.
But what about when it’s time to move on? The service would not be attractive if you were unable to take your library back. Fortunately you can, but the state of the library when exported is not quite as we’d like.
Here’s how to use bliss to fix your Youtube Music backups!
The techniques in this article also help Google Play Music users handle their exported music libraries, because the exports are in the same basic format.
The closure of Google Play Music in October 2020 was met with disappointment by many music collectors. Although still a streaming service, Google Play Music (GPM) permitted uploads of existing music files. It was a music locker, which allows certain advantages; an existing music library, lovingly built, can be backed up securely online. With other streaming services like Spotify not allowing uploads, some were worried they wouldn’t be able to secure a cloud-based home for their music libraries.
They needn’t have worried; YouTube Music stepped into that gap. Google offered an easy transfer from one service to the other.
But for those not interested in moving to the new service (many preferred GPM) or those that gave YouTube Music a try and had decided against using it, how were they to retrieve their music library now?
Google offer a service called Takeout which is a way of exporting data from different Google services. Takeout can be used to export YouTube Music libraries, including the actual music files and some playback statistics. And, up to October, GPM also allowed your library to be downloaded in the same way.
Unfortunately, the reactions to the exports generated by Takeout were not wholly positive. Some bemoaned the lack of structure in the resulting export:
Seriously Google? You force me away from Google Music, and then the damn takeout download has ALL MP3 files in a single directory, not grouped into albums?
On the upside, reports suggested the files are tagged which is probably the most essential thing. So long as you have the tags you can identify your music easily, and restructure it as you require.
Let’s get to the bottom of this and provide a little clarity on Takeout exports.
Requesting a Takeout
Let’s take a look at a Takeout music export; broadly, the same findings apply both to YouTube Music’s exports and Google Play Music’s.
Via Takeout, you can request your YouTube data to be exported. First, you have to scroll through all of the Google services listed; this includes Drive, Chrome, Photos and many, many more. Once you get to the bottom of the list and find YouTube you’ll also realise, of course, that YouTube extends to much more than just your music library.
To request only your music related data, click the All YouTube data included button and make sure to choose just music-library-songs and music-uploads (the former includes music library stats and spreadsheets of the structure of your music library, the latter is the files you have uploaded themselves).
Click Next step and you’ll get a few more options, and then a request for your export will be generated. A short while later you should get a notification that the export is ready for download.
Once downloaded, it’s time to open the archives and see what we’ve got!
Takeout X-ray
The structure of the Takeout archive is the same for both YouTube Music and Google Play Music:
$ tree
.
└── Takeout
├── archive_browser.html
└── YouTube and YouTube Music
├── Playlists
│ └── [...]
└── music-uploads
├── A Space Boy Dream.mp3
├── A Summer Wasting.mp3
├── Bloodflood.flac
├── Breezeblocks.flac
├── Chickfactor.mp3
├── Dirty Dream Number Two.mp3
├── Dissolve Me.flac
├── Ease Your Feet In The Sea.mp3
├── Fitzpleasure.flac
├── (Interlude 1).flac
├── (Interlude 2).flac
├── (Interlude 3).flac
├── Intro.flac
├── Is It Wicked Not To Care_.mp3
├── It Could Have Been A Brilliant Career.mp3
├── Matilda.flac
├── Ms.flac
├── music-uploads-metadata.csv
├── Seymour Stein.mp3
├── Simple Things.mp3
├── Sleep The Clock Around.mp3
├── Something Good.flac
├── Taro _ Handmade.flac
├── Tessellate.flac
└── The Boy With The Arab Strap.mp3
All files are in one folder - music-uploads - with no sub directories. That’s repeated in GPM:
$ tree
.
└── Takeout
├── archive_browser.html
└── Google Play Music
├── Playlists
│ └── [...]
├── Radio stations
│ └── [...]
└── Titles
├── 25.csv
├── 2raumwohnung - Pop Giganten - Urlaubshits(020)3.mp3
├── 2 Teufel.csv
├── 36 Grad.csv
├── 6 Billionen.csv
├── Al Bano _ Romina Power - Pop Giganten - Ur(002).mp3
├── Alice Cooper - The Best Of Alice Cooper(001)Poi.mp3
└── [...]
The only real difference in terms of structure is the naming of the Google Play Music/YouTube Music directory. In addition, the filenames themselves have a different pattern, but it’s unclear to what extent this depends on the original uploaded filename. In the case of YouTube Music, the filenames do not have the track numbers in, even though the original uploaded files did.
Happily, once you open the files the original tags do, indeed, appear to be present. This is very important because it makes identifying the music in the library, importing it into other music players and organizing it a lot easier.
Album art is stored at the original resolution. I tested with an image of 1425x1425 and the original resolution was retained.
There’s one CSV file for each song. The CSV contains some metadata not stored in the music files which relate to the use of the streaming service, for example play counts and ratings:
Disambiguation is a challenge. Both services disambiguate identical filenames by appending a `(1)` suffix (or subsequent integer). However, there appears no deterministic way that this is decided, and no reference from the CSV files to the music files. So the only way of knowing which track is which is by inspecting the tags (or listening!).
In addition, disambiguation appears a little over eager. For example, the album name is disambiguated in the filename because there are multiple tracks in any given album. And yet it's not required, because the filename would be unique based on the track names alone.
An aside: any “in progress” uploads get an auto generated filename:
├── k1sBwLQw7bw.flac
… so it might be worth waiting for any uploads to finish before creating your “Takeout”.
The biggest problem is the file and folder names in the export.
An album-per-folder offers a hint to software to group the tracks into an album. These exports might make importing the music more difficult because all of the files are in one folder.
There's no sequencing of tracks inside a release, so players that rely on filenames (e.g. car stereos) can't play the music in order for any given album.
This was not confirmed, but if the folder is of arbitrary size this could cause problems in some OSes and filesystems which have maximum numbers of files per folder.
In general, having an enormous folder like this is cumbersome and hard to work with.
In summary, then, while a Takeout treats your tags well, we need to do something about the filenames. File naming is the type of thing you want to do once, and it’s best done early. Let’s use bliss to fix the filenaming into a sane structure.
Fix YouTube Music exports with bliss
Your first decision is to choose how to store your music files in your filesystem. I generally recommend something like:
album artist/album name/disc number [optional]/tracknumber-track name
Choose the location of your music files. This should be the folder into which the Takeout archive was extracted. Click Next. On the next page, you get the chance to add your rules. In this case, we’ll set a rule to make sure all the files are stored in our desired location. Click Add a rule > Configure next to File paths and then set this pattern:
This mimics our desired pattern we described above. If you want something different, click Custom paths to explore the different ways you can customise the file paths.
Make sure the rule is set to run in Manual mode first.
Click Scan library and apply rules and bliss will begin assessing your music files.
As bliss scans, it works out which track is in which album. Once it has an idea of your albums it can work out what the file path should be. Those files it can see that are not named according to your file pattern will be shown:
You can click the Fix all file paths link to fix all the files in this album, or click more info to see the current and target file paths. In fact, I encourage you to do this early on to make sure bliss is suggesting appropriate file locations.
Once you’re more confident you can fully automate the file renaming. Either fix all outstanding issues in the Inbox by clicking Select by type > check Fix folder and file structure > Fix allor by changing the rule to run automatically - in the settings page change Manual to Automatic.
Once you’re done, you should have a reorganised music folder, with all your YouTube Music/Google Play Music organised into folders.
Thanks to Timo Schmidt for letting me see the structure of his Google Play Music Takeout export.
