This page contains a program to download music from myspace.
Myspace is a popular website these days (writing this in november 2010). Many underground bands put some music there, even them!
As a music fanatic I like to discover and listen to many many bands. I can't escape myspace.
There are many problems though.
Myspace is slow; a lot of useless stuff is present in each page. The layout of the pages is insane. And it uses Macromedia Flash crappy stuff to play the songs people put.
And it's hard to save audio tracks.
So, what about a little tool to download music from myspace? A very basic tool, that won't download tons and tons of useless data, that will focus on audio tracks available, and save them on disk, as fast as the connection allows?
Huh?
A few hours of hacking and here is the stuff I made. Well, I didn't do much, just glueing together existing tools and adding a bit of salt on top of it all.
The following program may totally fail to work at the time you try it. People at myspace don't seem to want you to save music on your disk. So, when they find out the tool available here they will change stuff on their system and the tool will miserably fail. Contact me so I can fix it, if I have the time. Or take a look at the tutorial below and do it yourself (and send your contribution for worldwide availability).
myspace-1.0.tar.gz (2010-11-15) First release.
Download the code. Edit the Makefile to adjust paths. Compile.
To run do: ./myspace <account>
The audio tracks will be saved in the current directory.
Yeah, it's a hacker tool. You need a terminal to run it. You need a system with gcc, libcurl, rtmpdump. If you have troubles, get in touch. Sorry, not easy to install, no "click here and there." I just have no time for that. If someone wants to do that, I'll happily add your contribution to this page.
Okay, the most important piece of information comes now.
How did I do it all?
You need tools. Here is what I used.
What we want to do: from the playlist on the myspace page, find out the real location of audio files to download them directly.
So I first tried to download the Flash program and decompile it (using flasm was of no help, swftools was better) and look inside. It was of no help. I couldn't find the playlist nor anything, so for URLs of songs...
I then analyzed the traffic with wireshark. Well, that was not very simple. I couldn't find the playlist from that. But I saw that the songs were played through a weird protocol, RTMPE, whatever that was. I though "hell, it's gonna be much harder than I though..." But a quick seach on the web led to rtmpdump which downloads RTMPE streams, as far as you give it the location.
With rtmpdump comes rtmpsuck, a very nice tool. You launch it and you surf myspace. (You need a system with iptables, like Linux, and configure it to redirect RTMPE traffic to rtmpsuck. The documentation of rtmpsuck explains it all.) Rtmpsuck gives you the address of the stream. We are one step closer.
You then try rtmpdump with the address given by rtmpsuck. That works. Cool. Things get easy again.
Now, the playlist. Where the heck is it?
Firebug is good there. You download the myspace page with firebug turned on and browse through the downloaded files. You find the playlist URL, a simple XML file with identifiers in the URL. You download the playlist in a terminal, with wget.
You look at the playlist. How from that to recreate audio files addresses? And there I was lucky. The audio files have a very weird name, some kind of hash or something. It would have been very hard to find out. But the playlist contains it all, in funky places, but it's all there. No need to compute anything, just extract data from the playlist.
Everything is there. Just a few lines of C code and there you are, a tool that glues all those informations together and, by using rtmpdump, downloads songs from myspace.
Look at the sources, especially defines.h to see the URLs.
Here follows a text file I wrote while analyzing all this.
1 - download the main page http://www.myspace.com/seaholder 2 - build the playlist url http://musicservices.myspace.com/Modules/MusicServices/Services/MusicPlayerService.ashx?action=getArtistPlaylist&artistId=12130035&artistUserId=76832666&playlistId=49225 artistID: look for artid in the HTML page artistUserId: look for profid in the HTML page playlistId: look for plid in the HTML page 3 - from the playlist get the music server, the page, the name and do: rtmpdump --swfVfy http://lads.myspacecdn.com/videos/MSMusicPlayer.swf -r rtmpe://l3fms-music01.myspacecdn.com:1935/myspace-music02 --playpath mp3:283/std_35d2878e2dc74213be85075fc175613a -o out.swf where in the playlist xml you have: http://d3.ac-music.myspacecdn.com/music02/283/sml_35d2878e2dc74213be85075fc175613a.jpg you deduce: - "music02" in "myspace-music02" - "283" in "mp3:283" - "35d2878e2dc74213be85075fc175613a" in "std_35d2878e2dc74213be85075fc175613a" 4 - translate to whatever file format you want, eg: ffmpeg -i out.swf -ar 44100 -ab 96kb out.ogg
How did I find out all those things, especially the command lines?
Well, you try. And you fail. So you try something else. And fail again.
Read the documentation of the tools you use. That's very very important.
This --swfVfy stuff for example, comes from the documentation,
or the use of iptables to redirect RTMPE traffic (yeah, shame on me, I
am a noob with iptables).
And survive frustration. That's probably the hardest thing. As a hacker, you generally start young and, I don't know, but you end up accepting frustration. You know it does not last. You know that in the end you succeed. That's just a state of mind. I often see people struggling hard with computers. They don't know how it works. They don't know why it fails. So they get very nervous and give up after a while. They can't cope with frustration. But computers are very reliable. So if it does not work as you expect, well, you probably did something wrong. Anyway.
Ah, I write this in november 2010. Since people at myspace put some effort in obfuscating everything on their website so that you have a very hard time downloading stuff, all this might be completely wrong at the time you try it. But you get the big picture. You just have to dig for informations using tools. Since some stuff is downloaded by firefox and myspace Flash player, you can save it in a way or another. In the worst case just capture data when it's sent to the audio card. It's live capture, not as elegant as my little hack, but at least you have the songs.
I was very lucky. I mean, this RTMPE protocol is a true pain in the ass. Fortunately it's very popular and many people struggled hard to understand and decypher it. So I had literally nothing to do. Just build URLs and pass those to rtmpdump. Less than ten hours of hacking in the end. That feels good (I did it fast) and bad (all was already done). Anyway, what do I care. It's just a little contribution. Maybe some people will do something much better in the future.
I wanted to do something like DownloadHelper, the firefox extension that lets you download stuff, from youtube for example. But the RTMPE protocol is tricky. I didn't look inside DownloadHelper. Maybe it's easier to integrate than I think. I don't know. It works as it is. It's light. I'm done. But a firefox extension that would do the same thing is a good project I think.
My beloved fellows, it's up to you.
Contact: sed@free.fr
Created: Sat, 13 Nov 2010 11:00:02 +0100
Last update:
Mon, 06 Dec 2010 15:06:35 +0100