本文介绍了一个使用Clojure编写的虾米音乐爬虫项目。该项目旨在通过爬取虾米音乐的数据来发现未听过的但可能喜欢的音乐。爬虫能够收集歌曲、专辑和艺术家的信息，并对数据进行分析，帮助用户找到各类型中最受欢迎的音乐。
Source code
https://github.com/bluesilence/Lisp/tree/master/clojure/projects/xiami-crawler
Background
Once upon a time...I was starving at new music that I might like at xiami.com.
www.xiami.com is the only website I used to listening to music.
- Why I love xiami:
- Tremendous amount of songs/albums/artists
- User-friendly interface
- You don't have to log on to play a song
- Why I wrote this xiami crawler:
- The recommendation system is based on songs you have listened. It doesn't solve my problem of discovering songs that I haven't listened but I might like.
- Furthermore, xiami doesn't support searching by the ranking on the rating of songs, which make it difficult to find the best music.
- The list of representative songs provided by xiami(http://www.xiami.com/genre?spm=a1z1s.3057857.1110925385.5.8zjSZT) haven't been ranked, or are ranked by some mysterious algorithm. I want to know the best rating songs of each genre.
- Fortunately, I was learning Clojure, and there was a piece of code of a crawler made by Clojure in the book "Clojure Programming".
So why not try to write a crawler that collects music infos from xiami and do the
DIY data analysis to find my favorite music?
Urls
The urls on xiami is quite simple and organized:
Songs
http://www.xiami.com/song/$song-id
Eg. http://www.xiami.com/song/1
Albums
http://www.xiami.com/song/$album-id
Eg. http://www.xiami.com/album/1
Artists
http://www.xiami.com/song/$artist-id
Eg. http://www.xiami.com/artist/1
So the crawler doesn't have to crawl the page for new urls. It just need to increment the id to get the url of the next page.
Data Schema
Here are the schema of the data needed to analyse:
Songs
- song-id
- song-name
- song-hot
- album-id
Albums
- album-id
- album-name
- album-category
- album-genre
- artist-id
- album-value
Artists
- artist-id
- artist-name
扫一扫
1195
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
