[Unmaintained] A simple and clean video/music/image downloader ๐Ÿ‘พ

804
140
Python

Unmaintained

Sorry for this.

For a similar project that is still actively developed, try Annie: https://github.com/iawia002/annie

Lulu

PyPI
Build Status
Build status
codecov

Lulu is a friendly you-get fork (โฌ Dumb downloader that scrapes the web).

Why fork?

Faster updates

Installation

Prerequisites

The following dependencies are required and must be installed separately.

Install via pip

$ pip3 install lulu

upgrade:

$ pip3 install -U lulu

Get Started

Hereโ€™s how you use Lulu to download a video from Bilibili:

$ lulu https://www.bilibili.com/video/av18295259/
site:                Bilibili
title:               ใ€ไธญๆ–‡ๅ…ซ็บงใ€‘ไฟ„็ฝ—ๆ–ฏไบบ็š„ๅๅญ—่ถ…ไนŽไฝ ็š„ๆƒณ่ฑก
stream:
    - format:        flv720
      container:     flv
      size:          175.4 MiB (183914793 bytes)
    # download-with: lulu --format=flv720 [URL]

Downloading ใ€ไธญๆ–‡ๅ…ซ็บงใ€‘ไฟ„็ฝ—ๆ–ฏไบบ็š„ๅๅญ—่ถ…ไนŽไฝ ็š„ๆƒณ่ฑก.flv ...
 100% (175.4/175.4MB) โ”œโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ”ค[1/1]    3 MB/s

Downloading ใ€ไธญๆ–‡ๅ…ซ็บงใ€‘ไฟ„็ฝ—ๆ–ฏไบบ็š„ๅๅญ—่ถ…ไนŽไฝ ็š„ๆƒณ่ฑก.cmt.xml ...

Download a video

When you get a video of interest, you might want to use the --info/-i option to see all available quality and formats:

$ lulu -i 'https://www.youtube.com/watch?v=jNQXAC9IVRw'
site:                YouTube
title:               Me at the zoo
streams:             # Available quality and codecs
    [ DEFAULT ] _________________________________
    - itag:          43
      container:     webm
      quality:       medium
      size:          0.5 MiB (564215 bytes)
    # download-with: lulu --itag=43 [URL]

    - itag:          18
      container:     mp4
      quality:       medium
    # download-with: lulu --itag=18 [URL]

    - itag:          5
      container:     flv
      quality:       small
    # download-with: lulu --itag=5 [URL]

    - itag:          36
      container:     3gp
      quality:       small
    # download-with: lulu --itag=36 [URL]

    - itag:          17
      container:     3gp
      quality:       small
    # download-with: lulu --itag=17 [URL]

The format marked with DEFAULT is the one you will get by default. If that looks cool to you, download it:

$ lulu 'https://www.youtube.com/watch?v=jNQXAC9IVRw'
site:                YouTube
title:               Me at the zoo
stream:
    - itag:          43
      container:     webm
      quality:       medium
      size:          0.5 MiB (564215 bytes)
    # download-with: lulu --itag=43 [URL]

Downloading zoo.webm ...
100.0% (  0.5/0.5  MB) โ”œโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ”ค[1/1]    7 MB/s

Saving Me at the zoo.en.srt ...Done.

(If a YouTube video has any closed captions, they will be downloaded together with the video file, in SubRip subtitle format.)

Or, if you prefer another format (mp4), just use whatever the option lulu shows to you:

$ lulu --itag=18 'https://www.youtube.com/watch?v=jNQXAC9IVRw'

Note:

  • At this point, format selection has not been generally implemented for most of our supported sites; in that case, the default format to download is the one with the highest quality.
  • ffmpeg is a required dependency, for downloading and joining videos streamed in multiple parts (e.g. on some sites like Youku), and for YouTube videos of 1080p or high resolution.
  • If you donโ€™t want lulu to join video parts after downloading them, use the --no-merge/-n option.

Download anything else

If you already have the URL of the exact resource you want, you can download it directly with:

$ lulu https://stallman.org/rms.jpg
Site:       stallman.org
Title:      rms
Type:       JPEG Image (image/jpeg)
Size:       0.06 MiB (66482 Bytes)

Downloading rms.jpg ...
100.0% (  0.1/0.1  MB) โ”œโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ”ค[1/1]  127 kB/s

Otherwise, lulu will scrape the web page and try to figure out if thereโ€™s anything interesting to you:

$ lulu http://kopasas.tumblr.com/post/69361932517
Site:       Tumblr.com
Title:      kopasas
Type:       Unknown type (None)
Size:       0.51 MiB (536583 Bytes)

Site:       Tumblr.com
Title:      tumblr_mxhg13jx4n1sftq6do1_1280
Type:       Portable Network Graphics (image/png)
Size:       0.51 MiB (536583 Bytes)

Downloading tumblr_mxhg13jx4n1sftq6do1_1280.png ...
100.0% (  0.5/0.5  MB) โ”œโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ”ค[1/1]   22 MB/s

Note:

  • This feature is an experimental one and far from perfect. It works best on scraping large-sized images from popular websites like Tumblr and Blogger, but there is really no universal pattern that can apply to any site on the Internet.

Pause and resume a download

You may use Ctrl+C to interrupt a download.

A temporary .download file is kept in the output directory. Next time you run lulu with the same arguments, the download progress will resume from the last session. In case the file is completely downloaded (the temporary .download extension is gone), lulu will just skip the download.

To enforce re-downloading, use the --force/-f option. (Warning: doing so will overwrite any existing file or temporary file with the same name!)

Multi-Thread Download

Use -T/--thread number option to enable multithreading to download(only works for multiple-parts video), number means how many threads you want to use.

Proxy settings

You may specify an HTTP proxy for lulu to use, via the --http-proxy/-x option:

$ lulu -x 127.0.0.1:8087 'https://www.youtube.com/watch?v=jNQXAC9IVRw'

However, the system proxy setting (i.e. the environment variable http_proxy) is applied by default. To disable any proxy, use the --no-proxy option.

Tips:

  • If you need to use proxies a lot (in case your network is blocking certain sites), you might want to use lulu with proxychains and set alias lulu="proxychains -q lulu" (in Bash).
  • For some websites (e.g. Youku), if you need access to some videos that are only available in mainland China, there is an option of using a specific proxy to extract video information from the site: --extractor-proxy/-y.

Load cookies

Not all videos are publicly available to anyone. If you need to log in your account to access something (e.g., a private video), it would be unavoidable to feed the browser cookies to lulu via the --cookies/-c option.

Note:

  • As of now, we are supporting two formats of browser cookies: Mozilla cookies.sqlite and Netscape cookies.txt.

Watch a video

Use the --player/-p option to feed the video into your media player of choice, e.g. mplayer or vlc, instead of downloading it:

$ lulu -p vlc 'https://www.youtube.com/watch?v=jNQXAC9IVRw'

Or, if you prefer to watch the video in a browser, just without ads or comment section:

$ lulu -p chromium 'https://www.youtube.com/watch?v=jNQXAC9IVRw'

Tips:

  • It is possible to use the -p option to start another download manager, e.g., lulu -p uget-gtk 'https://www.youtube.com/watch?v=jNQXAC9IVRw', though they may not play together very well.

Set the path and name of downloaded file

Use the --output-dir/-o option to set the path, and --output-filename/-O to set the name of the downloaded file:

$ lulu -o ~/Videos -O zoo.webm 'https://www.youtube.com/watch?v=jNQXAC9IVRw'

Tips:

  • These options are helpful if you encounter problems with the default video titles, which may contain special characters that do not play well with your current shell / operating system / filesystem.
  • These options are also helpful if you write a script to batch download files and put them into designated folders with designated names.

Reuse extracted data

Use --url/-u to get a list of downloadable resource URLs extracted from the page. Use --json to get an abstract of extracted data in the JSON format.

Warning:

  • For the time being, this feature has NOT been stabilized and the JSON schema may have breaking changes in the future.

Search on Google Videos and download

You can pass literally anything to lulu. If it isnโ€™t a valid URL, lulu will do a Google search and download the most relevant video for you. (It might not be exactly the thing you wish to see, but still very likely.)

$ lulu "Richard Stallman eats"

Supported Sites

Site URL Videos? Images? Audios?
YouTube https://www.youtube.com/ โœ“
Twitter https://twitter.com/ โœ“ โœ“
VK http://vk.com/ โœ“ โœ“
Vine https://vine.co/ โœ“
Vimeo https://vimeo.com/ โœ“
Vidto http://vidto.me/ โœ“
Videomega http://videomega.tv/ โœ“
Veoh http://www.veoh.com/ โœ“
Tumblr https://www.tumblr.com/ โœ“ โœ“ โœ“
TED http://www.ted.com/ โœ“
SoundCloud https://soundcloud.com/ โœ“
SHOWROOM https://www.showroom-live.com/ โœ“
Pinterest https://www.pinterest.com/ โœ“
MusicPlayOn http://en.musicplayon.com/ โœ“
MTV81 http://www.mtv81.com/ โœ“
Metacafe http://www.metacafe.com/ โœ“
Magisto http://www.magisto.com/ โœ“
Khan Academy https://www.khanacademy.org/ โœ“
Internet Archive https://archive.org/ โœ“
Instagram https://instagram.com/ โœ“ โœ“
InfoQ http://www.infoq.com/presentations/ โœ“
Imgur http://imgur.com/ โœ“
Heavy Music Archive http://www.heavy-music.ru/ โœ“
Google+ https://plus.google.com/ โœ“ โœ“
Freesound http://www.freesound.org/ โœ“
Flickr https://www.flickr.com/ โœ“ โœ“
FC2 Video http://video.fc2.com/ โœ“
Facebook https://www.facebook.com/ โœ“
eHow http://www.ehow.com/ โœ“
Dailymotion http://www.dailymotion.com/ โœ“
Coub http://coub.com/ โœ“
CBS http://www.cbs.com/ โœ“
Bandcamp http://bandcamp.com/ โœ“
AliveThai http://alive.in.th/ โœ“
755
ใƒŠใƒŠใ‚ดใƒผใ‚ดใƒผ
http://7gogo.jp/ โœ“ โœ“
niconico
ใƒ‹ใ‚ณใƒ‹ใ‚ณๅ‹•็”ป
http://www.nicovideo.jp/ โœ“
163
็ฝ‘ๆ˜“่ง†้ข‘
็ฝ‘ๆ˜“ไบ‘้Ÿณไน
http://v.163.com/
http://music.163.com/
โœ“ โœ“
56็ฝ‘ http://www.56.com/ โœ“
AcFun http://www.acfun.cn/ โœ“
Baidu
็™พๅบฆ่ดดๅง
http://tieba.baidu.com/ โœ“ โœ“
็ˆ†็ฑณ่Šฑ็ฝ‘ http://www.baomihua.com/ โœ“
bilibili
ๅ“”ๅ“ฉๅ“”ๅ“ฉ
http://www.bilibili.com/ โœ“
Dilidili http://www.dilidili.com/ โœ“
่ฑ†็“ฃ http://www.douban.com/ โœ“
ๆ–—้ฑผ http://www.douyutv.com/ โœ“
Panda
็†Š็Œซ
http://www.panda.tv/ โœ“
ๅ‡คๅ‡ฐ่ง†้ข‘ http://v.ifeng.com/ โœ“
้ฃŽ่กŒ็ฝ‘ http://www.fun.tv/ โœ“
iQIYI
็ˆฑๅฅ‡่‰บ
http://www.iqiyi.com/ โœ“
ๆฟ€ๅŠจ็ฝ‘ http://www.joy.cn/ โœ“
้…ท6็ฝ‘ http://www.ku6.com/ โœ“
้…ท็‹—้Ÿณไน http://www.kugou.com/ โœ“
้…ทๆˆ‘้Ÿณไน http://www.kuwo.cn/ โœ“
ไน่ง†็ฝ‘ http://www.le.com/ โœ“
่”ๆžFM http://www.lizhi.fm/ โœ“
็ง’ๆ‹ http://www.miaopai.com/ โœ“
ๅฐๅ’–็ง€ http://xiaokaxiu.com โœ“
็—žๅฎข้‚ฆ https://www.pixnet.net/ โœ“
PPTV่šๅŠ› http://www.pptv.com/ โœ“
้ฝ้ฒ็ฝ‘ http://v.iqilu.com/ โœ“
QQ
่…พ่ฎฏ่ง†้ข‘
http://v.qq.com/ โœ“
ไผ้น…็›ดๆ’ญ http://live.qq.com/ โœ“
Sina
ๆ–ฐๆตช่ง†้ข‘
ๅพฎๅš็ง’ๆ‹่ง†้ข‘
http://video.sina.com.cn/
http://video.weibo.com/
โœ“
Sohu
ๆœ็‹่ง†้ข‘
http://tv.sohu.com/ โœ“
Tudou
ๅœŸ่ฑ†
http://www.tudou.com/ โœ“
่™พ็ฑณ http://www.xiami.com/ โœ“ โœ“
้˜ณๅ…‰ๅซ่ง† http://www.isuntv.com/ โœ“
้Ÿณๆ‚ฆTai http://www.yinyuetai.com/ โœ“
Youku
ไผ˜้…ท
http://www.youku.com/ โœ“
ๆˆ˜ๆ——TV http://www.zhanqi.tv/lives โœ“
ๅคฎ่ง†็ฝ‘ http://www.cntv.cn/ โœ“
่Šฑ็“ฃ http://huaban.com/ โœ“
Naver
๋„ค์ด๋ฒ„
http://tvcast.naver.com/ โœ“
่Š’ๆžœTV http://www.mgtv.com/ โœ“
็ซ็ŒซTV http://www.huomao.com/ โœ“
ๅ…จๆฐ‘็›ดๆ’ญ http://www.quanmin.tv/ โœ“
้˜ณๅ…‰ๅฎฝ้ข‘็ฝ‘ http://www.365yg.com/ โœ“
่ฅฟ็“œ่ง†้ข‘ https://www.ixigua.com/ โœ“
ๅฟซๆ‰‹ https://www.kuaishou.com/ โœ“ โœ“
ๆŠ–้Ÿณ https://www.douyin.com/ โœ“
้พ™็ ็›ดๆ’ญ http://longzhu.com โœ“
ๅŠๆฌกๅ…ƒ https://bcy.net โœ“
pixivision https://www.pixivision.net โœ“

For all other sites not on the list, the universal extractor will take care of finding and downloading interesting resources from the page.

Development

Preparation

Install pipenv:

$ pip3 install pipenv

and fabric (Note: fabric doesnโ€™t support python3 now, install using pip2):

$ pip install fabric

Initialize virtualenv

$ pipenv --python 3

Install all dependencies:

$ pipenv install --dev

Use the shell:

$ pipenv shell

Run the tests:

$ fab test

Contributing

Lulu is an open source project and welcome contributions ๐Ÿ˜‰

Note

@iawia002 has pep8 obsessive-compulsive disorder, all code must follow pep8 guidelines.

You can use flake8 to check the code before submitting.

Authors

You can find the list of all contributors here.

License

MIT