How to crawl youtube videos using google api

Description:

At present, the more appropriate crawler idea is to subscribe to the user you want to crawl> get a list of subscriptions> get a list of subscribed videos> download videos

 

Regarding the use of Google APIs, it is recommended to first find the API menu that you need to try, and then use API Explorer to run it. If the result of the operation meets your expectations, then take the code to run locally, so that the entire process will be smoother

 

1. First you must have a google api account

https://console.developers.google.com/apis/

2. Get a list of subscribed channels

https://developers.google.com/youtube/v3/docs/subscriptions/list

Where mine is selected as true

curl \
  'https://www.googleapis.com/youtube/v3/subscriptions?part=snippet%2CcontentDetails&mine=true&key=[YOUR_API_KEY]' \
  --header 'Authorization: Bearer [YOUR_ACCESS_TOKEN]' \
  --header 'Accept: application/json' \
  --compressed

In the running result, items [0] .snippet.resourceId.channelId is the channel id

3. Get the id of the uploaded video list

https://developers.google.com/youtube/v3/docs/channels/list

Where id is the channelId obtained above

curl \
  'https://www.googleapis.com/youtube/v3/channels?part=snippet%2CcontentDetails%2Cstatistics&id=UC_x5XG1OV2P6uZZ5FSM9Ttw&key=[YOUR_API_KEY]' \
  --header 'Authorization: Bearer [YOUR_ACCESS_TOKEN]' \
  --header 'Accept: application/json' \
  --compressed

In the running result, items [0] .contentDetails.uploads is the playlist id

4. Get the videos in the video list

https://developers.google.com/youtube/v3/docs/playlistItems/list

Where playlistId is the playlist id obtained above

curl \
  'https://www.googleapis.com/youtube/v3/playlistItems?part=snippet%2CcontentDetails&maxResults=50&playlistId=UUpRmvjdu3ixew5ahydZ67uA&key=[YOUR_API_KEY]' \
  --header 'Authorization: Bearer [YOUR_ACCESS_TOKEN]' \
  --header 'Accept: application/json' \
  --compressed

In the running result, we can finally see the video id of Zhaosi ’s dream, items [0] .contentDetails.videoId

5. Get video address

https://www.youtube.com/embed/{videoId}