Get best of the week

Add thousands of clips to a YouTube playlist from a Telegram channel

With the purchase of a TV with the YouTube viewing function and a subscription to many Telegram channels where clips are laid out daily, I wanted to remove unnecessary movements between my mobile phone and the TV to watch the latest fresh content.



The idea is simple:

  • Given : One or more Telegram content-maker channels with daily uploading of the latest clips.
  • Task : Create a playlist on your YouTube with all these clips and automatically catch new ones.
  • Tools : Google Apps Script (GAS), YouTube API



Export Telegram Channel History


We are looking for a suitable Telegram channel that posts posts with YouTube clips (for example, take @fresh_farsh).

In the upper right corner there are three dots - " Export channel history ".


For the list of links, we only need text, so we uncheck all the boxes and set the maximum upload period, that is, " from the first message to the current date ."


The ChatExport folder is uploaded , inside several numbered messages files (2,3,4 ...). Html , the number of files depends on the amount of channel content.


Parsing a channel in a google spreadsheet


We need to make a list of YouTube links to add to the Google spreadsheet. I chose this site .

Enter the regular expression in the top line:

">(https:\/\/<a href="about:blank">www.youtube.com\/.*</a>?)<\/a>|">(https:\/\/youtu.be\/.*?)<\/a>

Below we insert the text from all messages (2,3,4) .html in a row. In the right pane, the result is visible.

Click the button to upload the result.


Press " Include full match in exported data " so that unnecessary matches are not unloaded. We select CSV , unload.


Create a Google spreadsheet.

We copy CSV into it , remove unnecessary lines, bring all links to youtu.be


Create a playlist channel, YouTube API


We go to YouTube .

On the top right, click on the account icon - " Create Channel ".


We come up with a name, remember the channel ID from the address bar:
https://www.youtube.com/playlist?list=<ID CHANNEL>

Now we need to enable the YouTube API.
We create the project in one of the ways:

  • We follow the link , then select " API and Services - Control Panel - Create Project "
  • By clicking on the link immediately .

We name the project, click " Create ."

Next, click " Enable APIs and Services ", look for " YouTube Data API v3 " - Enable .

So, the API is included .

We return to the Google table.

" Tools - script editor ."


We paste the code, where in the idCHannel we insert the ID of our channel, " File - Save " (CTRL + S).


Source code
idCHannel = 'PLZqUj_5dkPHaAYSECFNOKqnsr2JypZnoo' // ID 

//     Google    YouTube
function fromSheetToYouTube(){
  //    
  var ss = SpreadsheetApp.getActiveSpreadsheet()
  var s = ss.getSheets()[0]
  //      
  var data = s.getDataRange().getValues()
  //   . 
  for (var d=1; d<=200; d++) {
    try {
      YouTube.PlaylistItems.insert({
        snippet: {
          playlistId: idCHannel,
          resourceId: {
            kind: "youtube#video",
            videoId: data[d][0].replace('https://youtu.be/','')
          }
        }
      }, "snippet")
      //     B  ,   
      s.getRange("B"+parseInt(d+1)).setValue('')
      //      
      Utilities.sleep(1000)
      //   15    50 
      if (d%50 == 0) Utilities.sleep(15000)
    } catch (e) {
      console.error(e);
    }
  }
}


Now we need to connect the API to our script.

" Resources - Advanced Google Features - YouTube Data API - On ."



Now we are ready to run the script.

In the " Select function " drop-down menu, select " fromSheetToYouTube ". Click the " Run "

triangle on the left . If everything is in order and there are no red errors, " Running function fromSheetToYouTube ... " is on, then our script is running. Clips are added, you can check the playlist in parallel. Adding goes at a speed of 1 clip per second, plus pause every 50 clips (default 15 seconds). After execution, you can see the error log in the " View - Logs " menu or by pressing "CTRL + ENTER", there is also a link to view all startup errors scriptcalled „ Apps Script toolbar







“

If we see the error“ The request cannot be completed because you have exceeded your quota “, this means that we have exceeded the daily quota for API operations.

Ways to export clips. Quotas.


YouTube has API restrictions called quotas .

Description of YouTube quotas here .

The rest of the quotas can be seen here on the graphs (it is better to set the time interval immediately the smallest for clarity).

What we need to know from the description about quotas:
Projects that enable the YouTube Data API have a default quota allocation of 10 thousand units per day
10,000 quotas per day .

A write operation has a cost of approximately 50 units
One write operation has a cost of 50 quotas.

If you calculate the quotas calculator , then one operation of adding a video to the playlist playlistItems-insert will leave us a little more than 50 quotas if snippet is enabled.

Total we get 10000/53 about 188 clips.

If for some reason there is no access to the clip or the author deleted it, the request will be wasted, spending one reading quota.
That is, taking into account the percentage of “inaccessible / deleted” videos, we can send about 180 clips in one day .

Let's look at all the options for downloading clips from the slowest to the most effective


  1. , .
    — “ — URL», , " ".

    , . Selenium IDE — Katalon, Katalon Recorder (Selenium IDE++ Chrome Firefox)
  2. 180
    .
    , 15 , 15:00 180 .
  3. 180
    Google: "", "1".

    "" , "1", , "", ( ).

    fromSheetToYouTube ( ""), "1".

    "" — " — — ".



    " " .

    "1" "".

    "1" "".

    " …"


    "".


    " ".

    .

    : 5-6 1000 .

    , , .



    1. .
    2. Google , IP Google .


    VPN IP ( ).

    IP+VPN .

    , , .



    .

    TODO

    " " API Google (Google Sheets API v4) R googlesheets4.
  4. :

    • Google (. ).
    • .
  5. 50 . !
    , .

    YouTube , ID ( 50 ).
    : https://www.youtube.com/watch_videos?video_ids=<ID1,ID2,ID3>

    , , ID.

    https://www.youtube.com/watch?v=jSPUkvjaiBs&list=< ID>
    : https://www.youtube.com/playlist?disable_polymer=true&list=< ID>

    disable_polymer .

    , " …" ( 3 — «»).


    : 50 , .

    :

    • .
    • ! 50 .

Telegram GAS


If this is our channel or we contacted the channel administrator, where the clips were parsed from and he agreed to cooperate, you can write a small bot that tracks new channel clips and adds them to our YouTube playlist.

We make the bot according to the instructions , we come up with the name. After creating the bot, we need a token of the form 123456789: ABCD ...

We go into our Google table in the script editor.

You can use the same project as our previous fromSheetToYouTube script .

We insert the doPost function , it will catch messages from the bot on the input, parse links with the regular one and add a playlist to our YouTube.

Source code
//     Telegram
function doPost(e) {
  //     JSON
  var inp = JSON.parse(e.postData.contents)
  //   .   ,  channel_post,      message
  if (typeof inp.channel_post === 'undefined'){
    var inpTxt = inp.message.text
  }else{
    var inpTxt = inp.channel_post.text
  }
  //   
  var y = /(?<=youtube.com\/watch\?v=)(.*?)$|(?<=youtu.be\/)(.*?)$/.exec(inpTxt)[0]
  try {
    YouTube.PlaylistItems.insert({
      snippet: {
        playlistId: idCHannel, // 
        resourceId: {
          kind: "youtube#video",
          videoId: y // 
        }
      }
    }, "snippet")
      } catch (e) {
        console.error(e);
      }
}


Added, saved, must be published in order to get an access link.

Click " Publish - Deploy as a web application ... "

Select drop-down menus, as in the screenshot.


Next, we need the contents of the entire link https

: //script.google.com/macros/s... Once we have the bot token and a link to our application, we can link them through webhook .

https://api.telegram.org/bot<BOT TOKEN> / setWebHook? url = <LINK TO THE APPLICATION>
A message should be displayed

{"ok":true,"result":true,"description":"Webhook was set"}

I’ll write right away that resetting a webhook (untying the bot from the application) is done like this: https://api.telegram.org/bot<BOT TOKEN> /
setWebHook

{"ok":true,"result":true,"description":"Webhook was deleted"}

How to bypass blocking

On a number of providers, everything related to the operation of Telegram through a browser is blocked . Accordingly, a call to https://api.telegram.org/ will display " Unable to access the site ."

Here the VPN will help us, I used the Hotspot Shield Free VPN Proxy - Unlimited VPN extension for Google Chrome

So, our bot is connected to the script and if you write something in the bot, the doPost function will catch and process this.

The input structure of JSON messages is here .

In principle, some of these functions will be enough for some people - we subscribe to many channels, parse everything into one playlist, and then send new posts from all channels to the bots and we know that they will be instantly added to our playlist.

Automation

To automate the process, add the bot to the channel by the administrator, after reading about privacy policies here .

Now, with a new post in the channel, our script will receive

JSON.parse(e.postData.contents).channel_post.text

e is the input parameter.

When sending from a bot, respectively

JSON.parse(e.postData.contents).message.text

At your discretion, you can fasten the check for the existence of clips, replays, logs in cells, etc., using the manual .

Remember that the GAS is not processed at the first start immediately, but after about 10-30 seconds, so your first message to the bot will not come immediately.

Conclusion


So, we have learned several ways to add clips.

In addition, we have a Telegram bot attached to the channel, capable of catching messages both from the channel and addressed to it, parse them in GAS and add to the YouTube playlist.
The problem is solved.

Sources


Description of all methods of working with Telegram bots here .
YouTube API reference here
All Google Apps Script teams here

More articles:


All Articles