🤷🏿 👩🏼‍🏭 👻 Analytics for Telegram bots written in Python 👨🏽‍🔬 🛍️ 😇

At the moment, the boom in the creation of telegram bots has begun to fade, but the theme of their creation does not lose relevance. Many libraries have been written to facilitate interaction with the Telegram Bot API, but after creating the bot I still did not find a script (library) to get the bot statistics. Therefore, I decided to write a script for all bots in Python. We will receive statistics by logging user actions and processing logs in a convenient way.

Environmental requirements

To use the script, you need to install the following libraries:

pip install datetime
pip install pandas

How to embed analytics in your bot?

Download the py script from the repository for your OS and data.csv file. Place them in the folder where your bot is located.

After connecting the libraries you use in the file with the bot body, add the line:

import tg_analytic

After the bot commands add:

tg_analytic.statistics(<id >, <>)

If you are using the telebot library, then it should look like this:

Also, to get statistics directly from the bot, you need to add to message_handler (content_types = ['text']):

if message.text[:(  )] == '< >':
        st = message.text.split(' ')
        if 'txt' in st or '' in st:
            tg_analytic.analysis(st,message.chat.id)
            with open('%s.txt' %message.chat.id ,'r',encoding='UTF-8') as file:
                bot.send_document(message.chat.id,file)
                tg_analytic.remove(message.chat.id)
        else:
            messages = tg_analytic.analysis(st,message.chat.id)
            bot.send_message(message.chat.id, messages)

The keyword needs to be invented and entered so that only you can view the statistics of your bot. The telegram bot with the already implemented analytics is also stored in the repository, so that you can familiarize yourself with how to connect it using an example.

What commands to use to get statistics?

For example, the keyword would be “statistics”:

    if message.text[:10] == '' or message.text[:10] == 'C':
        st = message.text.split(' ')
        if 'txt' in st or '' in st:
            tg_analytic.analysis(st,message.chat.id)
            with open('%s.txt' %message.chat.id ,'r',encoding='UTF-8') as file:
                bot.send_document(message.chat.id,file)
                tg_analytic.remove(message.chat.id)
        else:
            messages = tg_analytic.analysis(st,message.chat.id)
            bot.send_message(message.chat.id, messages)

The body of the bot command is as follows:

statistics <number of days> <request parameters * >>

* - There are teams: "users", "teams" and "tkht". Can be used simultaneously. "Users" gives statistics on people for the required number of days. “Teams” gives statistics on teams for the number of days you need. When specifying txt, you will receive a file, otherwise the answer is in a telegram.

Command usage example

Statistics 2 team users

Statistics 2 teams

Statistics 2 users of the team

What does a script consist of?

In general, if you are not interested in the script, then at this stage you can finish reading the article and start implementing it in your bot.

The first part of the script is logging user actions. I decided to save only the date, user id and the command used by him:

def statistics(user_id, command):
    data = datetime.datetime.today().strftime("%Y-%m-%d")
    with open('data.csv', 'a', newline="", encoding='UTF-8') as fil:
        wr = csv.writer(fil, delimiter=';')
        wr.writerow([data, user_id, command])

The second part is the processing of data on demand and the output of the necessary statistics. We read data from csv in Pandas, group by users and commands:

season = int(bid[1])
#   Dataframe
df = pd.read_csv('data.csv', delimiter=';', encoding='utf8')
#     
number_of_users = len(df['id'].unique())
number_of_days = len(df['data'].unique())
#      
df_user = df.groupby(['data', 'id']).count().reset_index().groupby('data').count().reset_index()
list_of_dates_in_df_user = list(df_user['data'])
list_of_number_of_user_in_df_user = list(df_user['id'])
list_of_dates_in_df_user = list_of_dates_in_df_user[-season:]
list_of_number_of_user_in_df_user = list_of_number_of_user_in_df_user[-season:]
#      
df_command = df.groupby(['data', 'command']).count().reset_index()
unique_commands = df['command'].unique()
commands_in_each_day = []
list_of_dates_in_df_command = list(df_command['data'])
list_of_number_of_user_in_df_command = list(df_command['id'])
list_of_name_of_command_in_df_command = list(df_command['command'])
commands_in_this_day = dict()
for i in range(len(list_of_dates_in_df_command)):
     commands_in_this_day[list_of_name_of_command_in_df_command[i]] = list_of_number_of_user_in_df_command[i]
     if i + 1 >= len(list_of_dates_in_df_command) or list_of_dates_in_df_command[i] != list_of_dates_in_df_command[i + 1]:
     commands_in_each_day.append(commands_in_this_day)
     commands_in_this_day = dict()
commands_in_each_day = commands_in_each_day[-season:]

The response to the user is based on the data from the previous step:


message_to_user = '    %s %s: \n' % (season, day_type.get(season, ''))
message_to_user += '    %s %s: \n' % (number_of_days, day_type.get(season, ''))
if season > number_of_days:
     season = number_of_days
     message_to_user += '    , \n' \
                        '      \n'
if '' in bid:
    message_to_user += '    : ' + '%s' % number_of_users \
    + ' %s ' % users_type.get(number_of_users, '') + '\n' \
    '   %s %s: \n' % (season, day_type.get(season, ''))
    for days, number, comm_day in zip(list_of_dates_in_df_user, list_of_number_of_user_in_df_user, 
    commands_in_each_day):
      message_to_user += ':%s :%d   :%s\n' % (days, number, comm_day.get('/start', 0))
if '' in bid:
    message_to_user += '    %s %s: \n' %  
    (season,day_type.get(season, ''))
    for days, commands in zip(list_of_dates_in_df_user, commands_in_each_day):
       message_to_user += ':%s\n' % days
       for i in unique_commands:
           if i in commands:
                message_to_user += '%s - %s \n' % (i, commands.get(i))
          else:
                message_to_user += '%s - 0 \n' % i

In the end, after composing the response message, we check the user’s request for the presence of the “txt” command in order to decide in what format to respond:

if 'txt' in bid or '' in bid:
    with open('%s.txt' % user_id, 'w', encoding='UTF-8') as fil:
        fil.write(message_to_user)
        fil.close()
    else:
        return message_to_user

Conclusion

The script is tested and works in normal mode. If you are interested in seeing how it really works on a bot, you can test on any of my bots: @exchange_minsk_bot, @pogoda_belarus_bot, @fast_translate_bot.

Keyword: "statistics".

Have a nice use!