【402】Twitter Data Collection

Reference: three methods of determining whether a file exists Python

Reference: execute the file in python another python file

参考：How can I make a time delay in Python?

Reference: Twilio the SMS Python the Quickstart

---

1. collect real-time data for a region

Name: AUS.py

#Import the necessary methods from tweepy library
from tweepy.streaming import StreamListener
from tweepy import OAuthHandler
from tweepy import Stream
 
#Variables that contains the user credentials to access Twitter API
access_token = "*****"
access_token_secret = "*****"
consumer_key = "*****"
consumer_secret = "*****"

#This is a basic listener that just prints received tweets to stdout.
class StdOutListener(StreamListener):

    def on_data(self, data):
        print(data)
        return True

    def on_error(self, status):
        print(status)

        
if __name__ == '__main__':
    
    #This handles Twitter authetification and the connection to Twitter Streaming API
    l = StdOutListener()
    auth = OAuthHandler(consumer_key, consumer_secret)
    auth.set_access_token(access_token, access_token_secret)
    stream = Stream(auth, l)
    
    #This line filter Twitter Streams to capture data by the keywords: 'python', 'javascript', 'ruby'
    stream.filter(locations=[112, -44, 154, -9])

Cmd of code to run on  Python AUS.py> 2019 - 06 - 07 .txt  , real-time data storage.

The printed by the above codes can be stored directly into a text file. (Similar to print () can store content directly)

2. Automatic text messaging function

Since the data is stored to a certain amount of case Ben collapse will occur, thus increasing Twilio automatic text messaging function, met Ben collapse can send text messages in real time, to achieve the following:

File name: AUS_SMS.py

#Import the necessary methods from tweepy library
from tweepy.streaming import StreamListener
from tweepy import OAuthHandler
from tweepy import Stream
from twilio.rest import Client 
import time
 
#Variables that contains the user credentials to access Twitter API
access_token = "*****"
access_token_secret = "*****"
consumer_key = "*****"
consumer_secret = "*****"

#This is a basic listener that just prints received tweets to stdout.
class StdOutListener(StreamListener):

    def on_data(self, data):
        print(data)
        return True

    def on_error(self, status):
        print(status)

def textMessage(message):       
    account = '*****'
    token = '*****'
    myNumber='+*****'
    twilioNumber='+*****'
 
    client = Client(account, token)
    message = client.messages.create(to=myNumber, from_=twilioNumber, body=message)
        
if __name__ == '__main__':
    try:
        #This handles Twitter authetification and the connection to Twitter Streaming API
        l = StdOutListener()
        auth = OAuthHandler(consumer_key, consumer_secret)
        auth.set_access_token(access_token, access_token_secret)
        stream = Stream(auth, l)
        
        #This line filter Twitter Streams to capture data by the keywords: 'python', 'javascript', 'ruby'
        stream.filter(locations=[112, -44, 154, -9])
    except:
        textMessage("n(*≧▽≦*)n [HELP] Program crashed!!!\nTime: "+time.asctime())

3. Unlimited Run

Can be directly run the file through Python Python file that can be achieved through the establishment of unlimited wireless data collection cycle

File name: main.py

import os
import time

while True:    
    year = str(time.localtime().tm_year)
    mon = str(time.localtime().tm_mon)
    day = str(time.localtime().tm_mday)
    filename = year + '-' + mon.zfill(2) + '-' + day.zfill(2)
    i = 0
    while os.path.exists(os.getcwd() + '\\' + filename + '.txt'):
        i += 1
        filename = year + '-' + mon.zfill(2) + '-' + day.zfill(2) + '-' + str(i)
        time.sleep(1)
    os.system("python AUS_SMS.py > " + filename + '.txt')

For file names are named according to today's date, the same day if the file exists, then the back plus 1, then add 2 ,,, and so on. . .

() Method can achieve the effect cmd run Python files by os.system.