Friends often see small things to sell at idle or fish and some of the skills they are good at, after a good sleep can bring income for them.
Author: Ann Star Fruit
1. target scene
Friends often see small things to sell at idle or fish and some of the skills they are good at, after a good sleep can bring income for them.
A large number of goods on the busy fish, it is difficult to determine precisely what popular, which sell; one data entry to do data analysis, a waste of time at the same time, the efficiency is extremely inefficient.
The purpose of this article is to use Python to automate obtain certain goods in good selling merchandise for reference.
ps: This article only for technical exchanges, not for other purposes.
2. Preparations
Before writing code, you need to do the following preparations:
(1) development environment configured Android ADB
Pocoui dependent libraries in the installation (2) Python virtual environment
(3) mounting data visualization dependent libraries pyecharts
# pocoui
pip3 install pocoui
# 数据可视化图表
pip3 install pyecharts -U 3. Write code
We divided seven steps to achieve this are: Open the target application client, the search keyword to the Product List interface, excellent sliding distance calculation, selection of goods, obtain product link address, write the file sorting and counting of goods, configuration parameters .
Step 1. Open the target using pocoui automation applications.
def __pre(self):
"""
准备工作
:return:
"""
home()
stop_app(package_name)
start_my_app(package_name, activity)
# 等待到达桌面
self.poco(text='闲鱼').wait_for_appearance()
self.poco(text='鱼塘').wait_for_appearance()
self.poco(text='消息').wait_for_appearance()
self.poco(text='我的').wait_for_appearance()
print('进入闲鱼主界面') After entering the idle fish home, will get the application side clipboard data, when there is a password of a specific law, a dialog box will pop up immediately, it is necessary to close the dialog box simulation operation.
# 如果指定时间内内有淘口令，就关闭
for i in range(10, -1, -1):
close_element = self.poco('com.taobao.idlefish:id/ivClose')
if close_element.exists():
close_element.click()
break
time.sleep(1) After opening the application, the second step can be operated.
To search by keyword, analog input to the input box and click the Search button, the search had been waiting list appears.
另外，为了更加方便地处理数据，商品列表切换到列表模式，即一行只显示一个商品。
def __input_key_word(self):
"""
输入关键字
:return:
"""
# 进入搜索界面
perform_click(self.poco('com.taobao.idlefish:id/bar_tx'))
# 搜索框内输入文本
self.poco('com.taobao.idlefish:id/search_term').set_text(self.good_msg)
# 点击搜索按钮
while True:
# 等待检索结果列表出现
if not self.poco('com.taobao.idlefish:id/list_recyclerview').exists():
perform_click(self.poco('com.taobao.idlefish:id/search_button', text='搜索'))
else:
break
# 等待商品列表完全出现
self.poco('com.taobao.idlefish:id/list_recyclerview').wait_for_appearance()
# 切换到列表
perform_click(self.poco('com.taobao.idlefish:id/switch_search')) 第 3 步，计算优秀滑动距离。
为了保证爬取数据的高效性，获取计算出每次滑动的优秀距离。
首先先拿到当前界面的 UI 控件树，然后通过控件的属性 ID 拿到商品的坐标，进而得到每一项商品的高度。
最后，通过观察屏幕中出现商品的数目得到优秀滑动距离。
def __get_good_swipe_distance(self):
"""
获取每次滑动，最合适的距离
:return:
"""
element = Element()
# 保存当前的UI树到本地
element.get_current_ui_tree()
# 第一个商品Item的坐标
position_item = element.find_elment_position_by_id_and_index("com.taobao.idlefish:id/card_root",
"1")
# 商品的高度
item_height = position_item[1][1] - position_item[0][1]
# 通过观察，当前屏幕有3件商品
return item_height * 3 第 4 步，筛选商品。
上面的步骤拿到优秀的滑动距离，不停的滑动页面遍历列表元素的子 Item。
需要注意的是，为了避免滑动惯性导致的误差，每一次的滑动时长最好设置为 2s 以上。
通过商品 Item 筛选出想要数目大于预设数字的商品。
# 多少人想要
want_element_parent = item.offspring('com.taobao.idlefish:id/search_item_flowlayout')
if want_element_parent.exists():
# 想要数/已付款数目
want_element = want_element_parent.children()[0]
want_content = want_element.get_text()
# 过滤掉【已付款】等其他商品，只保留个人发布商品
if '人想要' not in want_content:
continue
# 拿到商品想要的具体数目，代表商品热度
want_num = get_num(want_content)
if int(want_num) < self.num_assign:
# print('不达标，过滤掉')
pass
else:
# 商品想要数达标，加入统计 第 5 步，获取商品链接地址。
对于上一步满足条件的商品，点击商品 Item 进入到商品详情页面。
接着点击右上角的分享按钮，会立即弹出分享对话框。
然后点击口令控件，会提示口令复制到系统剪切板成功。
# 点击更多
while True:
if self.poco('com.taobao.idlefish:id/ftShareName').exists():
break
print('点击更多~')
perform_click(self.poco(text='更多'))
# 点击复制淘口令
perform_click(self.poco('com.taobao.idlefish:id/ftShareName', text='淘口令'))
# 拿到口令码
taobao_code_element = self.poco('com.taobao.idlefish:id/tvWarnDetail')
taobao_code = taobao_code_element.get_text() 第 6 步，写入商品、排序并统计数据。
将上面获取到的商品标题、想要数、分享地址写入到 CSV 文件中。
然后读取数据文件，通过对表格中的第二列进行反向排序，使商品按照想要数进行降序排列。
def __sort_result(self):
"""
对爬取的结果进行排序
:return:
"""
reader = csv.reader(open(self.file_path), delimiter=",")
# 头部标题
head_title = next(reader)
# 按照第二列进行逆序排列
sortedlist = sorted(reader, key=lambda x: (int(x[1])), reverse=True)
# 写入头部数据
write_to_csv(self.file_path, [(head_title[0], head_title[1], head_title[2])], False)
for value in sortedlist:
write_to_csv(self.file_path, [(value[0], value[1], value[2])], False)
return sortedlist 最后拿到前 10 项数据，利用 pyecharts 生成统计图表。
def draw_image(self, sortedlist):
"""
画图
:param sortedlist:
:return:
"""
# 标题列表
titles = []
# 销量
sales_num = []
# 拿到爬取结果的标题、销量两个列表
with open(self.file_path, 'r') as csvfile:
# 读取文件
reader = csv.DictReader(csvfile)
# 加入列表中
for row in reader:
titles.append(row['title'])
sales_num.append(row['num'])
# 数目限制
if len(titles) > self.num:
titles = titles[:self.num]
sales_num = sales_num[:self.num]
# 画图
bar = (
Bar()
.add_xaxis(titles)
.add_yaxis("哪些好卖", sales_num)
.set_global_opts(title_opts=opts.TitleOpts(title="我要卖货"))
)
bar.render('%s.html' % self.good_msg) 第 7 步，配置参数。
编写 yaml 文件，指定要爬取商品的关键字、爬取时间、想要数考核指标数、筛选商品数目。
goods:
# 搜索商品1，包含搜索关键字、爬取时间
good1:
key_word: '资料' # 搜索关键字
key_num: 100 # 筛选【想要数】的临界点
num: 10 # 只筛选爆款
time: 600 # 爬取时间(秒) 4.结果结论
提前配置好商品关键字、爬取时间等参数，即可以爬取到符合要求的、最好卖的商品数据，最终以图表的方式展示出来。
