** Background: ** Through this new learning to master the use of the following function.
Map (the lambda X: X + 3,5)
.startswith ( "ADS")
.strip () before and after removal space
a.format (1) with a filling in which the characters in {}
code show as below:
from lxml import etree
import requests
BASEURL="https://www.dytt8.net"
HEADERS={"User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.87 Safari/537.36"}
def get_details_url(url):
response = requests.get(url, headers=HEADERS)
text = response.content.decode('gbk','ignore')
html = etree.HTML(text)
detail_urls = html.xpath("//table[@class='tbspan']//a/@href")
detail_urls = map(lambda urlw: BASEURL + urlw, detail_urls)
return detail_urls
def parse_detail_page(url):
mie={}
response=requests.get(url,headers=HEADERS)
text = response.content.decode('gbk', 'ignore')
html = etree.HTML(text)
title=html.xpath("//div[@class='title_all']//font[@color='#07519a']/text()")[0]
mie["title"] = title
zoome=html.xpath("//div[@id='Zoom']")[0]
imgs=zoome.xpath(".//img/@src")
cover=imgs[0]
mie["cover"]=cover
infos=zoome.xpath(".//text()")
def parse_info(info,role):
return info.replace(role,"").strip()
for index,info in enumerate(infos):
if info.startswith("◎年 代"):
info=parse_info(info,"◎年 代")
mie['year']=info
elif info.startswith("◎产 地"):
info=parse_info(info,"◎产 地")
mie['make place'] = info
elif info.startswith("◎类 别"):
info=parse_info(info,"◎类 别")
mie['catagory'] = info
elif info.startswith("◎豆瓣评分"):
info=parse_info(info,"◎豆瓣评分")
mie['豆瓣评分'] = info
elif info.startswith("◎片 长"):
info=parse_info(info,"◎片 长")
mie['片 长'] = info
elif info.startswith("◎导 演"):
info = parse_info(info, "◎导 演")
mie['导 演'] = info
elif info.startswith("◎主 演"):
info = parse_info(info, "◎主 演")
actors=[info]
for x in range(index+1,len(infos)):
actor=infos[x].strip()
if actor.startswith("◎"):
break
actors.append(actor)
mie["actors"]=actors
elif info.startswith("◎简 介"):
info = parse_info(info, "◎简 介")
progfiles = []
for x in range(index +1,len(infos)):
progfile=infos[x].strip()
if progfile.startswith("◎获奖情况"):
break
progfiles.append(progfile)
mie["progfile"]=progfiles
return mie
def spidera():
headerurl="https://www.dytt8.net/html/gndy/dyzz/list_23_{}.html"
movies=[]
for x in range(1,4):
url=headerurl.format(x)
detail_urls=get_details_url(url)
for detail_url in detail_urls:
move=parse_detail_page(detail_url)
movies.append(move)
if __name__ == '__main__':
spidera()
Run Results:
The following is a partial results taken:
{ 'Title': '2019-year-old war "Battle of Midway" BD Guoying bilingual double-word', 'cover': 'https://extraimage.net/images/2020/01/11/c629673a2abc2f2d1f31d21f4c6e4d32.jpg', ' year ':' 2019 ',' make place ':' mainland China / US ',' catagory ':' Drama / history / war ',' watercress score ':' 7.8 / 10 from 87,337 users ',' film \ u3000 \ long U3000 ':' 138 minutes ',' turned \ u3000 \ u3000 play ':' Roland Emmerich Roland Emmerich ',' actors ': [' ed skrein Ed Skrein ',' Patrick Wilson Patrick Wilson ',' Woody Harrelson Woody Harrelson ',' Luke Evans Luke Evans ',' Dennis Quaid Dennis Quaid ',' Mandy Moore Mandy Moore ',' Luther Alexander Vegh alexander Ludwig ',' Aaron Eckhart Aaron Eckhart ',' Darren Criss Darren Criss ',' Nick Jonas Nick Jonas ',' Luke Klein tank Luke Kleintank ',' Jack Weber Jake Weber ',' Keen Johnson Keean Johnson ',' Tadanobu Asano Tadanobu Asano ',' David Hulett David Hewlett ',' Mark Rolston Mark Rolston ',' Brennan Brown Brennan Brown ',' James card Pei Neiluo James Carpinello ',' Matthew McCall Matthew MacCaull ',' Greg Huo Wani Xi'an Greg Hovanessian ',' Jack Manley Jake Manley ',' Jeffrey Blake Geoffrey Blake ', 'Rachel Pei Leier · Voss Kitt Rachael Perrell Fosket', 'Cameron Broad Cameron Brodeur', 'Jie Kebo Blair Jacob Blair', 'Christie Brooke Christie Brooke', 'country village Falcon jun Kunimura ',' Taylor Elliott Burke Tyler Elliot Burke ',' Feng Etsushi Etsushi Toyokawa ',' Yimei Bo-Valls Amber Walls', 'Monica Bika Luo baby Monika Bicarova', 'Tony Christopher Tony Christopher', 'Garrett Sato, Garret Sato', 'Robert Crooks Robert Crooks', 'Eric Davis Eric Davis', 'Thomas Vallier Thomas Vallieres' 'John Lobato John Lobato', 'Paradiso Ramos Diezel Ramos',' Kenny Leu '],' progfile ': [' movie tells the story of an important turning point in the Pacific War of World War II - the middle sea Warfare: after this incident, the Japanese navy by the "dimensionality reduction against" US-Japan maritime strength reversed, thus reversing the whole situation in the Pacific theater. Videos by war soldiers and pilots a fresh story, and gradually lead the audience into that field an incredible war in 1942 occurred in early June in the Pacific near Midway ...... ',' [Download] ',' ',' click the magnetic chain battle of Midway .BD.1080p Guoying bilingual double word .mkv ',' ',' ',' ',' ',' ',' ',' ',' ftp:. // ygdy8: ygdy8 @ yg18.dydytt.net:4003/ sunlight movie www.ygdy8.com. battle of Midway .BD.1080p. Guoying bilingual double word .mkv ',' ',' ',' ',' ',' ',' ' ]}, { 'title': '2019 biography Nian drama "Mrs. Lori and her son" BD British double subtitles',' cover ':' https: // extraimage. Lori is one of the greatest painters of the twentieth century British nationals, unique style loved by the British public. But his paintings in the early but was derided as an art critic from the hands of children, because his characters, big head, thin body, like a match stick, like a root, known as "Matchstick Men." Although Ms. Lori's mother Elizabeth, his life in the obstruction of his artistic career, but Lori was able to become a good painter, but had to be attributed to his mother. After his father's death, Lori takes care of the sick mother, she often wait until sleep began painting in the attic, wholeheartedly, fame or fortune. Melancholy always complaining Elizabeth remembered her dissatisfaction and disappointment with his son, Lori was all his life in pursuit of how to make the mother happy, his life and his paintings are longing for mother's identity. The film by the former director Adrian renowned Royal Shakespeare Theater. Noble as director, actor Timothy got Cannes Secretary Bo played the great painter, and Cannes actress Vanessa Redgrave plays the mother of a control freak. Two superb acting, his live band but this satire humor mother and child. ',' 'Chimneys and black smoke in the sky, matches man appeared in the industrial landscape in static traveled alone. "Lori's work documents the industrialization of the British urban landscape, and the people of alienation, until now, still resonates. He was enlisted Royal Society of Arts and Manchester College in later years, once work sold for 3.7 million pounds. He repeatedly rejected British Prime Minister medal awarded, and rejected the title of Lord. '' [Download] ',' ',' magnetic chain Click wife Lori and her son .BD.1080p. Sino-British dual subtitles .mkv ',' ',' ',' ',' ',' ' , '', '', 'ftp: // ygdy8: [email protected]:.. 3002 / sunshine movie www.ygdy8.com wife Lori and her son .BD.1080p Sino-British dual subtitles .mkv ',' ',' ',' ',' ',' ',' ']}, {' title ':' 2019 years Drama biography "traitor / black gold traitor" BD word ',' cover ':' https: / /extraimage.net/images/2020/02/04/aba90630040057fc651a1531272628ed.jpg ',' year ':' 2019 ', 'Make place': 'Italy, France, Brazil, Germany', 'catagory': 'Drama / Biography / Crime', 'watercress score': '7.3 / 10 from 3,820 users', 'film \ u3000 \ u3000 long': '135 minutes',' guide \ u3000 \ u3000 play ':' Mark Bello Keogh Marco Bellocchio ',' actors': [ 'Pierre Fran Qiesi Ke-Fa Weinuo Pierfrancesco Favino', 'Ma Liya Fernanda Conkey multi Maria Fernanda Cândido ',' Luigi Lockheed Cascio Luigi Lo Cascio ',' Fausto Russo Alesi Fausto Russo Alesi ',' Nino Fazio Boer nino Porzio ',' Alessio Praticò ',' Gabriele Arena ',' Fabrizio Ferracane ',' Jacopo Garfagnoli ',' Patrick Simons', 'Marco Gambino', 'Calì Nicola', 'Goffredo Maria Bruno', 'Massimiliano Ubaldi', 'Marilina Marino', 'Nunzia Lo Presti'], 'progfile': [ 'according to the Italian film adapted from real events. In the early 1980s, the gang leaders Duo Masuo-Bu Xida emigrated from Italy to Brazil, but his brother and children as far away as Italy, but have been killed gangs. In order to punish the murderer, he started working with the Italian police, in the continuing efforts of the two parties, Duo Masuo-Bu Xida not only captures the killing of their loved ones behind, and Italy's deep-rooted gang uprooted ...... '] }, { 'title': '2019 years action thriller "Grand Island" BD dual subtitles in English', 'cover': 'https://extraimage.net/images/2020/01/29/ba8f6fdb69026c2911060c0a1608cb4d.jpg' , 'year': '2019', 'make place': 'US', 'catagory': 'action / thriller', 'watercress Rating': '4.4 / 10 from 136 users',' sheet \ u3000 \ u3000 length ': '97 minutes' , 'turned \ u3000 \ u3000 play': '· S · Kanpaneili Stephen Stephen S. Campanelli', 'actors': [ 'Nicholas Cage Nicolas Cage', 'Kai Erxi · Ge Lanmo Kelsey Grammer' 'North Entering · Hengao Zulay Henao', 'Luke Ben Ward Luke Benward', 'KaDee Strickland KaDee Strickland', 'Mary Emily Palmer Emily Marie Palmer', 'Oliver Trey Weiner Oliver Trevena', 'Beatrice Hernandez Beatrice Hernandez', 'Isabella Grace Roark', 'Haley Milsap'], 'progfile': [ 'a young father was the bizarre series of terrorist events framed for the murder criminals who, in order to prove his innocence, he must have experienced in their own to find clues and answers. '' Stephen S. Campanelli and-coming film director jointly directed by Iver William Jallah has directed "Christmas theft" and repeatedly collaboration with director Clint Eastwood, Nicolas Cage announced to join the sheet. '' [Download] ',' ',' Click on a magnetic chain Grand Island .BD.1080p. Sino-British dual subtitles .mkv ',' ',' ',' ',' ',' ',' ' , '', 'ftp: // ygdy8: [email protected]:.. 6006 / sunshine movie www.ygdy8.com Grand Island .BD.1080p British double subtitles .mkv', '', ' ',' ',' ',' ',' ']}, {' title ':' 2019 Nian thriller "Grace fall" BD British double subtitles ',' cover ':' https: // lookimg. com / images / 2020/02/06 / aQjFj.jpg ',' year ':' 2020 ',' make place ':' American ',' catagory ':' Thriller ',' watercress score ':' 6.3 / 10 from 841 users ',' sheet \ u3000 \ u3000 length ':' 120 minutes ',' turned \ u3000 \ u3000 play ':' Tyler Perry Tyler Perry ',' actors ': [' Adrian Pa Sida Adrian Pasdar ',' Tyler Perry Tyler Perry ',' Xixi Li Tyson Cicely Tyson ',' Felicia Russ Hyde Phylicia Rashad ',' Bu Laisuo Weber Bresha Webb ',' Maika De Brooks Mehcad Brooks ',' Sandra Ellis Lafferty Sandra Ellis Lafferty ',' Nathan Anderson Nathan Anderson ',' Michael Rose Michael Rose ',' Crystal Fox ',' Walter Fauntleroy ',' James William Ballard ',' Matthew Law ',' James Healy Jr. ',' Frances Mitchell '],' progfile ': [' after the film tells Grace Walters experienced ex-husband derailment of hurt feelings, and finally re relying on new romance stand up. Who knows possession of numerous secret under a bright appearance, Grace after discovering the truth, physical or mental harm, she began to fight back. '' [Download] ',' ',' Click on a magnetic chain of Grace fall .BD.1080p. Sino-British dual subtitles .mkv ',' ',' ',' ',' ',' ',' ' , '', 'ftp: // ygdy8: [email protected]:.. 4002 / sunshine movie www.ygdy8.com Grace fall .BD.1080p British double subtitles .mkv', '', ' ',' ',' ',' ',' ']}, {' title ': '2019 action thriller "21 Bridge / Manhattan runaway" BD British double subtitles', 'cover': 'https://lookimg.com/images/2020/02/06/aQtKo.jpg', 'year': '2019', 'make place': 'American', 'catagory': 'action / thriller / crime', 'watercress score': '6.5 / 10 from 1,158 users', 'film \ u3000 \ u3000 long': '99 minutes' 'guide \ u3000 \ u3000 play': 'Brian Kirk Brian Kirk', 'actors': [' Chadwick Boseman Chadwick Boseman ',' Sienna Miller Sienna Miller ' 'Taylor Kitsch Taylor Kitsch', 'J · K · Simmons JK Simmons',' Keith David Keith David ',' Stephen James Stephan James', 'Katie Katie McClellan McClellan '' Sha Yina Ryan Shayna Ryan ',' Gary Carr Gary Carr ',' Victoria Cartagena Victoria Cartagena ',' Dale Pa Stravinsky Dale Pavinski ',' Toby Hemingway Toby Hemingway '' Christian Isaiah Kristian Isaiah ',' Luis Kan Semi Louis Cancelmi ',' Jon Douglas Rainey Jon Douglas Rainey ',' Gerald Wade Jared Ward ',' Morlock · Omari Morocco Omari ',' Suzette Gunn Suzette Gun · n ',' Michael · J · Ke Leimi Michael J. Kraycik ',' · J · Michael Lyons Michael J. Lyons'], 'progfile': [ 'Film style similar alleged "insider" and "The Fugitive", Bosman decorated with a disgraced New York police detective, because they were involved in a city-wide hunt for the killer of a police action, he won the atonement opportunities - he found a huge conspiracy together: his police colleagues may be associated with a criminal empire. He must decide in the end who want to hunt, but who in fact turn in the hunt for him. Since this action, Manhattan first time in history the whole territory of the blockade, 17 bridges on the island with the outside world out of a total ban. '' [Download] ',' ',' Click on a magnetic chain 21 bridge .BD.1080p. Sino-British dual subtitles .mkv ',' ',' ',' ',' ',' ',' ', '', 'ftp: // ygdy8: [email protected]:. 7053 / sunshine movie www.ygdy8.com.21 bridge .BD.1080p Sino-British dual subtitles .mkv', '', '', '', '', '', '']},
