First , it may not be possible to get all elements directly through pyppeteer directly page.querySelectorAll() and then directly
str(await (await element_songer_name.getProperty("title")).jsonValue())
Get the data value, the main reason may be #document problem,
So when you encounter this problem, don’t try it with js right away, you can use the following code to solve it
frames = page.frames
# print(frames)
iframe = frames[1] # 第一个iframe frames[0]相当于是page.content() 整个页面的html 想要访问iframe的就要frames[0] [1]
# await iframe.click("#content-operation > a.u-btn2.u-btn2-2.u-btni-addply.f-fl")
elements_song_names = await iframe.querySelectorAll("td:nth-child(2) > div > div > div > span > a > b")
elements_songer_names = await iframe.querySelectorAll("td:nth-child(4) > div > span")Second, if you just want to use js to crawl, it is not impossible, in fact, it is very convenient, you can crawl directly, there is no problem with #document, but you have to think about how to get data suitable for "copying": directly print the array and you will
get Such a data format: not suitable for replication and reuse:
At this time, we have to change the printing method. First of all, the first method:
the data format obtained by console.table(song_names) is as follows:
It’s better to copy, and there are also single quotes, so it’s better to put it into the code for reuse.
The best way is to use console.log(JSON.stringify(song_names)) so that you can get the same result as python prints the array as follows:
And there is a copy button with a much better experience:
The resulting data format is:
["一笑懸命","パプリカ - (红辣椒)","ピリオドノック","星見る頃を過ぎても - (即使观星时刻已过)","YUBIKIRI-GENMAN","少女たちの終わらない夜","点描の唄 (ソロ ver.)","友だち - (好友)","何度も","二言目 - (TV动画《伪物语》OP1 / TVアニメ「偽物語」OP1テーマ)","それを愛と呼ぶなら - (倘若称之为爱)","EVERCHiLD","Blue","ハーフムーン・セレナーデ - ( 月半小夜曲)","Dear My Friend","大切な君へ (Acoustic ver.)","トリセツ - (电影《女主角失格》主题曲)","Ring of Fortune - (命运之环)","その声消えないよ - (音讯未绝)","杀破狼(Through The Fire Remix)","恋のテンキヨホウ (zerokoi ver.)","Little Sunshine","and I'm home - (エンディングキャラクターソング)","恋はキライだ - (讨厌恋爱)","夜空 - (TV动画《恋爱小行星》片尾曲)","Exodus (2016 Version)","Prayer X - (TV动画《战栗杀机》片尾曲 ； TVアニメ「BANANA FISH」EDテーマ)","愛唄~since 2007~","点描の唄 - (电影《青夏-与你相恋的30天-》插曲:映画「青夏 きみに恋した30日」挿入歌)","空の青さを知る人よ - (知晓天空之蓝的人啊)","恋の道","フルール","Ever Eternity","One Love","糸 - (线)","前前前世 [original ver.]","贈る言葉 - (电影《奔跑吧！T校篮球部》主题曲 / 映画『走れ!T校バスケット部』主題歌)","なみだ - (眼泪)","暗恋","爱呗","Cage - (「実物大ユニコーンガンダム立像」テーマソング)","散る散る満ちる - (散落飘零)","明日への手紙 - (致明日的信)","遇见 - (电影《向左走向右走》国语版插曲)","I Remember","贈る言葉 (ピアノ伴奏)","Who I Am - (日剧《七个秘书》片尾曲)","你是此生最美的风景","花びらたちのマーチ - (花瓣的游行)","光辉岁月(国语) (Live In Taiwan)","Forever Friends","大切な君へ","月灯り","六等星の夜 - (六等星之夜)","Crystalise","愛言葉 - (爱言叶)","Will (Tenkitsune x MLKSHK! Remix)","Into The Night","なきむし。 - (爱哭鬼。)","Nandemonaiya (English ver.) - (没什么大不了)","ひまわり - (向日葵)","今天你要嫁给我 - (Marry Me Today)","ツキアカリのミチシルベ","乙女どもよ。 - (少女们啊。)","もう少しだけ - (再多一点)","Summertime","来自天堂的魔鬼","Shining Friends","Rain (Long Ver.) -Extra Track-","いつかこの涙が","きみの て - (你的 手)","それが大事 - (最重要的事)","ただ声一つ（只想说一声）","リメンバー","Yesterday Once More","笑顔の花","君の知らない物語 - (你不知道的故事)","爱的双重魔力 - (电视剧《未婚妻》插曲)","ViEW","星星点灯","明日への手紙(ドラマバージョン)（2021 Remastering) - (写给明天的信)","summertime","HUMAN","さくら - (Sakura)","ツキアカリのミチシルベ - (月光的指引)","One Love","ひかりもの","aLIEz - (TV动画《ALDNOAH.ZERO》ED2:TVアニメ「アルドノア・ゼロ」ED2)","光るなら - (若能绽放光芒)","最後の花弁 (The meaning of love)","ツキアカリのミチシルベ - (TVアニメ「DARKER THAN BLACK-流星の双子-」OP1テーマ)","Love How It Hurts","君の知らない物語","月灯り (Instrumental)","After The Afterparty (feat. Lil Yachty) [Alan Walker Remix]","明日への扉 - (通往明天的门)","明日への扉 (オリジナル・カラオケ)","After The Afterparty (feat. Lil Yachty) [Alan Walker Remix]","BIRTHDAY","乙女どもよ。","私の言葉 - (我的话语)","シャンランラン feat.96猫 - (TV动画《飞翔的魔女》片头曲 )","A Letter","サマーブリーズにのって - (乘着夏日微风)","明日への扉","二言目 - (第二句话)","Fanfare of Adolescence - (群青开场号)","明日への扉 - (通往明天的门)","明日への扉 - (通往明天的门)","Crystalise","Amazing Trees -Extended Ver.-","A Brand New Day","何度も","マシュマリー (feat. 花里みのり&桃井愛莉&初音ミク) - (Marshmally)","Don't Turn Off The Lights","夜空。","夜空 - (TV动画《恋爱小行星》片尾曲)","What Makes You Beautiful","君の知らない物語 - (你不知道的故事)","月灯り - (月光)","月灯り","EGO [SODv] (feat. mizuki & SUGIZO)","rainfall","Amazing Trees","Lost - (电影《墨绘今生》插曲)","Crystalise","Cage - (「実物大ユニコーンガンダム立像」テーマソング)","gravityWall - (TV动画《Re:CREATORS》OP1)","ツキアカリのミチシルベ (TV size) - (月光的指引)","菫 - (TV动画《相合之物》片头曲)","Bad Apple!!","ベースラインやってる?笑 - (要来段bassline吗？笑)","回レ! 雪月花 - (旋转吧！雪月花)","ちいさな冒険者 - (小小冒险者)","おじゃま虫 - (麻烦鬼)","Drown - (TV动画《冰海战记》片尾曲2)","gravityWall - (TV动画《Re：创世主们》OP1 )","愛言葉 - (爱的话语)","点星（Point The Star） - (Point The Stars)","フリージア - (小苍兰)","知らないままで - (依然一无所知)","チキチキバンバン - (chikichikibanban)","冬の花 - (冬之花)","ただ声一つ - (只想说一声)","弱虫だって - (就算是胆小鬼)","Rubia - (崩坏3《渡尘》动画短片印象曲)","インドア系ならトラックメイカー - (内向都是作曲家)","Moon Halo - (崩坏3《薪炎永燃》动画短片印象曲)","Avid - (TV动画《86-不存在的战区-》ed)","心做し - (心理作用)","祝福 - (TV动画《机动战士高达：水星的魔女》片头曲)","忘れてもいいよ - (可以忘记了)","星が泳ぐ - (星海遨游)","Daisy - (TV动画《境界的彼方》片尾曲； TVアニメ「境界の彼方」EDテーマ)","ツギハギスタッカート - (拼凑的断音)","花の塔 - (花之塔 (TV动画《莉可丽丝》片尾曲))","STYX HELIX - (TV动画《Re:从零开始异世界生活》ED1)","灰色と青 - (灰色与青)","夜明けと蛍 - (夜阑与萤光)","打ち上げ花火","花の塔 - (花之塔)","Roads to Ride","LEVEL5 -judgelight-","パレード - (游行 Parade)","ほころび - (绽放)","それを愛と呼ぶなら - (若要称之为爱)","六等星の夜 - (六等星之夜)","アイシテ - (来爱我)","届かない恋 - (无法传达的恋情)","そばにいるね 青山テルマ feat. Soulja - (留在我身边)","素直になれたら - (若能更加直率)","最後の花弁 (The meaning of love) - (最后的花瓣（爱的意义）)","ツバメ (feat. ミドリーズ) - (燕子)","アオイハルカゼ - (青色春风)","どうして… - (どうして… - Doushite...)","最高の片想い - (弥足珍贵的单相思)","初恋 (TVアニメ「月がきれい」EDIT ver.) - (TV动画《月色真美》第3话插曲:TVアニメ「月がきれい」挿入歌)","Spirits","時雨 - (时雨)","あまねき祈り - (普通心愿)","ONE's hope","一瞬のクオリア - (一瞬间的感质)","きみの て - (你的手)","骄傲的少年 - (动画《那年那兔那些事儿》第二季片尾曲)","未だ知られぬ物語 ～ Anticipated Dream - (还未知晓的物语 ~ 悄然期待的梦)","モーメント","哀の隙間 - (哀的间隙)","オレンジ - (橘 Orange)","You Belong With Me (Taylor’s Version)","君のいない人生なんて ほんの少しも分からなくて","Love Story","さようなら、花泥棒さん (cover) - (再见了，盗花人)","花びら - (花瓣)","さようなら、花泥棒さん","アタック!ギャグマンガ日和 (Nazuka Ver.)","简单与初夏","ヒミツ (feat. 可不) - (秘密)","いっせーのーで - (一二三)","マシュマリー - (marsh mally)","くうになる (feat. 可不) - (放空)","After All","Rage Your Dream - (TV动画《头文字D》ED1)","約束 - (约定)","追梦赤子心 - (2013《快乐男声》主题曲)","走着走着花就开了","Something Just Like This(Megamix)","Run Free","A/Z <MODv>","だきしめるまで。 (feat. 可不) - (直到拥抱)","Brand New Day","You & I","Outerspace","Butterfly 数码宝贝主题曲 Piano ver.","夏の日 - (夏日)","誇り高きアイドル - (骄傲的偶像)","Treasure U","Wake (Studio)","BIRTHDAY","恋はキライだ - (讨厌恋爱)","コミュニケーション - (Communication)","Blue","ただ君に晴れ - (只为你拨云放晴)","RPG","ゆいいつむに - (独一无二)","halcyon","HANNARI","Creep","The Way I Still Love You","青春なんていらないわ - (无需青春)","summertime","Little Do You Know","Shotgun","Airwaves","First Date","Forgettable","Collapsing World - (崩溃的世界)","pure imagination - (纯粹的幻想)","Trial and Error (English Ver.) feat. Elira Pendora","Deja Vu（cover Uki Violeta）","水风船 - (ゲーム「翡翠の雫 緋色の欠片2」OPテーマ)","虚構の楽園 - (虚构的乐园)","ｻﾏｰﾌﾞﾘｰｽﾞにのって","雨のち想い出 - (雨后回忆)","散る散る満ちる - (散落飘零)","久遠 ~光と波の記憶~（Arrange Version） - (久远 ~光与波的记忆~（重编曲版）)","フルール - (花)","Inhibition","ひかりもの - (发光体)","WHITE ALBUM","星のパンフレット - (星星的小册子)","優しい彗星 - (温柔的彗星 (TV动画《BEASTARS》第二季片尾曲))","EXEC_COSMOFLIPS/.","ぼくらのレットイットビー - (让我们顺其自然)","マイライフ - (我的生活)","森林","Rubia","幾年月","夕日坂 - (夕阳下的坡道)","Sunshine Girl","泥だんごの天才いたよね - (有捏泥团子的天才哟)","だんご大家族 - (团子大家族)","Daylight - (日光)","Eutopia - (乌托邦)","色は匂へど 散りぬるを -Full Version- - (花朵艳丽 终会散落)","鳥の詩 - (鸟之诗)","手つかずの感情 - (未曾经历的感情)","「戦場ヶ原、蕩れ」","夜の向日葵 - (夜之向日葵)","雨爱 - (电视剧《海派甜心》片尾曲)","左边","銀の龍の背に乗って - (骑在银龙的背上)","銀の龍の背に乗って(TV MIX) - (骑在银龙的背上（Tv-Mix）)","ルージュ - (口红)","漫步人生路","やさしい女","漠河舞厅","青空","바람에 쓰는 편지 - (写在风中的信)","tutu - (芭蕾短裙)","水星","Feeling The Rain","星萤火","三葉のテーマ - (三叶的主题曲)","Ephemeral Memories - (转瞬即逝的记忆)","声","星茶会","DAYBREAK FRONTLINE","DAYBREAK FRONTLINE - (黎明前线)","メリーメリー álbum ver. - (merry merry)","Will","太陽と向日葵 - (太阳与向日葵)","螢塚 - (萤塚)","Adagio for Summer Wind - (夏风柔板)","神のまにまに - (神的随波逐流)","DAYBREAK FRONTLINE - (拂晓前线)","Into The Night - (夜に駆ける English Ver.)","約束の絆 - (约定的羁绊)","空の青さを知る人よ - (知晓天空之蓝的人啊)","蛍 - (萤)","and I'm home -Original Master Version.- - (TV动画《魔法少女小圆》第9集片尾曲)","Hesitate","1年2ヶ月20日 泣いちゃう前にver - (1年2个月20天)","カントリー・ロード - (Contry Road)","你是玫瑰","One Love - (电影《花样男子最终章》主题曲)","MY ALL","1时间","桜咲く - (樱花绽放)","なきむし。 - (爱哭鬼。)","なきむし。 Acoustic Ver. - (爱哭鬼。)","Home","glow","パズル - (拼图)","Letter Song - (致十年后的我)","Prayer X - (TV动画《战栗杀机》片尾曲)","春を告げる - (宣告春天)","何度も - (无数次)","Sunshine - (TV动画《滑头鬼之孙》OP2)","A Brand New Day","幻の命 - (虚无缥缈的生命)","世界は恋に落ちている - (我的世界已坠入爱河)","Diamond City Lights - (钻石之城的闪耀)","THERE IS A REASON - (剧场版《NO GAME NO LIFE 游戏人生 ZERO》主题曲)","unjust life","雪風 - (原曲:明日ハレの日、ケの昨日)","かわくだり","ヨワネハキ - (示弱)","my most precious treasure","光るなら - (若能绽放光芒)","My Soul - (忧伤还是快乐)","The Right Path","Old Memory","恋はヒラひらり - (爱在飘扬)","夏恋","いのちの名前 - (生命之名)","Snow Night","君想う、星屑の空。 - (想念你，夜空群星)","贈る言葉 - (电影《奔跑吧！T校篮球部》主题曲)","夏日漱石","君だったら - (如果是你)","友だち - (朋友)","Story","Slow & Easy","トリセツ - (电影《女主角失格》主题曲)","ただ、ありがとう - (只想说，谢谢妳)","始まりの唄 - (初始之歌)","drop pop candy","ロミオとシンデレラ - (罗密欧与辛德瑞拉)","その声消えないよ feat. Sunya - (你的声音不会消失)","Cage - (「実物大ユニコーンガンダム立像」テーマソング)","ここにいるよ - (我就在这里)","1年2ヶ月20日 - (1年2个月20天)","贝加尔湖畔","一生有你","离歌","Shelter","ハルジオン - (春紫菀)","彼女は旅に出る - (她踏上了旅途)","岁月神偷","侧脸","夢追人","何もかも有り余っている こんな時代も - (即便是这个所有一切都显得多余的时代)","No title - (无题)","終わりの世界から - (自终末世界)","mede:mede","アゲイン - (又一次)","In Love","Promise - (PS2游戏《寂静岭2》玛利亚结局片尾曲)","老人と海 - (老人与海)","Lanterns - (灯笼)","inside you - (日剧《丑闻专门律师QUEEN》片头曲:ドラマ『スキャンダル専門弁護士 QUEEN』オープニングテーマ)","Who I Am - (日剧《七个秘书》片尾曲)","もう少しだけ - (再多一点)","Close To The Sun","BLISS","Oblivion","Burning","Burning","狂おしいほど僕には美しい - (狂谬即美丽)","From Here","So High","そよ風 - (微风)","乙女どもよ。 - (少女们啊。)","ひまわり - (向日葵)","倒带","まちがいさがし - (错寻)","Journey","メグルユメ - (原曲:さくらさくら ～ Japanize Dream...)","勿念他归 - (《千本樱》钢琴版伴奏)","ヤキモチ - (吃醋)","妖々夢 - (妖々夢)","跟我在一起","Endless Tears ~終わらないメロディー~ feat. メロディー・チューバック","我的主题曲","Setsuna - (刹那)","思念是一种病","布拉格广场","PLANET","我的骄傲","我的骄傲","K歌之王 (国语版)","不期而遇的夏天 - (电影《不期而遇的夏天》主题曲)","K歌之王","稳稳的幸福 - (电影《越来越好之村晚》主题曲)","陪你度过漫长岁月 - (电影《陪安东尼度过漫长岁月》主题曲)","十年","爱情转移 - (电影《爱情呼叫转移》主题曲)","Runaway","雪解 (おてんば恋娘)","风神の调べ~优艶幻想曲集~","静寂の森(U.N.オーエンは彼女なのか? [东方红魔郷])","朝もお昼も夢の中も - (在清晨或者中午又或许在梦中)","NEXT TO YOU - (TV动画《寄生兽 生命的准则》配乐)","好きだよ。~100回の後悔~ (English Ver.)","Lupinus - (原曲:ネイティブフェイス)","Rather Be","Rather Be - (宁愿)","Burning","花とあれ - (花与你)","風の記憶 - (风的记忆)","Dragonfly Keeper","Strongest (Bogen Remix)","まちがいさがし - (找错游戏)","Stranger Things (Alan Walker Remix)","初夏的风","Dreamland","Fluctuate","南半球与北海道","Mardröm","Love How It Hurts","Apollo's Triumph - No Choir [Paul Dinletir Remix]","爱你(cover)","ねぇ - (呐)","明日への手紙(ドラマバージョン) - (写给明天的信)","ミカヅキ","내겐 그대뿐이죠 - (对我来说只有你)","キセキ - (奇迹)","Oblivion","覚醒 - (剧场版《PROMARE》主题曲)","The End of the World","なんでもないや (movie edit) - (没什么大不了)","スパークル [original ver.] - (火花)","Force","Believers","On My Way - (《和平精英》盛夏推广曲)","Alone, Pt. II","Diamond Heart","Different World","Lily","Lost Control","Sing Me to Sleep","Darkside","Alone","This Is Me (Alan Walker Relift) [From \"The Greatest Showman\"]","After the Afterparty (feat. Lil Yachty) [Alan Walker Remix]","New Silk Road - (新丝绸之路)","Rocky (Gonna Fly Now) - (洛奇)","Pirates Of The Caribbean (He's A Pirate) - (加勒比海盗)","Croatian Rhapsody - (克罗地亚狂想曲)","Exodus - (出埃及记)","リテラチュア - (旅途的华章)","April Showers (2018 Digital Remaster)","Various Artists-卡农【电音版】（Ysh remix）","しろつめくさ - (白车轴草)","Let's Run Away","深海","Into The Sun","一笑懸命 - (一笑悬命)","メロディー","キセキ~未来へ~ - (致以未来的足迹~)","恋のテンキヨホウ","ポケット - (《辛巴达：飞天公主与秘密之岛》剧场版主题曲)","あの頃~ジンジンバオヂュオニー~ - (那些年~紧紧抱着你~)","ハンブンコ","愛唄～since 2007～ - (电影《闪烁的爱情》主题曲)","卒業ソング","君想い","タンポポ","Look At Me Now","Here And Now","手紙 〜拝啓 十五の君へ〜 - (敬启—致十五岁的你)","Pillow talk","花びらたちのマーチ - (花瓣的游行)","Paprika","未来へ - (向着未来)","We'll Meet Again","道","哭泣站台","New Boy","Headlight -piano version-","Let Me Down Slowly","Yukiyanagi 雪柳~We're watching you~(Album version)","Headlight","Beautiful World","Seasons in the Sun","Lone Ranger","Boom Clap","Zombie","空も飛べるはず（Album Version） - (定能飞向天空)","ひとり上手 - (习惯孤独)","房间-新版 - (电影《超时空同居》插曲)","Lemon - (电视剧《非自然死亡》主题曲:ドラマ「アンナチュラル」主題歌)","Starfall - (崩坏3动画短片《天穹流星》印象曲)","なんでもないや/没什么大不了 -Acoustic ver.- - (没什么大不了)","あの夢をなぞって piano ver. - (描绘那场梦)","雨き声残響 - (雨声残响)","YUBIKIRI-GENMAN","夜行 - (Night Journey)","ONE","八年的爱","云烟成雨 - (动画《我是江小白》片尾曲)","I Love You (中文版)","Never Ending World - (ネバーエンディングワールド)","手纸 ~拝啓 绪五の君へ~ - (敬启~给十五岁的自己)","四季予你","海に行く - (去海边)","糸 - (线)","空の境","To the Lost and Forgotten","Un Año - (度日如年)","Estoy Aqui","ハローハロー - (你好你好 HelloHello)","stay with me","アイノカタチ - (MISIA提供楽曲セルフカバー)","爱呗","ひとりじゃない - (不再孤单前行)","ひとりじゃない ～繋がる想い～ present for 常盤木学園高等学校 - (Hitorijyanai ?Tsunagaruomoi? present for Tokiwaki Gakuen High School)","君に贈る詩 - (送给你的歌)","孟姜女","It's Ok","Don't Turn Off The Lights","这世界那么多人 - (电影《我要我们在一起》主题曲)","Blue Sky","M18","トモダチ/コイビト - (朋友/恋人)","Valder Fields","Sugar","越来越不懂","道","Why Would I Ever","Break Me Down","船上の音楽団 - (船上的音乐团)","Lucky","生きていたんだよな - (她曾活过啊)","追","红色高跟鞋 - (电影《爱情呼叫转移2》主题曲)","Fight","Dj Nanda V2 3","Forever Young","ロック风アレンジ","Dreamer","Love Is Gone (Acoustic)","Let Me Go","Tu vas me manquer (Pilule Bleue) - (我会想你（蓝色药片）)","Do Or Die","I'm Crazy in Love","星辰 (original Mix)","K-49","The Dawn（亡灵序曲）","All I Ever Wanted","sand dream","砂塵の彼方へ - (致沙丘的彼岸)","夜空。","Mine (Illenium Remix)","I wanted you","What Are You Waiting For?","I Said Hi","Félix - (牡丹花下)","Wonderful World","Stuttering","Enchanted","You Smile","The Sandman","Always Getting Over You","卒業メモリーズ〜サヨナラ、あなた。〜 - (毕业记忆 ～再见 亲爱的你)","Rage your dream - (燃烧你的梦想)","Hello friend(电音版)","What Makes You Beautiful - (是什么让你如此美丽)","恋の道 - (Koinomichi)","Can't Stop Love","無口な明日を嫌わないで","春風 SHUN PU","Keep You Safe","The Upside of Down","フロントメモリー - (电影《恋如雨止》主题曲:映画「恋は雨上がりのように」主題歌)","哈尼宝贝","theme of SSS","Honey Take My Hand","Life Is Beautiful","The Star","Raindrops","Shutterbug","Kachoufugetsu - (花鸟风月)","笑顔の花 - (笑靥如花)","だから僕は音楽を辞めた - (所以我放弃了音乐)","Summer Vibe","Brave Song - (TV动画《天使的心跳》ED1)","I Do","New Soul - (不谙世道)","The Show"]Therefore, it is also a good choice to use js to obtain data, but if it is a larger crawler, it is not very good to directly type js code in the browser console to obtain data (although we usually write dozens of lines in vscode first. The code is then copied to the console to run),
But for larger data crawlers, we still prefer to run them in the compiler (even after writing the js script project and packaging it into a browser script or plug-in before running it!)