• 大小: 7.65MB
    文件类型: .rar
    金币: 1
    下载: 0 次
    发布日期: 2023-11-03
  • 语言: Python
  • 标签: python  scrawl  

资源简介

使用python爬取某租房网站信息,然后利用高德API 进行位置显示、路径规划。本实验参考实验楼实验。

资源截图

代码片段和文件信息

import  csv
from bs4 import BeautifulSoup
import requests
url = “https://bj.58.com/pinpaigongyu/pn/{page}/?minprice=3000_4000“
page = 0
csv_file = open(“rent1.csv“‘w‘newline=““encoding=‘utf-8‘)
csv_writer = csv.writer(csv_filedelimiter=‘‘)
while True:
    page += 1
    print(“fetch: “url.format(page = page))
    response = requests.get(url.format(page=page))
    html = BeautifulSoup(response.textfeatures=“html.parser“)
    house_list = html.select(“.list > li“)

    if not house_list:
        break
    for house in house_list:
        house_title = house.select(“h2“)[0].string
        house_url = “https://bj.58.com/%s“%(house.select(“a“)[0][“href“])
        house_info_list = house_title.split()
        if “公寓“ in house_info_list[1] or “青年社区“ in house_info_list[1]:
            house_location = house_info_list[0]
        else:
            house_location = house_info_list[1]
            house_money = house.select(“.money“)[0].select(“b“)[0].string
            csv_writer.writerow([house_titlehouse_locationhouse_moneyhouse_url])

            # # csv_writer.writerow([house_title house_location house_url])
            # csv_writer.writerow([house_title house_location])
csv_file.close()

 属性            大小     日期    时间   名称
----------- ---------  ---------- -----  ----

     文件       2304  2018-11-02 20:18  RentHouse\.idea\codestyles\Project.xml

     文件      22283  2018-11-02 15:22  RentHouse\.idea\dbnavigator.xml

     文件        197  2018-11-02 15:23  RentHouse\.idea\misc.xml

     文件        270  2018-11-02 15:22  RentHouse\.idea\modules.xml

     文件        466  2018-11-02 15:23  RentHouse\.idea\RentHouse.iml

     文件      16219  2018-11-03 13:13  RentHouse\.idea\workspace.xml

     文件       1251  2018-11-03 10:21  RentHouse\venv\crawl.py

     文件       7735  2018-11-02 21:24  RentHouse\venv\index.html

     文件          4  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\beautifulsoup4-4.6.3.dist-info\INSTALLER

     文件       2995  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\beautifulsoup4-4.6.3.dist-info\metaDATA

     文件       2615  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\beautifulsoup4-4.6.3.dist-info\RECORD

     文件          4  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\beautifulsoup4-4.6.3.dist-info\top_level.txt

     文件         92  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\beautifulsoup4-4.6.3.dist-info\WHEEL

     文件      16688  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\bs4\builder\_html5lib.py

     文件      13197  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\bs4\builder\_htmlparser.py

     文件       9597  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\bs4\builder\_lxml.py

     文件      12230  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\bs4\builder\__init__.py

     文件      11545  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\bs4\builder\__pycache__\_html5lib.cpython-36.pyc

     文件       8815  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\bs4\builder\__pycache__\_htmlparser.cpython-36.pyc

     文件       7017  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\bs4\builder\__pycache__\_lxml.cpython-36.pyc

     文件       9075  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\bs4\builder\__pycache__\__init__.cpython-36.pyc

     文件      29917  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\bs4\dammit.py

     文件       7010  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\bs4\diagnose.py

     文件      72206  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\bs4\element.py

     文件      32468  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\bs4\testing.py

     文件       5582  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\bs4\tests\test_builder_registry.py

     文件       1067  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\bs4\tests\test_docs.py

     文件       4907  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\bs4\tests\test_html5lib.py

     文件       1728  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\bs4\tests\test_htmlparser.py

     文件       2645  2018-11-02 15:30  RentHouse\venv\Lib\site-packages\bs4\tests\test_lxml.py

............此处省略762个文件信息

评论

共有 条评论