riku
2024-01-10 a9e8e27e0503552b7b2a99c821da732175d4f071
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
U
ù#;e³-ã@s¾ddlmZddlmZmZmZmZmZmZm    Z    ddl
m Z Gdd„dej ƒZ Gdd„dej ƒZGdd    „d    ej ƒZGd
d „d ej ƒZGd d „d ej ƒZGdd„dej ƒZGdd„dej ƒZdS)é)Údb_conn)ÚColumnÚIntegerÚStringÚDATETIMEÚDOUBLEÚBOOLEANÚTEXT)Ú    DateUtilsc@speZdZdZdZeeddddZeedƒddZ    eed    ƒd
dZ
eed ƒd dZ ee d dZ eed    ƒddZdS)Ú    LoginInfou登录信息表Zdu_js_t_login_infoTõ主键©Ú primary_keyÚ autoincrementÚdocé2u 登录账号©réÿu 登录密码iôZcookiesucookie时间戳u区域N)Ú__name__Ú
__module__Ú __qualname__Ú__doc__Ú __tablename__rrÚidrZ    user_nameÚpasswordÚcookierZcookie_timestampÚregion©rrú;e:\PycharmProject\fugitive_dust_new\src\db\models\tables.pyr s r c@seZdZdZdZeeddddZeedƒddZ    ee
d    dZ eedƒd
dZ eed ƒd dZ ee
d dZee
ddZeeddZeeddZeeddZeeddZeeddZeeddZee
ddZee
ddZeeddZeeddZeeddZeeddZdS)ÚRequestTaskSettingu爬取任务配置参数表Zdu_js_t_request_task_settingTr r ru 用户名称ru配置更新时间u区县é
u    ç‰ˆæœ¬å·u数据爬取开始时间u数据爬取结束时间u6在爬取任务开始结束时间内,爬取的频率uK一次数据爬取任务中,按总时长分段执行的每段时间长度uE一次数据爬取任务中,分段执行时的每段的间隔时间u'爬取任务异常时再次尝试间隔u'爬取失败时再次尝试的总次数ucookie存储有效时长Úu*单次登陆失败重新尝试等待时间u登陆失败连续尝试次数u0整个爬取任务重新开始尝试等待时间us程序首次执行, ç«™ç‚¹æ•°æ®æœ€æ–°æ—¶åˆ»è¡¨ä¸ºç©ºæ—¶, æ•°æ®èŽ·å–å¼€å§‹æ—¶é—´ç›¸å¯¹å½“æ—¥å¾€å‰åç§»çš„å¤©æ•°N)rrrrrrrrrÚuserrÚ update_timerÚversionZrequest_start_timeZrequest_end_timeZrequest_interval_secondsZrequest_range_hourZrequest_range_interval_secondsZrequest_fail_wait_secondsZrequest_retry_timesZrequest_cookie_valid_durationZstatics_daily_timeZstatics_monthly_timeZlogin_fail_wait_secondsZlogin_retry_timesZtask_retry_wait_secondsZfirst_request_offset_daysrrrrrs*              rc@s eZdZdZdZeeddddZeedƒddZ    ee
d    dZ ee
d
dZ ee d dZeed ƒd dZeed dZeed ƒddZddœdd„Zedœdd„ZdS)Ú DustDataInfou监测数据表Zja_t_dust_site_data_infoTr r rõ 站点编号ru扬尘监测值u噪声监测值u 监测时间r u 监测质量u 状态标识N©ÚreturncCs^| d¡|_| d¡|_| d¡|_t | d¡¡|_| d¡|_| d¡|_| d¡|_    dS)    uÿ
        Args:
            json_obj: {
                "longitude": "121.32",
                "latitude": "30.840834",
                "Address": "西新泾黄汶泾",
                "DustValue": 0.207,
                "Grade": 1,
                "GroupID": 16,
                "GroupName": "金山区",
                "LST": "2023-10-16 12:00:00",
                "LST1": "1697457600",
                "Name": "大信.中信海直华东无人机总部基地项目——1号点",
                "MNCode": "SHXH0JS0100306",
                "NoiseValue": 59.4,
                "ProjectID": 45879,
                "Quality": "好",
                "SName": "上海小慧",
                "TypeName": "建筑工地",
                "flag": "N"
            }
        ÚMNCodeZ    DustValueZ
NoiseValueZLSTZQualityZGradeÚflagN)
ÚgetÚmn_codeÚ
dust_valueÚ noise_valuer
Ú str_to_timeÚlstÚqualityÚgrader*©ÚselfZjson_objrrrÚ__init__:s     zDustDataInfo.__init__cCstdd„|ƒ}t|ƒS)NcSst|ƒS)N)r%)ÚxrrrÚ<lambda>Zóz*DustDataInfo.json_to_cls.<locals>.<lambda>)ÚmapÚlist)Z    json_listÚmrrrÚ json_to_clsYszDustDataInfo.json_to_cls)rrrrrrrrrr,rr-r.rr0r1r2r*r5r:r<rrrrr%,s    r%c@s>eZdZdZdZeedƒdddZeeddZ    ee
d    dZ d
S) ÚSiteLatestTimeu站点数据最新时刻表Zdu_js_t_site_latest_timerTr&)rru 最新时间ru7站点状态, 0代表上线;1代表停运;2代表下线N) rrrrrrrr,rZ latest_timerZ device_statusrrrrr=^s
 r=c@sòeZdZdZdZeeddddZeedƒddZ    eed    ƒd
dZ
eed    ƒd dZ eedƒd dZ ee d dZeed    ƒd dZeedƒd dZee d dZeedƒd dZeed ƒd dZeedƒd dZeed ƒd dZeed ƒd dZeedƒd dZeedƒd dZeedƒd dZeedƒd dZeedƒd dZeedƒd dZeedƒd dZeedƒd dZeedƒd dZeedƒd dZ eedƒd dZ!ee d dZ"eed ƒd dZ#eed dZ$ee d dZ%ee d dZ&ee'd dZ(eedƒd dZ)eedƒd dZ*ee d dZ+eedƒd dZ,eed    ƒd dZ-eedƒd dZ.eedƒd dZ/ee d dZ0eedƒd dZ1ee d dZ2eedƒd dZ3eedƒd dZ4eedƒd dZ5eed ƒd dZ6eed    ƒd dZ7eedƒd dZ8ddœdd„Z9dd„Z:dS)Ú DustSiteInfou站点基本信息表Zja_t_dust_site_infoTu 项目编号r ru 设备编号rru 安装地址u 站点名称r!r éédNr'cCs| |¡dS)uJ
        Args:
            json_obj: {
                "Active": 1,
                "Address": "上海市金山区亭林镇周栅村荡新7组1056号",
                "Code": "HMHB00087",
                "DustValue": 0.065,
                "DutyCompany": "上海泓旻环保科技有限公司",
                "DutyCompanyID": 25,
                "GroupID": 16,
                "GroupName": "金山区",
                "ID": 41024,
                "IsOnline": 1,
                "Name": "上海阜阜建材有限公司",
                "IsTrouble": 0,
                "LST": "2023-10-27 08:15:00",
                "RingName": "外环外",
                "MNCode": "HMHB0JS0300084",
                "TypeName": "码头",
                "TypeID": 3,
                "NoiseValue": "68.9",
                "RingID": 4
            }
        N)Úupdater3rrrr5szDustSiteInfo.__init__cCs| d¡|_| d¡|_| d¡|_| d¡|_| d¡|_| d¡|_| d¡|_| d¡|_| d    ¡|_    | d
¡|_
| d ¡|_ | d ¡|_ | d ¡|_ | d¡|_| d¡|_| d¡|_| d¡|_d|_| d¡|_| d¡|_| d¡|_| d¡|_| d¡|_| d¡|_dS)NÚIDr)ZAddressÚNameZCodeZ    BeginDateZ DutyCompanyZ DutyCompanyIDZEndDateZEngineeringStageZGroupIDZ    GroupNameZIsOnlineZ    IsTroubleZKIndexZLinkmanZPhoneu    ä¸Šæµ·å¸‚ZRingIDZRingNameZTypeIDZTypeNameZActiveZ
TroubleNum)r+rr,ÚaddressÚnameÚcodeÚ
begin_dateÚ duty_companyÚduty_company_idÚend_dateÚengineering_stageÚgroup_idÚ
group_nameÚ    is_onlineÚ
is_troubleÚkindexÚlinkmanÚphoneÚprovinceÚring_idÚ    ring_nameÚtype_idÚtypenameÚactiveÚ trouble_numr3rrrrA¸s0                      zDustSiteInfo.update);rrrrrrrrrr,rDrErFrrGrHrIrJrKrLrMrNrOZjhpt_update_timerPZlatituderQZ    longituderRrSrTrUrVrWZ    stop_timerXrYZ insert_timeZstage_begin_daterZtspZ union_type_idZ wall_heightZ    ywsj_dateZ
build_areaZconstruction_unitZ control_levelZ
developersZdo_timeZ has_monitorZ    data_timeZengineering_stage_codeZequipment_codeZ
floor_areaZ jhpt_deleteZ noise_regionZ responsibler5rArrrrr>hsf          r>c@sbeZdZdZdZeeddddZeeddZ    eeddZ
ee ddZ ee ddZ dd    œd
d „ZdS) Ú RequestTasku爬取数据任务表Zdu_js_t_request_taskTõ 任务编号r r!rNr'cCs||_||_d|_d|_dS)NTF)rÚ
start_timeÚrunning_statusÚsuccess)r4rr\rrrr5ôszRequestTask.__init__)rrrrrrrrrr\Úend_timerr]r^r5rrrrrZés    rZc@sXeZdZdZdZeeddddZeeddZee    ddZ
ee d    ƒddZ ee ddZd
S) ÚRequestTaskLogu爬取数据任务日志表Zdu_js_t_request_task_logTu 日志编号r r[rr!r?N)rrrrrrrrZ
request_idrZ create_timerZlog_typer    Úlogrrrrr`ûs  r`N)Ú
db.db_connrÚ
sqlalchemyrrrrrrr    Zutils.date_utilsr
ÚBaser rr%r=r>rZr`rrrrÚ<module>s $  2