我下载不同的链接,我不控制下载哪些链接,所以我希望我的程序能够判断该链接之前是否已下载,如果已下载,则跳过该链接并继续当前的循环在。
while True:
print("--Sleeping for 30 seconds")
time.sleep(30)
submissions = r.get_subreddit(sub).get_random_submission()
print('--Submission Found in ' + sub)
print("--Submission.url: ", submissions.url)
if submissions.url in urlArray.urllist:
print("--This url has already been downloaded")
continue
print("--Adding url to array")
urlArray.urllist.append(submissions.url)
url = urllib.parse.unquote(submissions.url)
#if url doesnt have jpg and doesnt have "gallery" or "/a/"
if(not jpg in url and imgur in url and not "gallery" in url and not "/a/" in url):
print("--Url without jpg format: ", url)
jpgURL = submissions.url + ".jpg"
print("--Remade url is: ", jpgURL)
imgur_name2 = id_gen()
newurl2 = jpgURL.split('/')[-1].split('.')[0]
im.get_image(newurl2).download(path="C:\\Users\\KEVIN\\Pictures\\temp_pics", name=imgur_name2, overwrite=False, size=None)
print("--Downloaded remade url")
if jpg in url and imgur in url:
newurl = url.split('/')[-1].split('.')[0]
imgur_name = id_gen()
temp_pic = imgur_name + ".jpg"
print("--Downloading Image through Imgur")
im.get_image(newurl).download(path="C:\\Users\\KEVIN\\Pictures\\temp_pics", name=imgur_name, overwrite=False, size=None)
if reddit in url:
reddit_gen = id_gen()
reddit_pic = reddit_gen + ".jpg"
print("\n" + "--Downloading through I.Reddit")
request.urlretrieve(url, "C:\\Users\\KEVIN\\Pictures\\temp_pics\\" + reddit_pic)
if not reddit in url and not redd in url and not imgur in url and jpg in url:
print("--Not reddit or imgur but I can download the image")
randomGen = id_gen()
request.urlretrieve(url, "C:\\Users\\KEVIN\\Pictures\\temp_pics\\" + randomGen + ".jpg")
print("--Downloaded", "This image ID is: ", randomGen)
在这个循环中,我使用一个数组来处理 URL 重复,但正如您所知,一旦我重新启动程序,一切都会被删除。
将其保存为 csv。当我需要能够将信息从一个代码执行保存到另一个代码时,这就是我所做的。然后您可以在程序再次执行时加载 csv。
import csv
websites = ["www.example.com/1","www.example.com/2"]
with open('history.csv', 'w', newline='') as f:
writer = csv.writer(f)
writer.writerow(websites)
with open('history.csv', newline='') as f:
reader = csv.reader(f)
for row in reader:
print(row)
有关使用 csv 的更多详细信息,请参阅https://docs.python.org/3/library/csv.html。
本文收集自互联网,转载请注明来源。
如有侵权,请联系[email protected] 删除。
我来说两句